Skip to content

Commit 2ecc3ad

Browse files
iamzainhudafacebook-github-bot
authored andcommitted
add DTensor to VLE and fix 2D sharding group in EBC (#2626)
Summary: Adding DTensor state dict to VLE, this is identical to EBC DTensor path. Future work of this diff is to consolidate the logic of EBC/PEA/VLE state dict into one parent class because of their significant similarities A revert diff removed the 2D sharding logic in embedding bag collection mistakenly: D66800554, this diff adds it back in Differential Revision: D65555595
1 parent 33349ec commit 2ecc3ad

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

torchrec/distributed/embeddingbag.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
QuantizedCommCodecs,
6666
ShardedTensor,
6767
ShardingEnv,
68+
ShardingEnv2D,
6869
ShardingType,
6970
ShardMetadata,
7071
)
@@ -938,7 +939,11 @@ def _initialize_torch_state(self) -> None: # noqa
938939
ShardedTensor._init_from_local_shards(
939940
local_shards,
940941
self._name_to_table_size[table_name],
941-
process_group=self._env.process_group,
942+
process_group=(
943+
self._env.sharding_pg
944+
if isinstance(self._env, ShardingEnv2D)
945+
else self._env.process_group
946+
),
942947
)
943948
)
944949

0 commit comments

Comments
 (0)