Skip to content

Commit

Permalink
[NeMo-UX] Minor bug fix when TE/Apex not installed (#9749)
Browse files Browse the repository at this point in the history
* minor 2.0 bug fix when TE/Apex not installed

Signed-off-by: ashors1 <ashors@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: ashors1 <ashors1@users.noreply.github.com>

---------

Signed-off-by: ashors1 <ashors@nvidia.com>
Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
Co-authored-by: ashors1 <ashors1@users.noreply.github.com>
  • Loading branch information
ashors1 and ashors1 committed Jul 16, 2024
1 parent f2e3232 commit 34bfe1b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion nemo/lightning/megatron_parallel.py
Original file line number Diff line number Diff line change
Expand Up @@ -445,8 +445,8 @@ def infer_num_microbatches(self, data: Union[DataT, Iterator[DataT], List[Iterat
raise ValueError("Cannot infer `num_microbatches` from data, please specify it manually")

def init_model_parallel(self):
from apex.transformer.tensor_parallel.layers import set_defaults_if_not_set_tensor_model_parallel_attributes
from megatron.core import parallel_state
from megatron.core.tensor_parallel.layers import set_defaults_if_not_set_tensor_model_parallel_attributes

for model_module in self:
if not self._cpu:
Expand Down

0 comments on commit 34bfe1b

Please sign in to comment.