use torch scaled_dot_product_attention #1

…ter restoring from a checkpoint (NVIDIA#10225) Signed-off-by: ashors1 <ashors@nvidia.com>

* Update TRTLLM 0.12 * Add model config * Change config * Change deploy script * Apply isort and black reformatting Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> * Remove parameter --------- Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> Co-authored-by: meatybobby <meatybobby@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com>

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

…VIDIA#10234) Signed-off-by: Hemil Desai <hemild@nvidia.com>

…nd offsets in manifest (NVIDIA#10198) * Add tests for LazyNeMoIterator and fix case with manifest_only=True and offsets in manifest Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Address code review Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com>

…ckpoints (NVIDIA#9939) * perfor serialization using relative paths to allow users to move checkpoints after they're saved Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> * fix artifact load Signed-off-by: ashors1 <ashors@nvidia.com> * fix path artifact Signed-off-by: ashors1 <ashors@nvidia.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com>

* Add MemoryProfileCallback Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Apply isort and black reformatting Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> * Remove reference cycles, save snapshot on specific ranks Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Remove unnecessary imports Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Apply isort and black reformatting Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> * Update docstring Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> --------- Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Shriya Rishab <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com>

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Dong Hyuk Chang <donghyukc@nvidia.com>

…rocessing (NVIDIA#10052) Flow matching generative model with SSL pretraining framework Signed-off-by: Pin-Jui Ku <pku@nvidia.com> Co-authored-by: Kuray107 <Kuray107@users.noreply.github.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* Move nemotron transformers + tokenizer imports inline to reduce number of required deps Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> --------- Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> Co-authored-by: marcromeyn <marcromeyn@users.noreply.github.com>

* Wrap CPU model init with megatron_lazy_init_context Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Cleanup checkpoint-dir if saving fails Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: titu1994 <titu1994@users.noreply.github.com>

* [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 124bcff ! Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix bert flags Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Oliver Koenig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: slyne deng <slyned@nvidia.com> Co-authored-by: slyne deng <slyned@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

…0267) * Load model in the target export precision by default Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Enable megatron_amp_O2=true to actually use half-precision Signed-off-by: Jan Lasek <jlasek@nvidia.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Jan Lasek <jlasek@nvidia.com>

…n.plugins (NVIDIA#10223) * Add WandbPlugin, NsysPlugin and PreemptionPlugin to nemo.lightning.run.plugins Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Remove duplicate Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add entity to wandb logger Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add documentation Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Add warning Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Add comments Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com>

…0259) * handle absolute and relative logger directories Signed-off-by: Anna Shors <ashors@nvidia.com> * merge lines Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: Anna Shors <ashors@nvidia.com> Signed-off-by: ashors1 <ashors@nvidia.com>

* Add sdxl notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * Rename Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * final Update SDXL notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> --------- Signed-off-by: mingyuanm <mingyuanm@nvidia.com>

* add llama31 config * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * fix init method * typo * revert llama3-70b init method std --------- Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> Co-authored-by: suiyoubi <suiyoubi@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com>

Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos02.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos02.eos.clusters.nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com>

* add qwen2 * typo * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * qwen without pip install issue * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * remove calculate vocab size divisible --------- Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> Co-authored-by: suiyoubi <suiyoubi@users.noreply.github.com>

* Move inflect to lazy import Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use lazy imports for tokenizer libraries Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * sacremoses lazy import Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix cyclic import Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * import fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * move pangu Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

* create documentation for rampup bs * fix format * fix format * fix config format * move config stage * add example * fix table * fix table * fix grammar * fix grammar --------- Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Add sc1/sc2 to nemo-ux * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * typo * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * fix import * remove pip install dependency for sc * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * remove pip install dependency for sc2 * Apply isort and black reformatting Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> * typo Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> --------- Signed-off-by: suiyoubi <suiyoubi@users.noreply.github.com> Signed-off-by: Ao Tang <aot@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> Co-authored-by: suiyoubi <suiyoubi@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: cuichenx <cuichenx@users.noreply.github.com>

…DIA#10293) Signed-off-by: Pablo Garay <pagaray@nvidia.com>

…ction across ALL processes (NVIDIA#10261) * Adding Garbage-collection callback for a synchrozned gc across processed Signed-off-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> * 1 Signed-off-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> * Apply isort and black reformatting Signed-off-by: gautham-kollu <gautham-kollu@users.noreply.github.com> Signed-off-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> * PR feedback Signed-off-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> --------- Signed-off-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> Signed-off-by: gautham-kollu <gautham-kollu@users.noreply.github.com> Co-authored-by: Gautham Kollu <gkollu@login-eos01.eos.clusters.nvidia.com> Co-authored-by: gautham-kollu <gautham-kollu@users.noreply.github.com>

* Do not overwrite wandb name in NeMo Logger Signed-off-by: Hemil Desai <hemild@nvidia.com> * Do not overwrite tensorboard name Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fix tests Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com>

* [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 34e607e ! Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * refactor: Deprecate `async_grad_allreduce` Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Oliver Koenig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* salm export trtllm Signed-off-by: slyne deng <slyned@nvidia.com> * add export script Signed-off-by: slyne deng <slyned@nvidia.com> * fix style Signed-off-by: slyne deng <slyned@nvidia.com> * Apply isort and black reformatting Signed-off-by: Slyne <Slyne@users.noreply.github.com> --------- Signed-off-by: slyne deng <slyned@nvidia.com> Signed-off-by: Slyne <Slyne@users.noreply.github.com> Co-authored-by: slyne deng <slyned@nvidia.com> Co-authored-by: Slyne <Slyne@users.noreply.github.com>

…10107) * [TTS] Add config and modules for 22khz and 44khz audio codec Signed-off-by: Ryan <rlangman@nvidia.com> * Apply isort and black reformatting Signed-off-by: rlangman <rlangman@users.noreply.github.com> * [TTS] Add argument docstring to new modules Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: rlangman <rlangman@users.noreply.github.com> Co-authored-by: rlangman <rlangman@users.noreply.github.com>

…VIDIA#10226) * Add example script to run NeMo 2.0 llama pretraining with NeMo-Run Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fixes Signed-off-by: Hemil Desai <hemild@nvidia.com> * Address PR comments Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com>

* modify code structure and add strategy Signed-off-by: jasonwan <jasonwan@nvidia.com> * correct doc url Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * oextract common elements and add callback Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * add iomixin Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * update strategies Signed-off-by: jasonwan <jasonwan@nvidia.com> * update callback Signed-off-by: jasonwan <jasonwan@nvidia.com> * add training step to strategy Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * remove unused import Signed-off-by: jasonwan <jasonwan@nvidia.com> * add iomixin to strategy & precision Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: artbataev <artbataev@users.noreply.github.com> * add val/test steps to strategy Signed-off-by: jasonwan <jasonwan@nvidia.com> * add documentations Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * add default sharding for fsdp. add setup callback detection Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * extract checkpoint io logic Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * clean up unused imports Signed-off-by: jasonwan <jasonwan@nvidia.com> * sync new megatron strategy changes Signed-off-by: jasonwan <jasonwan@nvidia.com> * break down setup callback back into strategy Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * reorder stuff Signed-off-by: jasonwan <jasonwan@nvidia.com> * fix data logic Signed-off-by: jasonwan <jasonwan@nvidia.com> * minor fix Signed-off-by: jasonwan <jasonwan@nvidia.com> * add dtensor ckpt conversion support Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * support hsdp Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * remove iomixin Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * fix import Signed-off-by: jasonwan <jasonwan@nvidia.com> * refactor loss reduction Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * clean up Signed-off-by: jasonwan <jasonwan@nvidia.com> * add unittest Signed-off-by: jasonwan <jasonwan@nvidia.com> * Apply isort and black reformatting Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> * clean up Signed-off-by: jasonwan <jasonwan@nvidia.com> --------- Signed-off-by: jasonwan <jasonwan@nvidia.com> Signed-off-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> Signed-off-by: artbataev <artbataev@users.noreply.github.com> Co-authored-by: blahBlahhhJ <blahBlahhhJ@users.noreply.github.com> Co-authored-by: artbataev <artbataev@users.noreply.github.com>

* initial commit Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * PR draft Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * fixed scaling weights Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * fixed zarr loading, added flags, refactor Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> * fix expert key mapping Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * refactor Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> * fix: failed test was finishing with exit code 0 Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * test commit -- rerun github checks Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * bugfix: naming Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * bugfix v2: naming Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * apply code review changes Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> * fix TensorRTLLM build (fp8 still not supported) Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> * undo refactor Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * bugfix: arguments to dist_convert Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> --------- Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> Signed-off-by: Piotr Kamiński <67481570+Laplasjan107@users.noreply.github.com> Co-authored-by: Piotr Kaminski <pikaminski@nvidia.com> Co-authored-by: Laplasjan107 <Laplasjan107@users.noreply.github.com>

* bugfix: loading scaling factors Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * list instead of set Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> --------- Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> Co-authored-by: Piotr Kaminski <pikaminski@nvidia.com>

* Added sanity checks Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> * Apply isort and black reformatting Signed-off-by: jbieniusiewi <jbieniusiewi@users.noreply.github.com> * Updated error msg Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> * Added checks for NeMo 2.0 Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> * Updated NeMo 2.0 test Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> * Apply isort and black reformatting Signed-off-by: jbieniusiewi <jbieniusiewi@users.noreply.github.com> * fixed test_nemo_logger.py Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> --------- Signed-off-by: Jacek Bieniusiewicz <jbieniusiewi@nvidia.com> Signed-off-by: jbieniusiewi <jbieniusiewi@users.noreply.github.com> Co-authored-by: jbieniusiewi <jbieniusiewi@users.noreply.github.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

copy of NVIDIA#9852 Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>

* Update trtllm-build options Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Pull QUANT_CFG_CHOICES into try/catch for HAVE_MODELOPT consistency Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Remove deprecated parallel group setup Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Remove deprecated size settings Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Use max_seq_len instead of max_output_len [part I] Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

* Change default parallel_save to False (NVIDIA#9633) * Unwrap ckpt_io for model opt (async save) (NVIDIA#9622) (NVIDIA#9634) * add reset_lr documentation * fix style * fix style * fix style * add image * fix typo * fix plot * fix plot * change plot size * fix style * move image * add reset_lr to intro page --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com>

* expose num_dataset_builder_threads arg Signed-off-by: ashors1 <ashors@nvidia.com> * upate docstring Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* ci: Selective triggering Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * simplify pass-through Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * fix Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * fix Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * refactor tests that dont use template Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* override optimizer step to log Signed-off-by: Maanu Grover <maanug@nvidia.com> * import fix Signed-off-by: Maanu Grover <maanug@nvidia.com> * remove norm from output Signed-off-by: Maanu Grover <maanug@nvidia.com> * also log num zeros in grad Signed-off-by: Maanu Grover <maanug@nvidia.com> * bugfix Signed-off-by: Maanu Grover <maanug@nvidia.com> --------- Signed-off-by: Maanu Grover <maanug@nvidia.com>

* Added basic support for adding LoRA checkpoints in HF format when running deploy_vllm_triton.py Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Moved the conversion logic from the convert_nemo_to_canonical.py script to a reusable module, removed the tar unpacking, removed the dependencies on OmegaConf and NLPSaveRestoreConnector. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Implemented on-load conversion of Nemo format LoRA checkpoints into HF format for vLLM. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Added logger initialization, improved some messages. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Moved the LoRA converter script to nemo.export.utils. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Fixed the description of the query.py script. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * Apply isort and black reformatting Signed-off-by: apanteleev <apanteleev@users.noreply.github.com> * Fixed the missing file close. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: apanteleev <apanteleev@users.noreply.github.com> Co-authored-by: apanteleev <apanteleev@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok>

* Yi1.5 34b neva prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * apply black and isort Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add tokenizer and inference configs in docstring Signed-off-by: HuiyingLi <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com>

* move to cpu only for log probs Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <nithinraok@users.noreply.github.com>

* remove attention mask as default Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * fix non TE case Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com>

* Make get_optim_config iterable Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update error message Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

Signed-off-by: Hemil Desai <hemild@nvidia.com>

* [SD] Add te-dpa support Signed-off-by: Wil Kong <alpha0422@gmail.com> * [SD] Add te-dpa support, resolve compatibility with TE-master Signed-off-by: Wil Kong <alpha0422@gmail.com> * [SD] Add te-dpa support, add check for attention configs. Signed-off-by: Wil Kong <alpha0422@gmail.com> * Fix bugs of flash-attn and dpa in SD. Signed-off-by: Wil Kong <alpha0422@gmail.com> * Fix the issue of DPA API change. Signed-off-by: Wil Kong <alpha0422@gmail.com> * Apply isort and black reformatting Signed-off-by: alpha0422 <alpha0422@users.noreply.github.com> Signed-off-by: Wil Kong <alpha0422@gmail.com> --------- Signed-off-by: Wil Kong <alpha0422@gmail.com> Signed-off-by: alpha0422 <alpha0422@users.noreply.github.com> Co-authored-by: Mengdi Wang <didow@nvidia.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

Signed-off-by: ashors1 <ashors@nvidia.com>

…0279) * Add option to selectively load context in nemo.lightning.io Signed-off-by: Hemil Desai <hemild@nvidia.com> * Update docstring Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fixes Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Co-authored-by: Marc Romeyn <mromeijn@nvidia.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* bugfix Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> * Apply isort and black reformatting Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> * add quantisation flags to deploy_triton.py Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> --------- Signed-off-by: Piotr Kaminski <pikaminski@nvidia.com> Signed-off-by: Laplasjan107 <Laplasjan107@users.noreply.github.com> Co-authored-by: Piotr Kaminski <pikaminski@nvidia.com> Co-authored-by: Laplasjan107 <Laplasjan107@users.noreply.github.com>

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>

* Add comment to address a frequently asked question Signed-off-by: Chen Cui <chcui@nvidia.com> * wording Signed-off-by: Chen Cui <chcui@nvidia.com> * add docstring to GPTSFTPackedDataset Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com>

* Fix async checkpointing in nemo.lightning Signed-off-by: Hemil Desai <hemild@nvidia.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add unit tests Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com>

* Add Nemotron recipes and Long Context Recipe Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Add Nemotron4 recipes Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Change long context recipe for llama3 mixtral and nemotron. Add Expert Parallel Support Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Revert Chainedoptimizer change Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Update mixed_precision.py Signed-off-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> * Update mixtral_8x7b.py Signed-off-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> * Add option to not generate attention mask Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Remove nemtron Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> * Resolve mock conflict Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> --------- Signed-off-by: Boxiang Wang <boxiangw@nvidia.com> Signed-off-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

* Adding copyright to collections.llm & lightning Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> --------- Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> Co-authored-by: marcromeyn <marcromeyn@users.noreply.github.com>

…IDIA#10154) Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>

* remove qk layer scaling * add sc2

* remove grad clipping from mixed_precision plugin Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Raise an expection if user tries to use Trainer's clip_grad Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * Remove gradient_clip_val from recipes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

…in AutoResume and MegatronStrategy (NVIDIA#10295) * Add option to selectively restore model weights and optimizer states in AutoResume and MegatronStrategy Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fix PEFT checkpointing Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> * address comments Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> * Refactor selective restore Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * fix tokenizer issue for peft load Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> * fix peft optimizer states loading Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> * Fix fabric strategy Signed-off-by: Hemil Desai <hemild@nvidia.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Rename SelectiveRestoreConfig to RestoreConfig Signed-off-by: Hemil Desai <hemild@nvidia.com> * Update hf llama 3 paths Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fixes Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fix tests Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: cuichenx <cuichenx@users.noreply.github.com>

Signed-off-by: Malay Nagda <malayn@nvidia.com>

* fix links Signed-off-by: eharper <eharper@nvidia.com> * fix links Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com>

* improve TE import guards Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * small fixes Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * bug fixes Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com>

* ci: Add secrets detector Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * chore: Add baseline Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* expose TP overlap Signed-off-by: Jieming Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * add tp overlap recipes Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * turn on pipeline parallel overlap Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * refactor Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * Update base.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * Update megatron_parallel.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * remove env var Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * add optimization config Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix typo Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * refactor into megatron parallel setup Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * refactor Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix config ordering, add wgrad deferral Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * cleanup Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * use config Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * clean Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * enable wgrad defferal Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add grad bucket size Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * move everthing into a callback Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * cleanup Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix imports Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * move userbuffer init Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * cleanup Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix VP Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * address comments Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add gradient accum guard Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * Update base.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * address comments Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * address comments Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> --------- Signed-off-by: Jieming Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jieming Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com>

* save model weights and artifacts to separate directories Signed-off-by: ashors1 <ashors@nvidia.com> * add save_artifacts_on_train_end Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * do not save optimizer states in final checkpoint Signed-off-by: ashors1 <ashors@nvidia.com> * WIP support for saving only last k optimizer states Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * minor cleanup Signed-off-by: ashors1 <ashors@nvidia.com> * Revert support for saving last k optimizer states. This will be addressed in a subsequent PR. * use storage_options to determine when to skip saving optimizer states Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * fix variable names, make checkpoint load work when optimizer states don't exist in the checkpoint Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * FSDP updates, provide option to save optimizer states on_train_end Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * simplify implementation, remove save_best_model option Signed-off-by: ashors1 <ashors@nvidia.com> * update default value of ckpt_include_optimizer for fsdp Signed-off-by: ashors1 <ashors@nvidia.com> * remove unused imports Signed-off-by: ashors1 <ashors@nvidia.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> * cleanup Signed-off-by: ashors1 <ashors@nvidia.com> * make storage_options optional again Signed-off-by: ashors1 <ashors@nvidia.com> * fix failing tests Signed-off-by: ashors1 <ashors@nvidia.com> * address some comments Signed-off-by: ashors1 <ashors@nvidia.com> * use save_weights_only to determine whether to save optimizer states Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * add some comments Signed-off-by: ashors1 <ashors@nvidia.com> * fix tests Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * fixes Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * remove unnecessary line Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com>

* Split CPU unit tests * Split CPU unit tests * Fix:Run pytest in specific paths * Fix:Run pytest in specific paths * Fix:Run pytest in specific paths

* ci: Fix checkout of secrets detector Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: ashors1 <ashors@nvidia.com>

* remove save_best_model from default logger Signed-off-by: ashors1 <ashors@nvidia.com> * fix broken checkpoint restore Signed-off-by: ashors1 <ashors@nvidia.com> * fix fsdp Signed-off-by: ashors1 <ashors@nvidia.com> * rename weights path to avoid confusion Signed-off-by: ashors1 <ashors@nvidia.com> * Revert "rename weights path to avoid confusion". We'll add this in a separate PR This reverts commit 72bae8b. --------- Signed-off-by: ashors1 <ashors@nvidia.com>

* add base configs Signed-off-by: dimapihtar <dpihtar@gmail.com> * add auto configurator functionality Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * add runner Signed-off-by: dimapihtar <dpihtar@gmail.com> * add end-to-end example for auto configurator Signed-off-by: dimapihtar <dpihtar@gmail.com> * add unit tests for auto configurator Signed-off-by: dimapihtar <dpihtar@gmail.com> * add GPT configs Signed-off-by: dimapihtar <dpihtar@gmail.com> * add GPT configs Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * switch to dataclass Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * switch to dataclass Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix dataclasses usage Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove unused imports Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove extra function Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix docstring style Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * take Config object as input for model Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * add nemotron support Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove search_config.py Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * move configs creation to Basic class Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * move to common basic class Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * rename main config Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove base configs for models Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: artbataev <artbataev@users.noreply.github.com> * change auto conf functionality Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix docstring Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove unused imports Signed-off-by: dimapihtar <dpihtar@gmail.com> * add changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove activations_checkpoint_num_layers Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove gbs from config Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix logs Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix performance calculation Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix end-to-end example Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix model config Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * minor changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * minor changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * add README Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix README Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix README Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix readme Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix readme Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove extra arg Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove unused imports Signed-off-by: dimapihtar <dpihtar@gmail.com> * add nemo-run installation Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> Signed-off-by: artbataev <artbataev@users.noreply.github.com> Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com> Co-authored-by: artbataev <artbataev@users.noreply.github.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Marc Romeyn <mromeijn@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* save tokenizer to disk Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Track Hf tokenizer assets Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * raise exception if dst file exists Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * minor Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * remove print Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add tokenizercontext Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add TokenizerContext Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * restore tokenizer from separate dir Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * update artifact __init__.py Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * TokenizerContext connector Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * bugix on_import_ckpt Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * rm code Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Drop tokenizercontext Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * drop tokenizer load from tokenizercontext Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * undo Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * undo Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Move to util function Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * use save_hf_tokenizer_assets Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * add tokenizer restoration in resume.py Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * bot fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * rm Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * wrap tokenizer restoration in try/catch Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * load_artifacts Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * param fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * more fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * lazy import tensorboard Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * move code out of file context manager Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Allow skippable artifacts Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * rebase fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * checkpoint structure change update Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

* Add option to resume from specific path in AutoResume Signed-off-by: Hemil Desai <hemild@nvidia.com> * Fix path Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* ci: Toggle pre-release Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Add validate_dataset_asset_accessibility Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add CI tests for validate_dataset_asset_accessibility Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix for zipped lists Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: ko3n1g <16716991+ko3n1g@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* ci: Improve release workflow Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * ci: Fix cherry-picking Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* add mamba init * more ssm * add 370m * add hybrid * fix issue * integrate model and tokenizer config for ssm * add all mamba configs * modify state re pattern * revert gpt stuff * remove SSM class and training script * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * remove faulty export * add script to test * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * some recent fixes * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * test script tp/pp1 * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * add cicd * include MLM mamba dist ckpt commit * add license head and address more comments * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * add guard * remove guard from TransformerConfig * update scripts * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> --------- Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> Signed-off-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: Ali Taghibakhshi <ataghibakhsh@login-eos01.eos.clusters.nvidia.com> Co-authored-by: JRD971000 <JRD971000@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* ci: Cherrypick continue on error Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * ci: Fix cherry pick branch Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Adding nemo-run to requirements Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Updating nemo-run entrypoint inside setup.py Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Remove nemo-run from requirements until we have a pypi package Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Update entrypoint naming Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Setting up cli recipe for llama3-8b Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Move AutoTokenizer import inline for starcoder Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Move AutoTokenizer import inline for starcoder2 Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Use target for factories inside llama3_8b Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Update other recipes Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Fix some bugs in the recipes Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Adding some examples Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Adding repl example Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Starting to add a notebook example as well Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Fix wrong imports Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply isort and black reformatting Signed-off-by: pre-commit-ci[bot] <pre-commit-ci[bot]@users.noreply.github.com> * Fix wrong imports Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Fix typo + add script with default executor Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Add nemo-run to Dockerfile.ci Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Adding copyright to recipes Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Adding guides to recipes dir Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Adding hatchling to Dockerfile.ci Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Move install to different line Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * fix install Signed-off-by: Hemil Desai <hemild@nvidia.com> * Move llama3_pretraining to scripts for now Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Remove img folder & use images from release instead Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Updating default of num_nodes in all recipes Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Adding tests for all recipes Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> * ddAing docstrings Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> * Fix failing tests inside test_mixtral_8x7b_64k Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> * Rename fabric to _fabric to avoid name collision with package fabric Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add rename comment Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> Signed-off-by: pre-commit-ci[bot] <pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: marcromeyn <marcromeyn@users.noreply.github.com> Co-authored-by: Hemil Desai <hemild@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

* Split GPU unit tests * Make L0_Unit_Tests_GPU_Lightning optional since flaky * Add time for GPU_Core test * Add time for GPU_Audio test

* energon support draft Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * removing yi, not in PRs scope Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * remove mp_spawn + add energon req Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * removing seq packing Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * fixing encode_sft Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * fix use_energon Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * adding docstrings + addressing comments + CodeQL fixes Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * addressing comments Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * add yi_34b Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * adding assertion for energon + peft, not supported due to dataloader being saved with dist_ckpt Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> * energon usage warning Signed-off-by: paul-gibbons <paul@gibbonspaul.com> * Apply isort and black reformatting Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> --------- Signed-off-by: paul-gibbons <paul@gibbonspaul.com> Signed-off-by: paul-gibbons <paul-gibbons@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: paul-gibbons <paul-gibbons@users.noreply.github.com> Co-authored-by: Yu Yao <54727607+yaoyu-33@users.noreply.github.com>

Signed-off-by: Malay Nagda <malayn@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com>

* add some placeholders Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * few updates Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * update Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * fix logging Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * add neva mock dataset Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * update neva conversation Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * neva data update Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * fixes to conversation Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * data save Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Update paths and fix init Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * some fixes Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * fix combined embeddings logic Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * revert debug code Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix mock dataset for neva Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix dataset part and add llava Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix and updates Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix and updates Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix and updates Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Add checkpoint saving dev test script Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix and updates for neva pretraining Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Fix and updates for neva finetuning Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Fix minor issues in neva 2.0 upgrade Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Update init Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * update examples Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Formatting Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * fix formatting Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Add license and fix formatting Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Fix formatting Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix examples Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix neva api file Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Update tokens Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Update arg apis Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * Update model and pretrain script Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * finetune and inference fixes Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * address comments Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * fix Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix dataset resuming Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * address comments Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * clean up Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * update loss mask class Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * fix Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * fix args Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * address comments Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> * remove examples Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * remove factory Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> Co-authored-by: yaoyu-33 <yaoyu-33@users.noreply.github.com>

…#10436)

* Make flaky test optional * Make flaky test optional

* Make flaky test optional * Make flaky test optional * Make flaky test optional

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Make flaky test optional * Make flaky test optional * Make flaky test optional * Make flaky test optional --------- Signed-off-by: Pablo Garay <palenq@gmail.com>

Also edits get_git_hash to not error when git is not installed in the local container. Signed-off-by: Peter St. John <pstjohn@nvidia.com>

* Make flaky test optional * Make flaky test optional

* Make flaky test optional * Make flaky test optional * Make flaky test optional

* add parakeet-tdt_ctc-110m model Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <nithinraok@users.noreply.github.com>

* [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 8307fcd ! Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * update TE import paths Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * Update parallelisms.rst fix sed typo. Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> * fix for mcore dist opt refactor: move overlap_grad_reduce/overlap_param_gather to ddp config Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * remove overlap_grad_reduce overlap_param_gather from autoconfig Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * subclass TransformerConfig because megatronmodule expects it to have fp8 attr Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * revert change; Use ModelParallelConfig & add fp8 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix, set NVTE_APPLY_QK_LAYER_SCALIN=1 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

* add the mcore interface for optim arg; overlap param AG with optimizer Signed-off-by: Sangkug Lym <slym@nvidia.com> * Apply isort and black reformatting Signed-off-by: erhoo82 <erhoo82@users.noreply.github.com> --------- Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: erhoo82 <erhoo82@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: erhoo82 <erhoo82@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

Signed-off-by: Hemil Desai <hemild@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

* Optional test needs optional field set true * make flaky test optional

* add ci tests for Auto Configurator Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix ci tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix ci tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix ci tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* rename weights path to avoid confusion Signed-off-by: ashors1 <ashors@nvidia.com> * use pathlib utils rather than os Signed-off-by: ashors1 <ashors@nvidia.com> * update resume_from_path and context_path Signed-off-by: ashors1 <ashors@nvidia.com> * address comment Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

…tes (NVIDIA#10383) * [WIP] fix batch sampler to match megatron dataloaders Signed-off-by: ashors1 <ashors@nvidia.com> * make batchprogress configurable Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: Shriya Rishab <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

…NVIDIA#10466) * update adapter save logic to be compatible with `save_weights_only` Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> Co-authored-by: cuichenx <cuichenx@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

* NeMoLogger: update dir to log_dir Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * NeMologger: update calls Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Marc Romeyn <mromeijn@nvidia.com>

* cuda graph modules Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * bug fixes Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add mcore cuda graph interface Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add mcore cuda graph interface Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix whitespace Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * change te import path Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add cudagraph manager Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * separate external cudagraph flag Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * separate config flags Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * update mcore changes Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add check Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

…A#10361) * add token drop plugin Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add checks Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add expert parallel configs Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * amend comment Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * add comm overlap Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix rebase errors Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * fix typo Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * add test configs Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

* fix partial audio transcription order: Signed-off-by: Nithin Rao Koluguri <nithinraok> * update transcribe_speech.py Signed-off-by: Nithin Rao Koluguri <nithinraok> * fix canary transcription Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * for filepaths Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * add override config option Signed-off-by: Nithin Rao Koluguri <nithinraok> * remove unused imports Signed-off-by: Nithin Rao Koluguri <nithinraok> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * completely remove partial audio transcription Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * update doc strings Signed-off-by: Nithin Rao Koluguri <nithinraok> * remove unused imports Signed-off-by: Nithin Rao Koluguri <nithinraok> * support for translate_speech.py Signed-off-by: Nithin Rao Koluguri <nithinraok> * suggested changes from som Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <nithinraok@users.noreply.github.com> Co-authored-by: stevehuang52 <heh@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* ci: Fix base branch of secrets detector Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

…0470) * Call reload_model_params only if there's no optimizer state Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add nemo-ux resume test Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

NVIDIA#10429) * Add causal_conv1d import guard Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add mamba_ssm import guard Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Apply isort and black reformatting Signed-off-by: janekl <janekl@users.noreply.github.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: janekl <janekl@users.noreply.github.com> Co-authored-by: janekl <janekl@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

* Update doc for fp8 trt-llm export Signed-off-by: Piotr Kamiński <67481570+Laplasjan107@users.noreply.github.com> * Apply review suggestions Signed-off-by: Piotr Kamiński <67481570+Laplasjan107@users.noreply.github.com> * code review Signed-off-by: Piotr Kamiński <67481570+Laplasjan107@users.noreply.github.com> --------- Signed-off-by: Piotr Kamiński <67481570+Laplasjan107@users.noreply.github.com>

Co-authored-by: Mengdi Wang <didow@nvidia.com>

* Lower bound transformers to support nemotron Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> * Add py-modules Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> * Update py-modules Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Dong Hyuk Chang <donghyukc@nvidia.com>

* Filter optimizer state in TRTLLM * Add convert script * Fix script * Add max positional embedding * Apply isort and black reformatting Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> * Add header comment --------- Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> Co-authored-by: meatybobby <meatybobby@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

* add save_last_n_optim_states flag Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * add feature to drop optimizer states from checkpoint Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix drop_optim_states logic Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove file Signed-off-by: dimapihtar <dpihtar@gmail.com> * undo remove extra line Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix style Signed-off-by: dimapihtar <dpihtar@gmail.com> * add drop optimizer states support Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * change drop optim states logic Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove extra import Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove extra parameter Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix style Signed-off-by: dimapihtar <dpihtar@gmail.com> * add unit test for drop optim states feature Signed-off-by: dimapihtar <dpihtar@gmail.com> * add unit test for drop optim states feature Co-authored-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> * move drop optim scripts to NeMoModelCheckpoint Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * move drop_optim_states functionality to NeMoModelCheckpoint Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove drop_optim_states functionality from NLPDDPStrategy Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove extra param Signed-off-by: dimapihtar <dpihtar@gmail.com> * reset changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove extra line Signed-off-by: dimapihtar <dpihtar@gmail.com> * add safe save and remove ckpt when drop optim states Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * remove unused param Signed-off-by: dimapihtar <dpihtar@gmail.com> * add load state dict Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix drop optim strategy Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * return extra line Signed-off-by: dimapihtar <dpihtar@gmail.com> * remove ckpt on a single node Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * change params naming Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix bug Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bug Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * add file Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * restore current ckpt state dict when dropping optim states Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * minor changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * minor changes Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> --------- Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

* fix asr finetune Signed-off-by: stevehuang52 <heh@nvidia.com> * fix vad infer Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com>

* add skip_train support Signed-off-by: dimapihtar <dpihtar@gmail.com> * add drop_layers support Signed-off-by: dimapihtar <dpihtar@gmail.com> * add script example of how to trim the model Signed-off-by: dimapihtar <dpihtar@gmail.com> * add documentation for drop_layers feature Signed-off-by: dimapihtar <dpihtar@gmail.com> * add unit test for skip_train feature Signed-off-by: dimapihtar <dpihtar@gmail.com> * add unit test for skip_train feature Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix syntax error Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix skip_train test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add noop_forward_patch Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix doc file Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>

Signed-off-by: ashors1 <ashors@nvidia.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* Make llm/__init__ nemo_run dependency optional Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: Marc Romeyn <mromeijn@nvidia.com>

* move mamba installation Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * FIX Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * remove mounts Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add no-deps Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add --no-index Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * --no-binary Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * --no-binary Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * use git clone Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add --depth=1 to mcore git clone Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix typo Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Move mamba before nemo_run Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * revert megatron git clone Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

* Update infer_data_path.py Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Remove unused download_nemo_checkpoint function Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Change file permissions Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Bugfix for trt_llm_export_kwargs Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Update export test script and model metadata Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Apply isort and black reformatting Signed-off-by: janekl <janekl@users.noreply.github.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: janekl <janekl@users.noreply.github.com> Co-authored-by: janekl <janekl@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com>

* adds Llama 3.1 405b configurations Signed-off-by: Elnifio <elnifio0519@gmail.com> * refines comments Signed-off-by: Elnifio <elnifio0519@gmail.com> * removes untested parts Signed-off-by: Elnifio <elnifio0519@gmail.com> * removes associated unused libraries Signed-off-by: Elnifio <elnifio0519@gmail.com> * small patches Signed-off-by: Elnifio <elnifio0519@gmail.com> --------- Signed-off-by: Elnifio <elnifio0519@gmail.com> Co-authored-by: Shriya Rishab <69161273+ShriyaPalsamudram@users.noreply.github.com>

* create mixed precision plugin recipes Signed-off-by: Maanu Grover <maanug@nvidia.com> * cleanup Signed-off-by: Maanu Grover <maanug@nvidia.com> * fix annotation Signed-off-by: Maanu Grover <maanug@nvidia.com> * full definition instead of attach Signed-off-by: Maanu Grover <maanug@nvidia.com> * update docstrings Signed-off-by: Maanu Grover <maanug@nvidia.com> * update fp8 Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Apply isort and black reformatting Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> * refactor import Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> --------- Signed-off-by: Maanu Grover <maanug@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <JimmyZhang12@users.noreply.github.com>

* [NeMo-UX] Added nemotron recipes and tests Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * fixed issues with precision handling Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * Apply isort and black reformatting Signed-off-by: ahmadki <ahmadki@users.noreply.github.com> Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * missing LoRA config in finetuning Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * nemotron recipe imports Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * cleanup and linting Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * moved gradient clipping to OptimizerConfig, Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> * Added long context nemotron Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> --------- Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> Signed-off-by: ahmadki <ahmadki@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: ahmadki <ahmadki@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com>

* added energon dataloader for neva training * Apply isort and black reformatting Signed-off-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com> * specify global batch size to support grad accumulation * adding neva pretrain example * Apply isort and black reformatting Signed-off-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com> * change pretraine example to handle new ckpt reloading * fixed code quality warnings and unused imports Signed-off-by: ykarnati <ykarnati@nvidia.com> * minor changes for PR comments * Apply isort and black reformatting Signed-off-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com> * refactor conversation template config * Apply isort and black reformatting Signed-off-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com> * remove optional import --------- Signed-off-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com> Signed-off-by: ykarnati <ykarnati@nvidia.com> Co-authored-by: yashaswikarnati <yashaswikarnati@users.noreply.github.com>

* Add unit tests for model configs in nemo.collections.llm Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add nemotron tests Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add gemma Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add other models Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com>

* optim restore test Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add missing import Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: Hemil Desai <hemild@nvidia.com>

…tlib get_cmap (NVIDIA#10542) (NVIDIA#10553) Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com>

* replace unbiased with correction Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <nithinraok@users.noreply.github.com>

* Call original _te._get_extra_te_kwargs and modify device Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add dtype utils Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix Llama ckpt conversion dtype Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * propage ckpt precision changes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * minor improvement Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * bot happy Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: ashors1 <ashors@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* remove running validating after finetuning * Apply isort and black reformatting Signed-off-by: huvunvidia <huvunvidia@users.noreply.github.com> --------- Signed-off-by: huvunvidia <huvunvidia@users.noreply.github.com> Co-authored-by: Huy Vu2 <huvu@login-eos01.eos.clusters.nvidia.com> Co-authored-by: huvunvidia <huvunvidia@users.noreply.github.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

* remove exp dir * add back tests * Apply isort and black reformatting Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> * add run_id to dir * fix dir->log_dir --------- Signed-off-by: JRD971000 <JRD971000@users.noreply.github.com> Co-authored-by: JRD971000 <JRD971000@users.noreply.github.com>

* ci: Send direct alert on failed cherry-pick Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * also to me Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Add validation plugin to nemo.lightning.run Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Rename to ConfigValidationPlugin Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add tests for each recipes with plugins Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Add nsys plugin test Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add more tests Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com>

* fix minor import bug Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com> * fix pps bug * remove config file Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com> --------- Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

* fix type bugs Signed-off-by: stevehuang52 <heh@nvidia.com> * Update mixin.py add type hint Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Update mixin.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Update mixin.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> Co-authored-by: stevehuang52 <stevehuang52@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* ci: Fix issues with version bump Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * fix Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Extend modelopt spec for TEDotProductAttention to support sliding window attention Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Simplify import guarding Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

Signed-off-by: Piotr Żelasko <petezor@gmail.com>

Signed-off-by: Abhishree <abhishreetm@gmail.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

…10559) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok>

* Move update_config_with_dtype_overrides logging to debug Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * update comment Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Introducing MegatronStep to make things more extensible Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Improve megatron callbacks Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Some small fixes Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * remove debg code Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add forward_only to forward backward func Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add global-batch-sampler support to MegatronStep Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Marc Romeijn <mromeijn@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: Marc Romeijn <mromeijn@nvidia.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: sichu <sichu@nvidia.com>

…0623) Signed-off-by: Piotr Żelasko <petezor@gmail.com>

Signed-off-by: Boxiang Wang <boxiangw@nvidia.com>

* contiguous Signed-off-by: Chen Cui <chcui@nvidia.com> * fix load Signed-off-by: Chen Cui <chcui@nvidia.com> * add test script Signed-off-by: Chen Cui <chcui@nvidia.com> * Apply isort and black reformatting Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: artbataev <artbataev@users.noreply.github.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: cuichenx <cuichenx@users.noreply.github.com> Signed-off-by: artbataev <artbataev@users.noreply.github.com> Co-authored-by: cuichenx <cuichenx@users.noreply.github.com> Co-authored-by: artbataev <artbataev@users.noreply.github.com>

* update post process Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Fix wild card matching in nemo2 Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Apply isort and black reformatting Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: yaoyu-33 <yaoyu-33@users.noreply.github.com> Co-authored-by: yaoyu-33 <yaoyu-33@users.noreply.github.com>

* huvu/t5_nemo2.0 first commit from local * runable training * commit to save * update nemo/collections/llm/t5/data/pre_training.py, adding cicd test * updating codes * reset nemo/collections/nlp/parts/megatron_trainer_builder.py * reset megatron_lm_encoder_decoder_model.py, remove t5_release_test_config.sh * update init files * update Dockerfile.ci * fix wandb for cicd test * update training data path * remove uninstall TE * update .github/workflows/cicd-main.yml, disable fused/flashAttn * adjusting val_check_interval for action ci-cd tests * restore .github/workflows/cicd-main.yml * update * update nemologger args * just run pretraining in cicd test * update code llm strutures * adding tests/collections/llm/megatron_t5_pretraining.py * fix cicd * fix cicd * fix cicd * Apply isort and black reformatting Signed-off-by: huvunvidia <huvunvidia@users.noreply.github.com> --------- Signed-off-by: huvunvidia <huvunvidia@users.noreply.github.com> Signed-off-by: Huy Vu <86480512+huvunvidia@users.noreply.github.com> Co-authored-by: Huy Vu2 <huvu@login-eos01.eos.clusters.nvidia.com> Co-authored-by: huvunvidia <huvunvidia@users.noreply.github.com>

* ci: Add CICD result feedback Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* bug fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * bug fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * Apply isort and black reformatting Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> --------- Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>

* ci: Fix mention Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* check Signed-off-by: Nithin Rao Koluguri <nithinraok> * 1 Signed-off-by: Nithin Rao Koluguri <nithinraok> * added to examples/asr Signed-off-by: Nithin Rao Koluguri <nithinraok> * deprecates cuda.amp.autocast to replace with amp.autocast(**Args) Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * check Signed-off-by: Nithin Rao Koluguri <nithinraok> * 1 Signed-off-by: Nithin Rao Koluguri <nithinraok> * added to examples/asr Signed-off-by: Nithin Rao Koluguri <nithinraok> * deprecates cuda.amp.autocast to replace with amp.autocast(**Args) Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * tested on V100 Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> * replace cuda for jit scripted modules Signed-off-by: Nithin Rao Koluguri <nithinraok> * device type fix Signed-off-by: Nithin Rao Koluguri <nithinraok> * revert diar for CI Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <nithinraok@users.noreply.github.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <nithinraok@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

* Support LoRA in TensorRTMMExporter * Apply isort and black reformatting Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> * Add lora uids * Fix mm lora * Apply isort and black reformatting Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> * Fix lora * Fix conf and deploy script * Apply isort and black reformatting Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> --------- Signed-off-by: meatybobby <meatybobby@users.noreply.github.com> Co-authored-by: meatybobby <meatybobby@users.noreply.github.com>

* Add __all__ for mistral & mixtral Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add model import test Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add to cicd Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * skip mixtral test until use_safetnsors arg is resolved Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * syntax Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * typo Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * remove unused imports Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * remove streaming ckpt Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update ckpt paths Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

* Add lazy init for export Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

…obs (NVIDIA#10649) Signed-off-by: Hemil Desai <hemild@nvidia.com>

* add Canary cats Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * Apply isort and black reformatting Signed-off-by: karpnv <karpnv@users.noreply.github.com> * rm use_distributed_sampler=False Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * Apply isort and black reformatting Signed-off-by: karpnv <karpnv@users.noreply.github.com> * rm use_distributed_sampler Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * update lhotse Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * Apply isort and black reformatting Signed-off-by: karpnv <karpnv@users.noreply.github.com> * fix global_rank Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * OmegaConf.set_struct Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * review fix Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * Apply isort and black reformatting Signed-off-by: karpnv <karpnv@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> * predict_step return Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> --------- Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> Signed-off-by: karpnv <karpnv@users.noreply.github.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: pzelasko <pzelasko@users.noreply.github.com> Co-authored-by: karpnv <karpnv@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: pzelasko <pzelasko@users.noreply.github.com>

Co-authored-by: Huy Vu2 <huvu@login-eos01.eos.clusters.nvidia.com>

* Require setuptools>=70 and update deprecated api Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> * Apply isort and black reformatting Signed-off-by: thomasdhc <thomasdhc@users.noreply.github.com> --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: thomasdhc <thomasdhc@users.noreply.github.com> Co-authored-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: thomasdhc <thomasdhc@users.noreply.github.com>

* import fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add get_tokenizer_list import test Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * cycle import fix attempt Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix2 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * fix3 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> * avoid import Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * remove relative imports Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

* provide support for save_last='link' Signed-off-by: ashors1 <ashors@nvidia.com> * fix symlinks when top_k checkpoint not saved Signed-off-by: ashors1 <ashors@nvidia.com> * support symlinks with async checkpointing Signed-off-by: ashors1 <ashors@nvidia.com> * only unlink on rank 0 Signed-off-by: Anna Shors <ashors@nvidia.com> * fix race condition Signed-off-by: ashors1 <ashors@nvidia.com> * force linked checkpoint to correspond to last finalized checkpoint Signed-off-by: ashors1 <ashors@nvidia.com> * fix last_model_path after restore Signed-off-by: ashors1 <ashors@nvidia.com> * move symlink removal to strategy Signed-off-by: ashors1 <ashors@nvidia.com> * remove unneeded lines Signed-off-by: ashors1 <ashors@nvidia.com> * add some more documentation Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * address some comments Signed-off-by: ashors1 <ashors@nvidia.com> * fix syntax Signed-off-by: ashors1 <ashors@nvidia.com> * avoid overwriting _save_last_checkpoint Signed-off-by: ashors1 <ashors@nvidia.com> * fix base call Signed-off-by: ashors1 <ashors@nvidia.com> * small fix Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * add test for save_last=link Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * clean up test Signed-off-by: ashors1 <ashors@nvidia.com> * use megatroncheckpointio in test Signed-off-by: ashors1 <ashors@nvidia.com> * add async test and clean up Signed-off-by: ashors1 <ashors@nvidia.com> * fix remaining merge conflicts Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * check number of saved checkpoints Signed-off-by: ashors1 <ashors@nvidia.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> * run test on gpu only Signed-off-by: ashors1 <ashors@nvidia.com> * fix a small bug and add a resume test Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * remove old comment Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: Anna Shors <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com>

Signed-off-by: mingyuanm <mingyuanm@nvidia.com>

* ci: Switch to `--runtime=nvidia` Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * fix success marker Signed-off-by: Oliver Koenig <okoenig@nvidia.com> * f Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: Oliver Koenig <okoenig@nvidia.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

Signed-off-by: WoodieDudy <WoodieDudy@users.noreply.github.com>

Signed-off-by: WoodieDudy <goshagks@gmail.com>

use torch scaled_dot_product_attention #1

Are you sure you want to change the base?

use torch scaled_dot_product_attention #1

Commits on Aug 22, 2024

Commits on Aug 23, 2024

Commits on Aug 24, 2024

Commits on Aug 26, 2024

Commits on Aug 27, 2024

Commits on Aug 28, 2024

Commits on Aug 29, 2024

Commits on Aug 30, 2024

Commits on Aug 31, 2024

Commits on Sep 2, 2024

Commits on Sep 3, 2024

Commits on Sep 4, 2024

Commits on Sep 5, 2024

Commits on Sep 6, 2024

Commits on Sep 7, 2024

Commits on Sep 8, 2024

Commits on Sep 9, 2024

Commits on Sep 10, 2024

Commits on Sep 11, 2024

Commits on Sep 12, 2024

Commits on Sep 13, 2024

Commits on Sep 15, 2024

Commits on Sep 16, 2024

Commits on Sep 17, 2024

Commits on Sep 18, 2024

Commits on Sep 19, 2024

Commits on Sep 20, 2024

Commits on Sep 21, 2024

Commits on Sep 22, 2024

Commits on Sep 23, 2024

Commits on Sep 24, 2024

Commits on Sep 25, 2024

Commits on Sep 26, 2024

Commits on Sep 27, 2024

Commits on Sep 30, 2024

Commits on Oct 1, 2024

Commits on Oct 3, 2024

Commits on Oct 5, 2024

Commits on Oct 7, 2024

Commits on Oct 8, 2024