Integrating mcore export #10238

shanmugamr1992 · 2024-08-23T07:13:48Z

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

nemo/export/tensorrt_llm.py

…10234) Signed-off-by: Hemil Desai <hemild@nvidia.com>

…nd offsets in manifest (#10198) * Add tests for LazyNeMoIterator and fix case with manifest_only=True and offsets in manifest Signed-off-by: Piotr Żelasko <petezor@gmail.com> * Address code review Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> * fix tests Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com>

…ckpoints (#9939) * perfor serialization using relative paths to allow users to move checkpoints after they're saved Signed-off-by: ashors1 <ashors@nvidia.com> * Apply isort and black reformatting Signed-off-by: ashors1 <ashors1@users.noreply.github.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> * fix artifact load Signed-off-by: ashors1 <ashors@nvidia.com> * fix path artifact Signed-off-by: ashors1 <ashors@nvidia.com> * remove unused import Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: ashors1 <ashors@nvidia.com> Signed-off-by: ashors1 <ashors1@users.noreply.github.com> Co-authored-by: ashors1 <ashors1@users.noreply.github.com>

* Add MemoryProfileCallback Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Apply isort and black reformatting Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> * Remove reference cycles, save snapshot on specific ranks Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Remove unnecessary imports Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> * Apply isort and black reformatting Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> * Update docstring Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> --------- Signed-off-by: Shriya Palsamudram <spalsamudram@nvidia.com> Signed-off-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Shriya Rishab <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: ShriyaPalsamudram <ShriyaPalsamudram@users.noreply.github.com>

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Dong Hyuk Chang <donghyukc@nvidia.com>

…rocessing (#10052) Flow matching generative model with SSL pretraining framework Signed-off-by: Pin-Jui Ku <pku@nvidia.com> Co-authored-by: Kuray107 <Kuray107@users.noreply.github.com>

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* Move nemotron transformers + tokenizer imports inline to reduce number of required deps Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> * Apply isort and black reformatting Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> --------- Signed-off-by: Marc Romeyn <mromeijn@nvidia.com> Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com> Co-authored-by: marcromeyn <marcromeyn@users.noreply.github.com>

* Wrap CPU model init with megatron_lazy_init_context Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Cleanup checkpoint-dir if saving fails Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Apply isort and black reformatting Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: akoumpa <akoumpa@users.noreply.github.com> Co-authored-by: akoumpa <akoumpa@users.noreply.github.com>

* [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 124bcff ! Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix bert flags Signed-off-by: Oliver Koenig <okoenig@nvidia.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Oliver Koenig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: slyne deng <slyned@nvidia.com> Co-authored-by: slyne deng <slyned@nvidia.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

* Load model in the target export precision by default Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Enable megatron_amp_O2=true to actually use half-precision Signed-off-by: Jan Lasek <jlasek@nvidia.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Jan Lasek <jlasek@nvidia.com>

…n.plugins (#10223) * Add WandbPlugin, NsysPlugin and PreemptionPlugin to nemo.lightning.run.plugins Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Remove duplicate Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add entity to wandb logger Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add documentation Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Add warning Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * PR feedback Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> * Add comments Signed-off-by: Hemil Desai <hemild@nvidia.com> * Apply isort and black reformatting Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: hemildesai <hemildesai@users.noreply.github.com> Co-authored-by: hemildesai <hemildesai@users.noreply.github.com>

* handle absolute and relative logger directories Signed-off-by: Anna Shors <ashors@nvidia.com> * merge lines Signed-off-by: ashors1 <ashors@nvidia.com> --------- Signed-off-by: Anna Shors <ashors@nvidia.com> Signed-off-by: ashors1 <ashors@nvidia.com>

* Add sdxl notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * Rename Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * final Update SDXL notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> --------- Signed-off-by: mingyuanm <mingyuanm@nvidia.com>

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

shanmugamr1992 · 2024-09-24T20:23:26Z

jenkins

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

shanmugamr1992 · 2024-09-26T18:30:43Z

jenkins

oyilmaz-nvidia

LGTM.

shanmugamr1992 · 2024-09-26T20:44:00Z

jenkins

janekl · 2024-09-27T08:48:06Z

We might also want to bump MCore commit here to 45bf4c1821c2a87ca02e8a0377d61097bac92d07 so that it's clear how to use newly integrated code

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

github-actions · 2024-09-27T17:48:18Z

[🤖]: Hi @shanmugamr1992 👋,

I just wanted to let you know that, you know, a CICD pipeline for this PR just finished successfully ✨

So it might be time to merge this PR or like to get some approvals 🚀

But I'm just a 🤖 so I'll leave it you what to do next.

Have a great day!

//cc @ko3n1g

github-actions · 2024-09-28T05:55:05Z

[🤖]: Hi @shanmugamr1992 👋,

I just wanted to let you know that, you know, a CICD pipeline for this PR just finished successfully ✨

So it might be time to merge this PR or like to get some approvals 🚀

But I'm just a 🤖 so I'll leave it you what to do next.

Have a great day!

//cc @ko3n1g

Shanmugam Ramasamy and others added 3 commits August 23, 2024 00:13

Integrating mcore export

7e7eb7f

Integrating mcore export

d6351bb

Apply isort and black reformatting

996ea05

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

github-advanced-security bot found potential problems Aug 23, 2024

View reviewed changes

nemo/export/tensorrt_llm.py Fixed Show fixed Hide fixed

Laplasjan107 mentioned this pull request Aug 26, 2024

Export fp8 te nemo to trt-llm #10096

Merged

8 tasks

hemildesai and others added 18 commits August 27, 2024 15:38

Move trt imports in nemo.collections.llm inside respective functions (#…

7c0584a

…10234) Signed-off-by: Hemil Desai <hemild@nvidia.com>

Lower bound transformers to support nemotron (#10240)

9214a4e

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Dong Hyuk Chang <donghyukc@nvidia.com>

[Audio] SSL Pretraining framework for flow-matching model for audio p…

c690c4f

…rocessing (#10052) Flow matching generative model with SSL pretraining framework Signed-off-by: Pin-Jui Ku <pku@nvidia.com> Co-authored-by: Kuray107 <Kuray107@users.noreply.github.com>

Revert torchrun fix for model import (#10251)

04ca831

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

salm export trtllm (#10245)

5964387

Signed-off-by: slyne deng <slyned@nvidia.com> Co-authored-by: slyne deng <slyned@nvidia.com>

[🤠]: Howdy folks, let's bump Dockerfile.ci to ef85bc9 ! (#10250)

8524596

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com>

Add sdxl notebook (#10139)

97ce34a

* Add sdxl notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * Rename Signed-off-by: mingyuanm <mingyuanm@nvidia.com> * final Update SDXL notebook Signed-off-by: mingyuanm <mingyuanm@nvidia.com> --------- Signed-off-by: mingyuanm <mingyuanm@nvidia.com>

Updating some coments

c52a0a4

shanmugamr1992 requested review from pablo-garay and ko3n1g as code owners August 27, 2024 22:42

github-actions bot added NLP CI common Multi Modal audio labels Aug 27, 2024

github-actions bot removed the stale label Sep 17, 2024

Shanmugam Ramasamy and others added 4 commits September 23, 2024 11:01

Rebase and integrate latest mcore changes

57bb895

Apply isort and black reformatting

822ec5b

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

Merge branch 'main' into integrate_mcore_export

8600d31

Merge branch 'main' into integrate_mcore_export

a691e55

Shanmugam Ramasamy and others added 3 commits September 25, 2024 10:53

ADD support for layernorm1p

e05fe2c

Apply isort and black reformatting

28a0eb5

Signed-off-by: shanmugamr1992 <shanmugamr1992@users.noreply.github.com>

Merge branch 'main' into integrate_mcore_export

6d20aed

oyilmaz-nvidia previously approved these changes Sep 26, 2024

View reviewed changes

oyilmaz-nvidia added NLP Run CICD and removed audio labels Sep 26, 2024

Merge branch 'main' into integrate_mcore_export

aaa4a09

github-actions bot removed the NLP label Sep 26, 2024

ko3n1g added Run CICD and removed Run CICD labels Sep 27, 2024

shanmugamr1992 added 2 commits September 27, 2024 10:43

Merge branch 'main' into integrate_mcore_export

370945e

Update Dockerfile.ci

68c635e

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

shanmugamr1992 dismissed oyilmaz-nvidia’s stale review via 68c635e September 27, 2024 17:45

shanmugamr1992 added Run CICD and removed Run CICD labels Sep 27, 2024

Merge branch 'main' into integrate_mcore_export

15bc02f

shanmugamr1992 added Run CICD and removed Run CICD labels Sep 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrating mcore export #10238

Integrating mcore export #10238

shanmugamr1992 commented Aug 23, 2024

shanmugamr1992 commented Sep 24, 2024

shanmugamr1992 commented Sep 26, 2024

oyilmaz-nvidia left a comment

shanmugamr1992 commented Sep 26, 2024

janekl commented Sep 27, 2024

github-actions bot commented Sep 27, 2024

github-actions bot commented Sep 28, 2024

Integrating mcore export #10238

Are you sure you want to change the base?

Integrating mcore export #10238

Conversation

shanmugamr1992 commented Aug 23, 2024

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

shanmugamr1992 commented Sep 24, 2024

shanmugamr1992 commented Sep 26, 2024

oyilmaz-nvidia left a comment

Choose a reason for hiding this comment

shanmugamr1992 commented Sep 26, 2024

janekl commented Sep 27, 2024

github-actions bot commented Sep 27, 2024

github-actions bot commented Sep 28, 2024