Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: TRTLLM API handle tokenizers without pad_id (e.g., tiktoken) #399

Merged
merged 6 commits into from
Nov 20, 2024
Merged

feat: TRTLLM API handle tokenizers without pad_id (e.g., tiktoken) #399

merged 6 commits into from
Nov 20, 2024

Conversation

terrykong
Copy link
Collaborator

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Changelog

  • Please update the CHANGELOG.md under next version with high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

Checklist when contributing a new algorithm

  • Does the trainer resume and restore model state all states?
  • Does the trainer support all parallelism techniques(PP, TP, DP)?
  • Does the trainer support max_steps=-1 and validation?
  • Does the trainer only call APIs defined in alignable_interface.py?
  • Does the trainer have proper logging?

Additional Information

  • Related to # (issue)

terrykong and others added 3 commits November 11, 2024 17:45
@terrykong
feat: TRTLLM API handle tokenizers without pad_id (e.g., tiktoken)
79630dd 
Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong
ci: adds cherry-pick to enable GPT TRTLLM version with bias=False
72ad72b 
Signed-off-by: Terry Kong <terryk@nvidia.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
3dd6eb5 
for more information, see https://pre-commit.ci

Signed-off-by: NeMo-Aligner CI <nemo-aligner-ci@nvidia.com>
@github-actions github-actions bot added the Utils label Nov 12, 2024
@terrykong terrykong added the Run CICD Set + un-set to retrigger label Nov 13, 2024
@terrykong terrykong enabled auto-merge (squash) November 13, 2024 22:50
@terrykong
go with pad_id=-42 all the time
83f1c0f 
Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong
revert the dockerfile change
bd9fa0b 
Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong
Merge branch 'main' into tk/trtllm-next
80f7960
@terrykong terrykong added Run CICD Set + un-set to retrigger and removed Run CICD Set + un-set to retrigger labels Nov 20, 2024
@terrykong terrykong merged commit eb2db8b into main Nov 20, 2024
17 checks passed
@terrykong terrykong deleted the tk/trtllm-next branch November 20, 2024 20:27
abukharin3 pushed a commit to abukharin3/NeMo-Aligner that referenced this pull request Nov 22, 2024
@terrykong @pre-commit-ci
feat: TRTLLM API handle tokenizers without pad_id (e.g., tiktoken) (N…
f5b93c9 
…VIDIA#399)

Signed-off-by: Terry Kong <terryk@nvidia.com>
Signed-off-by: NeMo-Aligner CI <nemo-aligner-ci@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: abukharin <abukharin@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gshennvm gshennvm gshennvm approved these changes

Assignees
No one assigned
Labels
Run CICD Set + un-set to retrigger Utils
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@terrykong @gshennvm