Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Add Liger-Kernel usage to SFTTrainer page #2007

Merged
merged 10 commits into from
Sep 3, 2024

Conversation

ryankert01
Copy link
Contributor

@ryankert01 ryankert01 commented Sep 2, 2024

What does this PR do?

Document a famous package Liger-Kernel usage in SFT trainer page that can can effectively increase multi-GPU training throughput by 20% and reduces memory usage by 60%.

Reference: Liger-Kernel

followup of: huggingface/transformers#32860

fixed #184

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ryankert01 ryankert01 changed the title Add Liger-Kernel usage to SFTTrainer page [Docs] Add Liger-Kernel usage to SFTTrainer page Sep 2, 2024
docs/source/sft_trainer.mdx Outdated Show resolved Hide resolved
docs/source/sft_trainer.mdx Outdated Show resolved Hide resolved
docs/source/sft_trainer.mdx Outdated Show resolved Hide resolved
docs/source/sft_trainer.mdx Outdated Show resolved Hide resolved
docs/source/sft_trainer.mdx Show resolved Hide resolved
ryankert01 and others added 4 commits September 3, 2024 06:58
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
@ryankert01 ryankert01 requested a review from ByronHsu September 2, 2024 23:12
Copy link
Member

@lewtun lewtun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the docs - they look great!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@lewtun lewtun merged commit 728a9a3 into huggingface:main Sep 3, 2024
1 check passed
@ryankert01 ryankert01 deleted the add-liger-kernel branch September 3, 2024 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SFTTrainer] Add doc in huggingface sfttrainer website
4 participants