Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support for LLM.int8 #6

Merged
merged 11 commits into from
Jan 31, 2023
Merged

support for LLM.int8 #6

merged 11 commits into from
Jan 31, 2023

Conversation

pszemraj
Copy link
Owner

  • add support for summarization models with LLM.int8 (per hf implementation)
  • add tf32 support
  • update docs

pszemraj added 11 commits January 30, 2023 18:25
@pszemraj
🚧 util to check optional dep
76d3357 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
📌 min transformers version 4.26
d05a324 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
✨ mwe implementation
79c19a3 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
🚚 🎨 organize
ce14cc9 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
✨ add 8bit precision
c7956db 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
⚡️ add tf32 helper fn
dded546 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
📝 update docs
8a093a9 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
✏️ 📝 improve readme
eb8984f 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
📝 add bnb install
e57854e 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
📝 rm huge warning sign
f9cc1e1 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj
📝 🎨
dd96ee2 
Signed-off-by: Peter <74869040+pszemraj@users.noreply.github.com>
@pszemraj pszemraj added documentation Improvements or additions to documentation enhancement New feature or request labels Jan 30, 2023
@pszemraj pszemraj self-assigned this Jan 30, 2023
pszemraj
pszemraj commented Jan 30, 2023
View reviewed changes
Copy link
Owner Author

@pszemraj pszemraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 changes look fine

@pszemraj
Copy link
Owner Author

basic test nb, things work

@pszemraj pszemraj merged commit 9108f66 into main Jan 31, 2023
@pszemraj pszemraj deleted the 8bit branch January 31, 2023 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@pszemraj pszemraj

Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@pszemraj