Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLM Inference] support qwen2 a8w8c8 inference #8925

Merged
merged 5 commits into from
Aug 20, 2024

Conversation

ckl117
Copy link
Contributor

@ckl117 ckl117 commented Aug 12, 2024

PR types

New features

PR changes

Others

Description

support qwen2 a8w8c8 inference

Copy link

paddle-bot bot commented Aug 12, 2024

Thanks for your contribution!

Copy link

codecov bot commented Aug 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 162 lines in your changes missing coverage. Please review.

Project coverage is 54.79%. Comparing base (12107af) to head (2b187f5).
Report is 219 commits behind head on develop.

Files with missing lines Patch % Lines
...dlenlp/experimental/transformers/qwen2/modeling.py 0.00% 162 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8925      +/-   ##
===========================================
- Coverage    54.96%   54.79%   -0.17%     
===========================================
  Files          646      643       -3     
  Lines       103133   101253    -1880     
===========================================
- Hits         56687    55482    -1205     
+ Misses       46446    45771     -675     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ckl117 ckl117 changed the title qwen2-a8w8c8 [LLM Inference] support qwen2 a8w8c8 inference Aug 19, 2024
Comment on lines 636 to 637
print(f"weight_scale_json_path = {weight_scale_json_path}")
# print(f'weight_scale_map_dict = {weight_scale_map_dict}')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

调试信息删掉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@DesmonDay
Copy link
Contributor

建议后面增加相应的单测,保证功能。 @ckl117 @yuanlehome

Copy link
Contributor

@DesmonDay DesmonDay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ZHUI ZHUI merged commit ecfe0ac into PaddlePaddle:develop Aug 20, 2024
10 of 12 checks passed
@ckl117 ckl117 deleted the qwen2-a8w8c8 branch September 9, 2024 05:51
Mangodadada pushed a commit to Mangodadada/PaddleNLP that referenced this pull request Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants