Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug fixes] Fix ring attention #8740

Merged
merged 3 commits into from
Jul 11, 2024

Conversation

zhangyuqin1998
Copy link
Contributor

@zhangyuqin1998 zhangyuqin1998 commented Jul 10, 2024

PR types

Bug fixes

PR changes

Others

Description

由于slice算子的+=运算符非inplace,导致计算出现bug,此pr进行了修复。使用下标索引替换slice算子。
修复tensor.to(dtype)引起的性能下降,替换成paddle.cast。

Copy link

paddle-bot bot commented Jul 10, 2024

Thanks for your contribution!

Copy link

codecov bot commented Jul 10, 2024

Codecov Report

Attention: Patch coverage is 0% with 34 lines in your changes missing coverage. Please review.

Project coverage is 55.71%. Comparing base (e681019) to head (812d4cc).
Report is 216 commits behind head on develop.

Files with missing lines Patch % Lines
paddlenlp/transformers/ring_flash_attention.py 0.00% 34 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8740      +/-   ##
===========================================
- Coverage    55.73%   55.71%   -0.03%     
===========================================
  Files          623      623              
  Lines        97464    97393      -71     
===========================================
- Hits         54324    54259      -65     
+ Misses       43140    43134       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@ForFishes ForFishes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhangyuqin1998 zhangyuqin1998 requested a review from ForFishes July 10, 2024 13:24
Copy link
Member

@ForFishes ForFishes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wawltor wawltor merged commit 2f0f8b1 into PaddlePaddle:develop Jul 11, 2024
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants