Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replace all rmsnorm and silu #1057

Merged
merged 1 commit into from
Aug 12, 2024
Merged

feat: replace all rmsnorm and silu #1057

merged 1 commit into from
Aug 12, 2024

Conversation

zhyncs
Copy link
Member

@zhyncs zhyncs commented Aug 12, 2024

Motivation

as titled

tested with chatglm, commandr, deepseek, gemma, mixtral, qwen, qwen2, stablelm

not tested with deepseek v2, grok, minicpm, mixtral_quant, qwen2 moe

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs zhyncs added the wip label Aug 12, 2024
@zhyncs zhyncs self-assigned this Aug 12, 2024
@zhyncs zhyncs marked this pull request as draft August 12, 2024 13:03
@zhyncs
Copy link
Member Author

zhyncs commented Aug 12, 2024

After running this PR, both ag "import SiluAndMul" | grep vllm | wc -l and ag "import RMSNorm" | grep vllm | wc -l return 0.

@zhyncs zhyncs removed the wip label Aug 12, 2024
@zhyncs zhyncs marked this pull request as ready for review August 12, 2024 15:08
@zhyncs
Copy link
Member Author

zhyncs commented Aug 12, 2024

other:

gemma: gelu_and_mul (without tanh)
gpt_bigcode: gelu
gemma2: GemmaRMSNorm ref https://github.com/huggingface/transformers/pull/29402/files

I expect to complete in other PRs.

@zhyncs zhyncs merged commit 6a38efa into sgl-project:main Aug 12, 2024
4 checks passed
@zhyncs zhyncs deleted the fi branch August 12, 2024 16:16
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants