Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Misc] Update compressed tensors lifecycle to remove prefix from create_weights #7825

Merged
merged 5 commits into from
Aug 27, 2024
Merged

[Misc] Update compressed tensors lifecycle to remove prefix from create_weights #7825

merged 5 commits into from
Aug 27, 2024

Conversation

dsikka
Copy link
Contributor

@dsikka dsikka commented Aug 23, 2024 •
edited
Loading

Summary

  • Update compressed tensors to match the lifecycle of other quant methods for when unqunatized layers are initialized
  • Update when the scheme is selected to eliminate the need for prefix in create_weights
  • Tested against a non-uniform model from llm-compressor

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

  • Comment /ready on the PR
  • Add ready label to the PR
  • Enable auto-merge.

🚀

@dsikka dsikka changed the title [Misc] Update compressed tensors lifecycle to remove prefix from create_weights [Misc] Update compressed tensors lifecycle to remove prefix from create_weights Aug 23, 2024
@dsikka dsikka changed the title [Misc] Update compressed tensors lifecycle to remove prefix from create_weights [Misc] Update compressed tensors lifecycle to remove prefix from create_weights Aug 23, 2024
@dsikka
Copy link
Contributor Author

dsikka commented Aug 24, 2024

/ready

@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 24, 2024
@dsikka dsikka requested a review from mgoin August 26, 2024 21:07
@mgoin mgoin merged commit 015e6cc into vllm-project:main Aug 27, 2024
44 checks passed
@mgoin mgoin deleted the update_prefix branch August 27, 2024 00:09
triple-Mu pushed a commit to triple-Mu/vllm_official that referenced this pull request Sep 4, 2024
@dsikka @triple-Mu
[Misc] Update compressed tensors lifecycle to remove prefix from `c…
92c0d5c 
…reate_weights` (vllm-project#7825)
Jeffwan pushed a commit to aibrix/vllm that referenced this pull request Sep 19, 2024
@dsikka @Jeffwan
[Misc] Update compressed tensors lifecycle to remove prefix from `c…
39a4a59 
…reate_weights` (vllm-project#7825)
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
@dsikka @Alvant
[Misc] Update compressed tensors lifecycle to remove prefix from `c…
c61360a 
…reate_weights` (vllm-project#7825)

Signed-off-by: Alvant <alvasian@yandex.ru>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
@dsikka
[Misc] Update compressed tensors lifecycle to remove prefix from `c…
fbc5302 
…reate_weights` (vllm-project#7825)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mgoin mgoin mgoin approved these changes

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dsikka @mgoin