-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Observer Restructure: Remove Observers, calibration
, and applying frozen
steps from lifecycle
#189
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dsikka
changed the title
[WIP] Remove Observers
[WIP] Observer Restructure: Remove Observers
Oct 14, 2024
dsikka
force-pushed
the
remove-observers
branch
from
October 17, 2024 19:33
c37684d
to
d1bee5a
Compare
dsikka
force-pushed
the
remove-observers
branch
from
October 21, 2024 20:09
b8380f7
to
10b7a6c
Compare
dsikka
changed the title
[WIP] Observer Restructure: Remove Observers
Observer Restructure: Remove Observers, Oct 22, 2024
calibration
, and applying frozen
steps from lifecycle
kylesayrs
reviewed
Oct 23, 2024
kylesayrs
reviewed
Oct 23, 2024
dsikka
force-pushed
the
remove-observers
branch
from
October 24, 2024 00:52
495425b
to
b6eeaca
Compare
kylesayrs
reviewed
Oct 24, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks like a clean cut, I'd like to do a little more testing before I approve
kylesayrs
approved these changes
Oct 30, 2024
rahul-tuli
approved these changes
Oct 30, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
apply_quantization_status
will only be able to applyinitialize_module_for_quantization
andcompress_quantized_weights
. TheFROZEN
andCALIBRATION
statuses are still valid QuantizationStatus values but compressed tensors will no longer be able run calibration or "apply" frozen statuses (i.e. remove observers)Once landed, we'll need to land llm-compressor updates to add in Observers right after. Both PRs should be reviewed together: https://github.com/vllm-project/llm-compressor/pull/837/files#diff-b9472770e3291ffd5f7e0adbc074a5b2f83ca00e8dc937e230dcc365f4e2f954
Testing