Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for new autogptq quant_config.checkpoint_format #332

Merged
merged 2 commits into from
Mar 29, 2024

Conversation

Qubitium
Copy link
Contributor

@Qubitium Qubitium commented Mar 27, 2024

Reason for PR: make sure loading of gptq marlin quants is compatible with latest autogptq

Due to autogpq update AutoGPTQ/AutoGPTQ#603 quants made with autogptq >=0.8.0-dev will use a new checkpoint_format property to hold marlin and future formats.

Compat tested with both marlin and non-marlin in both old is_marlin_format and new checkpoint_format

@merrymercy @hnyls2002 Ready for review.

@Qubitium Qubitium changed the title [WIP] Add support for new autogptq quant_config.checkpoint_format Add support for new autogptq quant_config.checkpoint_format Mar 28, 2024
@Qubitium Qubitium marked this pull request as ready for review March 28, 2024 14:46
@merrymercy merrymercy merged commit eddaa2b into sgl-project:main Mar 29, 2024
@Qubitium Qubitium deleted the marlin-compat branch March 29, 2024 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants