Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

Support GGJT v3 #252

Closed
philpax opened this issue May 19, 2023 · 2 comments
Closed

Support GGJT v3 #252

philpax opened this issue May 19, 2023 · 2 comments
Labels
issue:enhancement New feature or request meta:breaking-change Will require action on behalf of the developer
Milestone
0.2

Comments

@philpax
Copy link
Collaborator

philpax commented May 19, 2023

There is a new quantization level in llama.cpp, which means there will be models published with it in the near-future. We will need to support this.

ggerganov/llama.cpp#1508
@philpax philpax added the issue:enhancement New feature or request label May 19, 2023
@philpax philpax added this to the 0.2 milestone May 20, 2023
@philpax
Copy link
Collaborator Author

philpax commented May 21, 2023

To clarify: this has landed as v3 of the format with qnt level 2. We don't currently support this as we're trying to figure out if there's a good way to handle the breaking changes, but we're likely to update to it shortly.

@philpax philpax changed the title Support quantization level 2 Support GGJT v3 May 21, 2023
@philpax philpax added the meta:breaking-change Will require action on behalf of the developer label May 22, 2023
@okpatil4u okpatil4u mentioned this issue May 23, 2023
Closed
@philpax philpax closed this as completed May 25, 2023
@philpax
Copy link
Collaborator Author

philpax commented May 25, 2023

This is now done, with the caveat that all of your v2 models will now be busted. #261 is the plan for that.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
issue:enhancement New feature or request meta:breaking-change Will require action on behalf of the developer
Projects
None yet
Milestone
0.2
Development

No branches or pull requests
1 participant
@philpax