Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect command for model quantization in README.md #1199

Closed
javadyaali opened this issue Apr 26, 2023 · 2 comments
Closed

Incorrect command for model quantization in README.md #1199

javadyaali opened this issue Apr 26, 2023 · 2 comments

Comments

@javadyaali
Copy link

The correct command for quantizing the model is not reflected accurately in lines 206 and 207 of README.md (Prepare Data & Run). To perform model quantization, you should use the following command instead.

# quantize the model to 4-bits (using method 2 = q4_0)
./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
@slaren
Copy link
Collaborator

slaren commented Apr 26, 2023

If this isn't working for you, you need to update to current master, check #1191 that was merged today.

@slaren slaren closed this as not planned Won't fix, can't repro, duplicate, stale Apr 26, 2023
@javadyaali
Copy link
Author

Thanks @slaren for mentioning that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants