Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LLM Quantization, GGML, GGUF 에 관해서 #151

Open
d-h-lee opened this issue Nov 25, 2023 · 2 comments
Open

LLM Quantization, GGML, GGUF 에 관해서 #151

d-h-lee opened this issue Nov 25, 2023 · 2 comments
Labels
readable Readable and interesting!

Comments

@d-h-lee
Copy link
Contributor

d-h-lee commented Nov 25, 2023

대형언어모델 때문에 참 이슈가 많은데, 그 "대형"이라는 부분에서 Quantization 관련된 실용적인 개발이 많이 진행중입니다.

몇 가지 툴들이 보이는데, 어떤 실용적인 툴들이 있는지도 서베이가 필요합니다.

그리고, 그 중에서 GGML에 관해서 중요한 변화가 일어나는 중(GGUF로 변경중)이라 이번 기회에 GGML/GGUF에 대해서 살펴보는게 좋을것같습니다.

ggerganov/ggml#220

https://huggingface.co/TheBloke/Llama-2-7B-GGUF 실 사용 용도로 영향을 주는 것들의 예시입니다.
@pinga999
Copy link
Contributor

pinga999 commented Nov 25, 2023 •
edited
Loading

Simple article for GGUF GGML

https://medium.com/@phillipgimmi/what-is-gguf-and-ggml-e364834d241c

@d-h-lee
Copy link
Contributor Author

d-h-lee commented Nov 28, 2023

AutoAWQ: https://github.com/casper-hansen/AutoAWQ

@cth127 cth127 added the readable Readable and interesting! label Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
readable Readable and interesting!
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cth127 @d-h-lee @pinga999