Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does QQQ linear support H100? #12

Closed
donglinz opened this issue Aug 21, 2024 · 1 comment
Closed

Does QQQ linear support H100? #12

donglinz opened this issue Aug 21, 2024 · 1 comment

Comments

@donglinz
Copy link

I tried to run it on H100, but it seems there is an illegal memory access inside the kernel.

RuntimeError: CUDA error: an illegal memory access was encountered
@HandH1998
Copy link
Owner

We don't have a H100, and we don't try it on H100. But the vllm viewer seems to have tried it on H100 in vllm-project/vllm#5218 (comment). Our PR has been merged into vllm now, you can try qqq in vllm on H100 and see if it can work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants