Skip to content

feat: support cuda graph for batched multi-query(prefill/append) attention#275

Merged
yzh119 merged 6 commits intomainfrom prefill-cuda-graphJun 2, 2024

Commits

Commits on Jun 2, 2024