Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

43 Open 110 Closed
43 Open 110 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Benchmarking MLA
#700 opened Dec 25, 2024 by YLGH
Different sequence numbers calculate inconsistent results
#696 opened Dec 24, 2024 by sitabulaixizawaluduo
@yzh119
9
Load the qwen2_5_insturcut_7b model, because the deployment request response using sglang is slower than that of vllm deployment. When troubleshooting, print("mutates_args",mutates_args,"schema_str",schema_str) is this result reasonable?
#695 opened Dec 24, 2024 by qingzhong1
1
Can't install latest flashinfer for torch 2.5
#692 opened Dec 23, 2024 by sleepwalker2017
@yzh119
Deprecation Notice: Python 3.8 Wheel Support to End in future releases
#682 opened Dec 18, 2024 by yzh119
2
[Roadmap] FlashInfer v0.2 to v0.3 roadmap
#675 opened Dec 17, 2024 by yzh119
12 tasks
[Bug] FlashInfer latest main wheel issue bug Something isn't working priority: high
#669 opened Dec 16, 2024 by zhyncs
4
Value error when running decode attention with FP8 KV cache
#661 opened Dec 13, 2024 by sugunav14
@yzh119
3
ValueError: The dtype of q torch.bfloat16 does not match the q_data_type torch.float16 specified in plan function.
#638 opened Nov 25, 2024 by Godlovecui
1
append_kv_cache's documentation is out of date
#616 opened Nov 19, 2024 by reyoung
@yzh119
1
[Question] Overflow risks when batch size and sequence length grows extremely large
#596 opened Nov 8, 2024 by rchardx
1
[Feature Request] Add an argument to control the number of CTAs used in attention APIs
#591 opened Nov 7, 2024 by yzh119
Have any plans to optimize the decode kernel for NV-Hopper
#576 opened Oct 31, 2024 by JamesLim-sy
3
FlashInfer generating NANs on A100 GPU
#574 opened Oct 30, 2024 by dbarbuzzi
@yzh119
3
[Feature request] Adding optional cpu_indptr/cpu_qo_indptr parameter to plan method to avoid synchronized device to host copy.
#565 opened Oct 28, 2024 by reyoung
3
ImportError: cannot import name '_grouped_size_compiled_for_decode_kernels' from 'flashinfer.decode'
#549 opened Oct 23, 2024 by Hutlustc
4
Runtime error with single_prefill_with_kv_cache while Compilation
#541 opened Oct 20, 2024 by YudiZh
6
[Question] Sampling kernel only support FP32 now?
#531 opened Oct 15, 2024 by yz-tang
1
Runtime error with head dim 224
#528 opened Oct 14, 2024 by yz-tang
@yzh119
3
Question about use_tensor_cores = True or False
#520 opened Oct 10, 2024 by sleepwalker2017
2
Is lean attention supported by flash infer?
#515 opened Oct 8, 2024 by sleepwalker2017
16
Compilation issue on old cuda versions
#514 opened Oct 3, 2024 by yzh119
[Feat] Suggest of using MappingUtils to compute coordiantes automatically for different warpSize
#512 opened Sep 27, 2024 by yiakwy-xpu-ml-framework-team
bug: partial unit tests failed
#479 opened Aug 28, 2024 by zhyncs
@zhyncs
2
failed to dispatch head_dim 96
#455 opened Aug 20, 2024 by ZX-ModelCloud
@yzh119
ProTip! What’s not been updated in a month: updated:<2024-11-25.