Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Performance] Introducing Prefix-Cached Chunked Prefill with flash-attn backend and 10% throughput gained under prompt <1K #6819

Closed
wants to merge 1 commit into from

[Performance] Introducing Prefix-Cached Chunked Prefill

40a9f31
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Closed

[Performance] Introducing Prefix-Cached Chunked Prefill with flash-attn backend and 10% throughput gained under prompt <1K #6819

[Performance] Introducing Prefix-Cached Chunked Prefill
40a9f31
Select commit
Loading
Failed to load commit list.

Annotations

1 warning

This job succeeded