[Feature]: Integrate flash-infer FP8 KV Cache Chunked-Prefill (Append Attention)

[jon-chuang]

🚀 The feature, motivation and pitch

From new Flash Infer Release https://github.com/flashinfer-ai/flashinfer/releases/tag/v0.1.4

cc @comaniac

Additional context

Follow up to: #7208, #7185

[jon-chuang]

Actually, @comaniac, I noticed that there are explicit asserts forbidding use of flash infer kernels for chunked prefill

vllm/vllm/attention/backends/flashinfer.py

Line 195 in 774cd1d

# Currently chunked prefill is not supported

As pointed out in: flashinfer-ai/flashinfer#392 (comment)

---

My understanding is that this is because vLLM runs prefill and decode in two separate kernel invocations by default (as is the case for flash-attention, see: #6052), and this applies to flash-infer as well?

Perhaps the first step is to unify the flash infer kernels to use a single kernel, similar to #6052, or at least clarify in what scenario it is ok to run flash-infer kernels for chunked prefill, because according to @yzh119 in flashinfer-ai/flashinfer#392, this should be supported by flash-infer already.

[jon-chuang]

Anw, please assign it to me, I will investigate further

[comaniac]

We are already working on this cc @Yard1

[pavanimajety]

@comaniac Any updates or open PRs on this that we can take a look at?

[taegeonum]

@comaniac Any updates?