Skip to content

[Speculative decoding] Add serving benchmark for llama3 70b + speculative decoding#6964

Merged
cadedaniel merged 2 commits intovllm-project:mainfrom cadedaniel:spec-serving-testJul 31, 2024
+22-1

Commits

Commits on Jul 30, 2024