Code

[Core] Optimize Async + Multi-step#8050

Code

Merged

comaniac merged 8 commits intovllm-project:mainvllm-project/vllm:mainfrom neuralmagic:async_multi_step_optneuralmagic/nm-vllm:async_multi_step_optCopy head branch name to clipboardSep 3, 2024

+326-248

Commits

Commits on Aug 31, 2024

• Optimize async + multi-step by making async fully async with respect to all operations

  alexm-neuralmagic

  committed
  dafa498

• format

  alexm-neuralmagic

  committed
  ca993c7

• cleanup

  alexm-neuralmagic

  committed
  f054d70

Commits on Sep 1, 2024

• fix tests

  alexm-neuralmagic

  committed
  98a55d7

• ping

  alexm-neuralmagic

  committed
  4474b12

Commits on Sep 2, 2024

• Improve asyncio queues append of request outputs

  alexm-neuralmagic

  committed
  904006a

Commits on Sep 3, 2024

• Cody's review comments

  alexm-neuralmagic

  committed
  3a8726a

• More Cody's comments

  alexm-neuralmagic

  committed
  997c525