Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync with upstream@v0.5.4-7-g9118217f #120

Closed
wants to merge 168 commits into from
Closed

Sync with upstream@v0.5.4-7-g9118217f #120

wants to merge 168 commits into from

Conversation

dtrifiro
Copy link

@dtrifiro dtrifiro commented Aug 5, 2024 •
edited by github-actions bot
Loading

mgoin and others added 30 commits July 23, 2024 14:04
@mgoin
[Misc] Add ignored layers for fp8 quantization (vllm-project#6657)
0eb0757
@yecohn
[Frontend] Add Usage data in each chunk for chat_serving. vllm-projec…
58f5303 
…t#6540 (vllm-project#6652)
@tjohnson31415
[Model] Pipeline Parallel Support for DeepSeek v2 (vllm-project#6519)
507ef78 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
@ywang96
Bump transformers version for Llama 3.1 hotfix and patch Chameleon (v…
1bedf21 
…llm-project#6690)
@youkaichao
[build] relax wheel size limit (vllm-project#6704)
72fc704
@mgoin
[CI] Add smoke test for non-uniform AutoFP8 quantization (vllm-projec…
01c16ed 
…t#6702)
@tdoublep
[Bugfix] StatLoggers: cache spec decode metrics when they get collect…
2f808e6 
…ed. (vllm-project#6645)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@thesues @mgoin
[bitsandbytes]: support read bnb pre-quantized model (vllm-project#5753)
87525fa 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
@SolitaryThinker
[Bugfix] fix flashinfer cudagraph capture for PP (vllm-project#6708)
5e8ca97
@njhill
[SpecDecoding] Update MLPSpeculator CI tests to use smaller model (vl…
c882a7f 
…lm-project#6714)
@ywang96
[Bugfix] Fix token padding for chameleon (vllm-project#6724)
0a740a1
@WoosukKwon
[Docs][ROCm] Detailed instructions to build from source (vllm-project…
ccc4a73 
…#6680)
@Alexei-V-Ivanov-AMD
[Build/CI] Update run-amd-test.sh. Enable Docker Hub login. (vllm-pro…
b570811 
…ject#6711)
@liuyhwangyh
[Bugfix]fix modelscope compatible issue (vllm-project#6730)
f4f8a9d
@luizanao
Adding f-string to validation error which is missing (vllm-project#6748)
5451463
@njhill
[Bugfix] Fix speculative decode seeded test (vllm-project#6743)
2cf0df3
@AllenDou
[Bugfix] Miscalculated latency lead to time_to_first_token_seconds in…
40468b1 
…accurate. (vllm-project#6686)
@dtrifiro
[Frontend] split run_server into build_server and run_server (vllm-pr…
ee81258 
…oject#6740)
@Yard1
Add fp8 support to reshape_and_cache_flash (vllm-project#6667)
0e63494
@Yard1
[Core] Tweaks to model runner/input builder developer APIs (vllm-proj…
5448f67 
…ect#6712)
@mgoin
[Bugfix] Bump transformers to 4.43.2 (vllm-project#6752)
421e218
@hongxiayang
[Doc][AMD][ROCm]Added tips to refer to mi300x tuning guide for mi300x…
d88c458 
… users (vllm-project#6754)
@youkaichao
[core][distributed] fix zmq hang (vllm-project#6759)
740374d
@ezliu
[Frontend] Represent tokens with identifiable strings (vllm-project#6626
5689e25 
)
@HwwwwwwwH
[Model] Adding support for MiniCPM-V (vllm-project#4087)
9e169a4
@comaniac
[Bugfix] Fix decode tokens w. CUDA graph (vllm-project#6757)
309aaef
@alexm-neuralmagic
[Bugfix] Fix awq_marlin and gptq_marlin flags (vllm-project#6745)
0310029
@CatherineSue
[Bugfix] Fix encoding_format in examples/openai_embedding_client.py (v…
316a41a 
…llm-project#6755)
@HwwwwwwwH @DarkLight1337
[Bugfix] Add image placeholder for OpenAI Compatible Server of MiniCP…
b75e314 
…M-V (vllm-project#6787)

Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@robertgshaw2-neuralmagic @mgoin
[ Misc ] fp8-marlin channelwise via compressed-tensors (vllm-proj…
889da13 
…ect#6524)

Co-authored-by: mgoin <michael@neuralmagic.com>
Isotr0py and others added 20 commits August 2, 2024 22:36
@Isotr0py
[Model] Refactor and decouple weight loading logic for InternVL2 model (
0c25435 
vllm-project#7067)
@zachzzc
[Bugfix] Fix block table for seqs that have prefix cache hits (vllm-p…
fb2c1c8 
…roject#7018)
@jeejeelee
[LoRA] ReplicatedLinear support LoRA (vllm-project#7081)
99d7cab
@KuntaiDu
[CI] Temporarily turn off H100 performance benchmark (vllm-project#7104)
67d745c
@youkaichao
[ci][test] finalize fork_new_process_for_each_test (vllm-project#7114)
44dcb52
@fialhocoelho @njhill
[Frontend] Warn if user max_model_len is greater than derived `max_…
825b044 
…model_len` (vllm-project#7080)

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
@kevinbu233 @DarkLight1337
Support for guided decoding for offline LLM (vllm-project#6878)
654bc5c 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@youkaichao
[misc] add zmq in collect env (vllm-project#7119)
9fadc7b
@youkaichao
[core][misc] simply output processing with shortcut code path (vllm-p…
83c644f 
…roject#7117)
@jeejeelee @DarkLight1337
[Model]Refactor MiniCPMV (vllm-project#7020)
179a6a3 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@tdoublep
[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size …
b1c9aa3 
…to 1 when using MLPSpeculator (vllm-project#7105)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@youkaichao
[misc][distributed] improve libcudart.so finding (vllm-project#7127)
16a1cc9
@jeejeelee
Clean up remaining Punica C information (vllm-project#7027)
f80ab35
@HwwwwwwwH @DarkLight1337
[Model] Add multi-image support for minicpmv (vllm-project#7122)
7b86e7c 
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@DarkLight1337
[Frontend] Reapply "Factor out code for running uvicorn" (vllm-projec…
cc08fc7 
…t#7095)
@ChristopherCho @ywang96
[Model] SiglipVisionModel ported from transformers (vllm-project#6942)
c0d8f16 
Co-authored-by: Roger Wang <ywang@roblox.com>
@cadedaniel
[Speculative decoding] Add periodic log with time spent in proposal/s…
82a1b1a 
…coring/verification (vllm-project#6963)
@bong-furiosa
[SpecDecode] Support FlashInfer in DraftModelRunner (vllm-project#6926)
e963045
@njhill
[BugFix] Use IP4 localhost form for zmq bind (vllm-project#7163)
003f8ee
@dtrifiro
Sync with upstream@0.5.4 (pre)
5efb049
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Aug 5, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Aug 5, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtrifiro

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved label Aug 5, 2024
@github-actions github-actions bot changed the title sync with 0.5.4 Sync with upstream@v0.5.4-7-g9118217f Aug 6, 2024
@dtrifiro dtrifiro closed this Aug 6, 2024
@dtrifiro dtrifiro deleted the sync-with-0.5.4 branch August 6, 2024 09:00
prarit pushed a commit to prarit/vllm that referenced this pull request Oct 18, 2024
@charlifu
add emtpy_cache() after each padding (opendatahub-io#120)
98f31cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

Assignees
No one assigned
Labels
approved do-not-merge/work-in-progress ok-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.