Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[distributed] remove pynccl's redundant change_state #11749

Merged
merged 1 commit into from
Jan 6, 2025
Merged

[distributed] remove pynccl's redundant change_state #11749

merged 1 commit into from
Jan 6, 2025

Conversation

cennn
Copy link
Contributor

@cennn cennn commented Jan 5, 2025 •
edited by github-actions bot
Loading

I've optimized the PyNcclCommunicator codebase by eliminating the redundant function change_state.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 5, 2025

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

youkaichao
youkaichao approved these changes Jan 5, 2025
View reviewed changes
Copy link
Member

@youkaichao youkaichao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for cleaning it up!

@youkaichao
Copy link
Member

also cc @tlrmchlsmth

@youkaichao youkaichao enabled auto-merge (squash) January 5, 2025 16:16
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 5, 2025
@youkaichao
Copy link
Member

failing tests are unrelated.

@youkaichao youkaichao disabled auto-merge January 6, 2025 01:05
@youkaichao youkaichao merged commit 9e764e7 into vllm-project:main Jan 6, 2025
66 of 72 checks passed
gshtras added a commit to ROCm/vllm that referenced this pull request Jan 8, 2025
@gshtras
Revert "[distributed] remove pynccl's redundant change_state (vllm-pr…
9be84c0 
…oject#11749)"

This reverts commit 9e764e7.
This was referenced Jan 8, 2025
Merged
Merged
gshtras added a commit to ROCm/vllm that referenced this pull request Jan 8, 2025
@gshtras
Revert nccl changes (#351)
c040f0e 
* Revert "[distributed] remove pynccl's redundant change_state (vllm-project#11749)"

This reverts commit 9e764e7.

* Revert "[distributed] remove pynccl's redundant stream (vllm-project#11744)"

This reverts commit 635b897.
hongxiayang pushed a commit to ROCm/vllm that referenced this pull request Jan 15, 2025
@tjtanaa @jeejeelee
@terrytangyuan @DarkLight1337 @ruisearch42 @Jeffwan @lucas-tucker @ywang96 @sroy745 @WoosukKwon @Isotr0py @robertgshaw2-redhat @mgoin @simon-mo @HandH1998 @youkaichao @MengqingCao @AlexHe99 @Chen-0210 @ErezSC42 @selalipop @heheda12345 @rajveerb @hj-wei @KuntaiDu @liangfu @tlrmchlsmth @bigPYJ1151 @whyiug @khluu @ayylemao @JohnGiorgi @sakunkun @ApostaC @joerunde @houseroad @serihiro @CloseChoice @chunyang-wen @kathyyu-google @bjmsong @mdattack @wchen61 @nathan-az @SachinVarghese @aurickq @sfc-gh-aqiao @zinccat @WangErXiao @Bryce1010 @bet0x @jikunshang @xcnick @yanburman @cennn @RuixiangMa @comaniac @surajssd @gshtras @Concurrensee @ys950902 @Abatom @XiaobingSuper @russellb @zhouyuan @jiangjiadi @DamonFool @hmellor @divakar-amd @npanpaliya @ilya-lavrenov @wallashss @yma11 @rasmith @maxdebayser @mfournioux @guspan-tanadi @yeqcharlotte @wangxiyuan @charlesfrye @remimin @tjohnson31415 @frreiss @llsj14 @shaochangxu @NickLucche @sixsixcoder @rafvasq @Akshat-Tripathi @mosalov @avshalomman @yyccli @liaoyanqing666 @noemotiovon @alex-jw-brooks @shen-shanshan @e1ijah1 @Alexei-V-Ivanov-AMD @vllmellm
[MFM-20250115] Merge from ROCm/main to llama_fp8 (#360)
d9385b4 
* [Misc] Move weights mapper (vllm-project#11443)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Fix issues in CPU build Dockerfile. Fixes vllm-project#9182 (vllm-project#11435)

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* [Model] Automatic conversion of classification and reward models (vllm-project#11469)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (vllm-project#11472)

* [Misc] Update disaggregation benchmark scripts and test logs (vllm-project#11456)

Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>

* [Frontend] Enable decord to load video from base64 (vllm-project#11492)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Improve GitHub links (vllm-project#11491)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc] Move some multimodal utils to modality-specific modules (vllm-project#11494)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* Mypy checking for vllm/compilation (vllm-project#11496)

Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

* [Misc][LoRA] Fix LoRA weight mapper (vllm-project#11495)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Doc] Add `QVQ` and `QwQ` to the list of supported models (vllm-project#11509)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

* [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (vllm-project#10681)

Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Model]  Modify MolmoForCausalLM MLP  (vllm-project#11510)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Add placeholder module (vllm-project#11501)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Add video example to openai client for multimodal (vllm-project#11521)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [1/N] API Server  (Remove Proxy) (vllm-project#11529)

* [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (vllm-project#11523)

Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

* [2/N] API Server: Avoid ulimit footgun (vllm-project#11530)

* Deepseek v3 (vllm-project#11502)

Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>

* [Docs] Document Deepseek V3 support (vllm-project#11535)

Signed-off-by: simon-mo <simon.mo@hey.com>

* Update openai_compatible_server.md (vllm-project#11536)

Co-authored-by: Simon Mo <simon.mo@hey.com>

* [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (vllm-project#11394)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [V1] Fix yapf (vllm-project#11538)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [CI] Fix broken CI (vllm-project#11543)

* [misc] fix typing (vllm-project#11540)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (vllm-project#11534)

* [BugFix] Fix quantization for all other methods (vllm-project#11547)

* [Platform] Move model arch check to platform (vllm-project#11503)

Signed-off-by: Mengqing Cao <cmq0113@163.com>

* Update deploying_with_k8s.md with AMD ROCm GPU example (vllm-project#11465)

Signed-off-by: Alex He <alehe@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Bugfix] Fix TeleChat2ForCausalLM weights mapper (vllm-project#11546)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Abstract the logic for reading and writing media content (vllm-project#11527)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc]  Add xgrammar in doc (vllm-project#11549)

Signed-off-by: ccjincong <chenjincong11@gmail.com>

* [VLM] Support caching in merged multi-modal processor (vllm-project#11396)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [MODEL] LoRA support for Jamba model (vllm-project#11209)

Signed-off-by: Erez Schwartz <erezs@ai21.com>

* [Misc]Add BNB quantization for MolmoForCausalLM  (vllm-project#11551)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (vllm-project#11566)

Signed-off-by: Isotr0py <2037008807@qq.com>

* [Bugfix] Fix for ROCM compressed tensor support (vllm-project#11561)

* [Doc] Update mllama example based on official doc (vllm-project#11567)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [V1] [4/N] API Server: ZMQ/MP Utilities (vllm-project#11541)

* [Bugfix] Last token measurement fix (vllm-project#11376)

Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

* [Model] Support InternLM2 Reward models (vllm-project#11571)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Model] Remove hardcoded image tokens ids from Pixtral (vllm-project#11582)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (vllm-project#11515)

Signed-off-by: hjwei <hjwei_xd@163.com>

* [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (vllm-project#11581)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Doc] Minor documentation fixes (vllm-project#11580)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [bugfix] interleaving sliding window for cohere2 model (vllm-project#11583)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [V1] [5/N] API Server: unify `Detokenizer` and  `EngineCore` input (vllm-project#11545)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [Doc] Convert list tables to MyST (vllm-project#11594)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [v1][bugfix] fix cudagraph with inplace buffer assignment (vllm-project#11596)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Misc] KV cache transfer connector registry (vllm-project#11481)

Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

* Remove print statement in DeepseekScalingRotaryEmbedding (vllm-project#11604)

* [v1] fix compilation cache (vllm-project#11598)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Docker] bump up neuron sdk v2.21 (vllm-project#11593)

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

* [Build][Kernel] Update CUTLASS to v3.6.0 (vllm-project#11607)

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

* [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (vllm-project#11618)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [platforms] enable platform plugins (vllm-project#11602)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [VLM] Abstract out multi-modal data parsing in merged processor (vllm-project#11620)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [V1] [6/N] API Server: Better Shutdown (vllm-project#11586)

* [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (vllm-project#11631)

* [benchmark] Remove dependency for H100 benchmark step (vllm-project#11572)

* [Model][LoRA]LoRA support added for MolmoForCausalLM (vllm-project#11439)

Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Fix OpenAI parallel sampling when using xgrammar (vllm-project#11637)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (vllm-project#6909)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (vllm-project#11565)

* [V1] Simpify vision block hash for prefix caching by removing offset from hash (vllm-project#11646)

* [V1][VLM] V1 support for selected single-image models. (vllm-project#11632)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Benchmark] Add benchmark script for CPU offloading  (vllm-project#11533)

Signed-off-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: KuntaiDu <kuntai@uchicago.edu>

* [Bugfix][Refactor] Unify model management in frontend (vllm-project#11660)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

* [VLM] Add max-count checking in data parser for single image models (vllm-project#11661)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

* [Misc] Optimize Qwen2-VL LoRA test (vllm-project#11663)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Misc] Replace space with - in the file names (vllm-project#11667)

Signed-off-by: Lu Fang <lufang@fb.com>

* [Doc] Fix typo (vllm-project#11666)

Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>

* [V1] Implement Cascade Attention (vllm-project#11635)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [VLM] Move supported limits and max tokens to merged multi-modal processor (vllm-project#11669)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (vllm-project#11674)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [mypy] Pass type checking in vllm/inputs (vllm-project#11680)

Signed-off-by: Tobias Pitters <tobias.pitters@gmail.com>

* [VLM] Merged multi-modal processor for LLaVA-NeXT (vllm-project#11682)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* According to vllm.EngineArgs, the name should be distributed_executor_backend (vllm-project#11689)

* [Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (vllm-project#10013)

Signed-off-by: Kathy Yu <feiyangyu@google.com>

* [V1][Minor] Optimize token_ids_cpu copy (vllm-project#11692)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Bugfix] Change kv scaling factor by param json on nvidia gpu (vllm-project#11688)

Signed-off-by: bjmsong <bjmsong@126.com>
Co-authored-by: bjmsong <bjmsong@126.com>

* Resolve race conditions in Marlin kernel (vllm-project#11493)

Signed-off-by: wchen61 <wchen61@foxmail.com>

* [Misc] Minimum requirements for SageMaker compatibility (vllm-project#11576)

* Update default max_num_batch_tokens for chunked prefill (vllm-project#11694)

* [Bugfix] Check chain_speculative_sampling before calling it (vllm-project#11673)

Signed-off-by: Lu Fang <lufang@fb.com>

* [perf-benchmark] Fix dependency for steps in benchmark pipeline (vllm-project#11710)

* [Model] Whisper model implementation (vllm-project#11280)

Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>

* [V1] Simplify Shutdown (vllm-project#11659)

* [Bugfix] Fix ColumnParallelLinearWithLoRA slice (vllm-project#11708)

Signed-off-by: ZincCat <zincchloride@outlook.com>

* [V1] Improve TP>1 Error Handling + Stack Trace (vllm-project#11721)

Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

* [Misc]Add BNB quantization for Qwen2VL (vllm-project#11719)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* Update requirements-tpu.txt to support python 3.9 and 3.11 (vllm-project#11695)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [V1] Chore: cruft removal (vllm-project#11724)

* [V1] log GPU blocks num for MultiprocExecutor (vllm-project#11656)

* Update tool_calling.md (vllm-project#11701)

* Update bnb.md with example for OpenAI (vllm-project#11718)

* [V1] Add `RayExecutor` support for `AsyncLLM` (api server) (vllm-project#11712)

* [V1] Add kv cache utils tests. (vllm-project#11513)

Signed-off-by: xcnick <xcnick0412@gmail.com>

* [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (vllm-project#11233)

Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>

* [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (vllm-project#11717)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix precision error in LLaVA-NeXT (vllm-project#11735)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Model] Remove unnecessary weight initialization logic (vllm-project#11736)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Bugfix][V1] Fix test_kv_cache_utils.py (vllm-project#11738)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [MISC] Replace c10::optional with std::optional (vllm-project#11730)

Signed-off-by: Lu Fang <lufang@fb.com>

* [distributed] remove pynccl's redundant stream (vllm-project#11744)

* fix: [doc] fix typo (vllm-project#11751)

Co-authored-by: Lancer <maruixiang6688@gmail.com>

* [Frontend] Improve `StreamingResponse` Exception Handling (vllm-project#11752)

* [distributed] remove pynccl's redundant change_state (vllm-project#11749)

* [Doc] [1/N] Reorganize Getting Started section (vllm-project#11645)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Remove block size constraint (vllm-project#11723)

* [V1] Add BlockTable class (vllm-project#11693)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Misc] Fix typo for valid_tool_parses  (vllm-project#11753)

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

* [V1] Refactor get_executor_cls (vllm-project#11754)

* [mypy] Forward pass function type hints in lora (vllm-project#11740)

Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

* k8s-config: Update the secret to use stringData (vllm-project#11679)

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>

* [VLM] Separate out profiling-related logic (vllm-project#11746)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc][2/N] Reorganize Models and Usage sections (vllm-project#11755)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix max image size for LLaVA-Onevision (vllm-project#11769)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [doc] explain how to add interleaving sliding window support (vllm-project#11771)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* format

* [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (vllm-project#11685)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

* deepseek overflow fix (#349)

* [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm-project#11772)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#11776)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [XPU] Make pp group initilized for pipeline-parallelism (vllm-project#11648)

Signed-off-by: yisheng <yi.sheng@intel.com>

* [Doc][3/N] Reorganize Serving section (vllm-project#11766)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Kernel][LoRA]Punica prefill  kernels fusion (vllm-project#11234)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>

* [Bugfix] Update attention interface in `Whisper` (vllm-project#11784)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [CI] Fix neuron CI and run offline tests (vllm-project#11779)

Signed-off-by: Liangfu Chen <liangfc@amazon.com>

* fix init error for MessageQueue when n_local_reader is zero (vllm-project#11768)

* [Doc] Create a vulnerability management team (vllm-project#9925)

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* [CI][CPU] adding build number to docker image name (vllm-project#11788)

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* [V1][Doc] Update V1 support for `LLaVa-NeXT-Video` (vllm-project#11798)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (vllm-project#11800)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [doc] add doc to explain how to use uv (vllm-project#11773)

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [V1] Support audio language models on V1 (vllm-project#11733)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [doc] update how pip can install nightly wheels (vllm-project#11806)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Doc] Add note to `gte-Qwen2` models (vllm-project#11808)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [optimization] remove python function call for custom op (vllm-project#11750)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix] update the prefix for qwen2 (vllm-project#11795)

Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>

* [Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)

Signed-off-by: Sourashis Roy <sroy@roblox.com>

* [Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 (vllm-project#11794)

* [Doc] Group examples into categories (vllm-project#11782)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc] sort torch profiler table by kernel timing (vllm-project#11813)

* Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (vllm-project#11824)

* Fixed docker build for ppc64le (vllm-project#11518)

Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>

* [OpenVINO] Fixed Docker.openvino build (vllm-project#11732)

Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* [Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

* [Docs] reorganize sponsorship page (vllm-project#11639)

Signed-off-by: simon-mo <simon.mo@hey.com>

* [Bug] Fix pickling of `ModelConfig` when RunAI Model Streamer is used (vllm-project#11825)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [misc] improve memory profiling (vllm-project#11809)

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [doc] update wheels url (vllm-project#11830)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Docs] Update sponsor name: 'Novita' to 'Novita AI' (vllm-project#11833)

* [Hardware][Apple] Native support for macOS Apple Silicon (vllm-project#11696)

Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

* [torch.compile] consider relevant code in compilation cache (vllm-project#11614)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [VLM] Reorganize profiling/processing-related code (vllm-project#11812)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Move examples into categories (vllm-project#11840)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Doc][4/N] Reorganize API Reference (vllm-project#11843)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [CI/Build][Bugfix] Fix CPU CI image clean up (vllm-project#11836)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [Bugfix][XPU] fix silu_and_mul (vllm-project#11823)

Signed-off-by: yan ma <yan.ma@intel.com>

* [Misc] Move some model utils into vision file (vllm-project#11848)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Expand Multimodal API Reference (vllm-project#11852)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Misc]add some explanations for BlockHashType (vllm-project#11847)

* [TPU][Quantization] TPU `W8A8` (vllm-project#11785)

Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

* [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (vllm-project#11698)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

* [Docs] Add Google Cloud Meetup (vllm-project#11864)

* Revert nccl changes (#351)

* Revert "[distributed] remove pynccl's redundant change_state (vllm-project#11749)"

This reverts commit 9e764e7.

* Revert "[distributed] remove pynccl's redundant stream (vllm-project#11744)"

This reverts commit 635b897.

* [CI] Turn on basic correctness tests for V1 (vllm-project#10864)

* treat do_lower_case in the same way as the sentence-transformers library (vllm-project#11815)

Signed-off-by: Max de Bayser <mbayser@br.ibm.com>

* [Doc] Recommend uv and python 3.12 for quickstart guide (vllm-project#11849)

Signed-off-by: mgoin <michael@neuralmagic.com>

* [Misc] Move `print_*_once` from utils to logger (vllm-project#11298)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

* [Doc] Intended links Python multiprocessing library (vllm-project#11878)

* [perf]fix current stream (vllm-project#11870)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix] Override dunder methods of placeholder modules (vllm-project#11882)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] fix beam search input errors and latency benchmark script (vllm-project#11875)

Signed-off-by: Ye Qi <yeq@meta.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>

* [Doc] Add model development API Reference (vllm-project#11884)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [platform] Allow platform specify attention backend (vllm-project#11609)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

* [ci]try to fix flaky multi-step tests (vllm-project#11894)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Misc] Provide correct Pixtral-HF chat template (vllm-project#11891)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* fp8 support (#352)

Co-authored-by: Yida Wu <yidawu@amd.com>

* [Docs] Add Modal to deployment frameworks (vllm-project#11907)

* [Doc][5/N] Move Community and API Reference to the bottom (vllm-project#11896)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Simon Mo <simon.mo@hey.com>

* [VLM] Enable tokenized inputs for merged multi-modal processor (vllm-project#11900)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Show default pooling method in a table (vllm-project#11904)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [torch.compile] Hide KV cache behind torch.compile boundary (vllm-project#11677)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [Bugfix] Validate lora adapters to avoid crashing server (vllm-project#11727)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* [BUGFIX] Fix `UnspecifiedPlatform` package name (vllm-project#11916)

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

* [ci] fix gh200 tests (vllm-project#11919)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [misc] remove python function call for custom activation op (vllm-project#11885)

Co-authored-by: youkaichao <youkaichao@gmail.com>

* [platform] support pytorch custom op pluggable (vllm-project#11328)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

* Replace "online inference" with "online serving" (vllm-project#11923)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [ci] Fix sampler tests (vllm-project#11922)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Doc] [1/N] Initial guide for merged multi-modal processor (vllm-project#11925)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [platform] support custom torch.compile backend key (vllm-project#11318)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

* [Doc] Rename offline inference examples (vllm-project#11927)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [Docs] Fix docstring in `get_ip` function (vllm-project#11932)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>

* Doc fix in `benchmark_long_document_qa_throughput.py` (vllm-project#11933)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>

* [Hardware][CPU] Support MOE models on x86 CPU (vllm-project#11831)

Signed-off-by: jiang1.li <jiang1.li@intel.com>

* [Misc] Clean up debug code in Deepseek-V3 (vllm-project#11930)

Signed-off-by: Isotr0py <2037008807@qq.com>

* [Misc] Update benchmark_prefix_caching.py fixed example usage (vllm-project#11920)

Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>

* [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (vllm-project#11939)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

* [mypy] Fix mypy warnings in api_server.py (vllm-project#11941)

Signed-off-by: Fred Reiss <frreiss@us.ibm.com>

* [ci] fix broken distributed-tests-4-gpus (vllm-project#11937)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (vllm-project#11672)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

* [Bugfix] fused_experts_impl wrong compute type for float32 (vllm-project#11921)

Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>

* [CI/Build] Move model-specific multi-modal processing tests (vllm-project#11934)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Doc] Basic guide for writing unit tests for new models (vllm-project#11951)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

* [Bugfix] Fix RobertaModel loading (vllm-project#11940)

Signed-off-by: NickLucche <nlucches@redhat.com>

* [Model] Add cogagent model support vLLM (vllm-project#11742)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [V1] Avoid sending text prompt to core engine (vllm-project#11963)

Signed-off-by: Roger Wang <ywang@roblox.com>

* [CI/Build] Add markdown linter (vllm-project#11857)

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>

* [Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [Hardware][CPU] Multi-LoRA implementation for the CPU backend (vllm-project#11100)

Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

* [Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)

* [V1][Core][1/n] Logging and Metrics (vllm-project#11962)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [Model] Support GGUF models newly added in `transformers` 4.46.0 (vllm-project#9685)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

* [V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (vllm-project#11973)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

* [MISC] fix typo in kv transfer send recv test (vllm-project#11983)

* [Bug] Fix usage of `.transpose()` and `.view()` consecutively. (vllm-project#11979)

* [CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>

* [Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)

Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>

* [Misc]Minor Changes about Worker (vllm-project#11555)

Signed-off-by: Chenguang Li <757486878@qq.com>

* [platform] add ray_device_key (vllm-project#11948)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)

Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>

* [Kernel] unified_attention for Attention.forward (vllm-project#11967)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

* [Doc][V1] Update model implementation guide for V1 support (vllm-project#11998)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

* [Doc] Organise installation documentation into categories and tabs (vllm-project#11935)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

* [platform] add device_control env var (vllm-project#12009)

Signed-off-by: youkaichao <youkaichao@gmail.com>

* [Platform] Move get_punica_wrapper() function to Platform (vllm-project#11516)

Signed-off-by: Shanshan Shen <467638484@qq.com>

* bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (vllm-project#11982)

Signed-off-by: elijah <f1renze.142857@gmail.com>

* Using list

* Revert "[misc] improve memory profiling (vllm-project#11809)"

This reverts commit 889e662.

* Multi-lingual P3L (#356)

* Commiting the *multilingual* P3L test.

* Created a *multi-lingual* P3L test.

* Making ruff happy.

* .

* Added a reference to the language-scripture Confluence table.

* Typo fixing.

* Harmonizing naming.

* Fixing comments in the header.

---------

Co-authored-by: Alexei V. Ivanov <alivanov@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

* Trying to make scales work with compileable attention

* Docs lint

* linter formatting bug fixes

* inherit config file updates under fused_moe from main branch.

* match tests for the MOE layers with main.

---------

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Alex He <alehe@amd.com>
Signed-off-by: ccjincong <chenjincong11@gmail.com>
Signed-off-by: Erez Schwartz <erezs@ai21.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Signed-off-by: hjwei <hjwei_xd@163.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
Signed-off-by: Tobias Pitters <tobias.pitters@gmail.com>
Signed-off-by: Kathy Yu <feiyangyu@google.com>
Signed-off-by: bjmsong <bjmsong@126.com>
Signed-off-by: wchen61 <wchen61@foxmail.com>
Signed-off-by: ZincCat <zincchloride@outlook.com>
Signed-off-by: xcnick <xcnick0412@gmail.com>
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
Signed-off-by: yisheng <yi.sheng@intel.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Signed-off-by: Ye Qi <yeq@meta.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Fred Reiss <frreiss@us.ibm.com>
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Chenguang Li <757486878@qq.com>
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Shanshan Shen <467638484@qq.com>
Signed-off-by: elijah <f1renze.142857@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yuan Tang <terrytangyuan@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Co-authored-by: Jiaxin Shan <seedjeffwan@gmail.com>
Co-authored-by: Lucas Tucker <47258766+lucas-tucker@users.noreply.github.com>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: sroy745 <142070531+sroy745@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: AlexHe99 <alehe@amd.com>
Co-authored-by: Chen1022 <112855051+ccjincong@users.noreply.github.com>
Co-authored-by: ErezSC42 <erezs@ai21.com>
Co-authored-by: Selali <selali.adobor@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Rajveer Bachkaniwala <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: hj-wei <hjwei_xd@163.com>
Co-authored-by: Kuntai Du <kuntai@uchicago.edu>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Co-authored-by: whyiug <whyiug@hotmail.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
Co-authored-by: Matthias Vogler <60004995+ayylemao@users.noreply.github.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: John Giorgi <johnmgiorgi@gmail.com>
Co-authored-by: sakunkun <zhou.qianjun@zte.com.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Kazuhiro Serizawa <nserihiro@gmail.com>
Co-authored-by: Tobias Pitters <31857876+CloseChoice@users.noreply.github.com>
Co-authored-by: Chunyang Wen <chunyang.wen@gmail.com>
Co-authored-by: Kathy Yu <143133934+kathyyu-google@users.noreply.github.com>
Co-authored-by: bjmsong <wq.songbob@gmail.com>
Co-authored-by: bjmsong <bjmsong@126.com>
Co-authored-by: wchen61 <wchen61@foxmail.com>
Co-authored-by: Nathan Azrak <42650258+nathan-az@users.noreply.github.com>
Co-authored-by: Sachin Varghese <sachin.mathew31@gmail.com>
Co-authored-by: Aurick Qiao <aurickq@users.noreply.github.com>
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
Co-authored-by: ZincCat <52513999+zinccat@users.noreply.github.com>
Co-authored-by: WangErXiao <863579016@qq.com>
Co-authored-by: Hust_YangXian <bryceyx@gmail.com>
Co-authored-by: Alberto Ferrer <albertof@barrahome.org>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: xcnick <xcnick0412@gmail.com>
Co-authored-by: Yan Burman <yanburman@users.noreply.github.com>
Co-authored-by: cennn <61925104+cennn@users.noreply.github.com>
Co-authored-by: Lancer <402430575@qq.com>
Co-authored-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Suraj Deshmukh <surajd.service@gmail.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: YiSheng5 <yi.sheng@intel.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
Co-authored-by: XiaobingZhang <xiaobingzhangupc@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Yuan <yuan.zhou@intel.com>
Co-authored-by: jiangjiadi <34134495+jiangjiadi@users.noreply.github.com>
Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
Co-authored-by: Jie Fu (傅杰) <jiefu@tencent.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Nishidha <nishidha.panpaliya@partner.ibm.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Wallas Henrique <wallashss@users.noreply.github.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: rasmith <Randall.Smith@amd.com>
Co-authored-by: Maximilien de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Guspan Tanadi <36249910+guspan-tanadi@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <ye.charlotte.qi@gmail.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: Yida Wu <yidawu@amd.com>
Co-authored-by: Charles Frye <cfrye59@gmail.com>
Co-authored-by: minmin <rmm0811@gmail.com>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Fred Reiss <frreiss@us.ibm.com>
Co-authored-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: shaochangxu <85155497+shaochangxu@users.noreply.github.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: sixgod <evethwillbeok@outlook.com>
Co-authored-by: Rafael Vasquez <rafvasq21@gmail.com>
Co-authored-by: Akshat Tripathi <Akshat.tripathi6568@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Avshalom Manevich <12231371+avshalomman@users.noreply.github.com>
Co-authored-by: Yangcheng Li <liyangcheng.lyc@alibaba-inc.com>
Co-authored-by: Siyuan Li <94890248+liaoyanqing666@users.noreply.github.com>
Co-authored-by: Chenguang Li <757486878@qq.com>
Co-authored-by: Alex Brooks <alex.brooks@ibm.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: elijah <30852919+e1ijah1@users.noreply.github.com>
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
Co-authored-by: Alexei V. Ivanov <alivanov@banff-cyxtera-s65-4.amd.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@youkaichao youkaichao youkaichao approved these changes

@tlrmchlsmth tlrmchlsmth tlrmchlsmth approved these changes

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cennn @youkaichao @tlrmchlsmth