adapt vllm distributed module to sglang

[yizhang2077]

Motivation

Modifications

Move vllm distributed module (v0.6.4.post1) to sglang, and current model is stilll using vllm.distributed module

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

What to do Next

• Install sgl-kernel for custom allreduce and replace vllm.distributed to sglang.srt.distributed
• Add tensorrt allreduce module

[zhyncs]

QQ why not use the v0.6.4.post1

[yizhang2077]

QQ why not use the v0.6.4.post1

I see pyproject.toml require vllm>=0.6.3.post1

[zhyncs]

This is to maintain compatibility, 0.6.3.post1 uses torch 2.4, and 0.6.4.post1 uses torch 2.5.1. The current main branch is compatible with both torch 2.4 and torch 2.5.1. Regarding the distributed part, I suggest directly updating to v0.6.4.post1

[yizhang2077]

This is to maintain compatibility, 0.6.3.post1 uses torch 2.4, and 0.6.4.post1 uses torch 2.5.1. The current main branch is compatible with both torch 2.4 and torch 2.5.1. Regarding the distributed part, I suggest directly updating to v0.6.4.post1

ok，currently it use 0.6.4.post1 as base

[zhyncs]

Overall LGTM just left some comments Thanks!

[zhyncs]

LGTM. I think we can merge it into remove-vllm-distributed and verify afterward. You can then create another PR from remove-vllm-distributed.

[zhyncs]

isort....................................................................Failed
- hook id: isort
- files were modified by this hook

Fixing /home/runner/work/sglang/sglang/python/sglang/srt/_custom_ops.py

[yizhang2077]

I think the test failed is caused by pr #2266 , @zhaochenyang20

[zhyncs]

LGTM Since it is not currently in use, I think it's safe to merge. More detailed testing and verification can be conducted when integrating the custom all-reduce CUDA kernel into sgl-kernel. cc @yizhang2077