Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add check_env #645

Merged
merged 1 commit into from
Jul 18, 2024
Merged

feat: add check_env #645

merged 1 commit into from
Jul 18, 2024

Conversation

zhyncs
Copy link
Member

@zhyncs zhyncs commented Jul 18, 2024

Motivation

as titled cc @merrymercy @Ying1123 @hnyls2002

python3 -m sglang.check_env
Python: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
CUDA available: True
GPU 0: NVIDIA A100-SXM4-80GB
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.2, V12.2.140
CUDA Driver Version: 535.54.03
PyTorch: 2.3.0+cu121
sglang: 0.1.21
flashinfer: 0.0.9+cu121torch2.3
aiohttp: 3.9.5
fastapi: 0.111.0
hf_transfer: 0.1.6
huggingface_hub: 0.23.4
interegular: 0.3.3
packaging: 24.1
pillow: Module Not Found
psutil: 6.0.0
pydantic: 2.8.2
rpyc: 6.0.0
uvicorn: 0.30.1
uvloop: 0.19.0
zmq: 26.0.3
vllm: 0.5.1
openai: 1.35.13
anthropic: 0.31.1
NVIDIA Topology:
	GPU0	NIC0	NIC1	NIC2	NIC3	NIC4	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	SYS	SYS	SYS	PXB	NODE	32-63,96-127	1		N/A
NIC0	SYS	 X 	NODE	NODE	SYS	SYS
NIC1	SYS	NODE	 X 	NODE	SYS	SYS
NIC2	SYS	NODE	NODE	 X 	SYS	SYS
NIC3	PXB	SYS	SYS	SYS	 X 	NODE
NIC4	NODE	SYS	SYS	SYS	NODE	 X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_bond_0
  NIC1: mlx5_bond_1
  NIC2: mlx5_bond_2
  NIC3: mlx5_bond_3
  NIC4: mlx5_bond_4

Modification

as titled

Checklist

  1. Ensure pre-commit or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@Ying1123 Ying1123 merged commit d93388d into sgl-project:main Jul 18, 2024
@zhyncs zhyncs deleted the check-env branch July 18, 2024 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants