sglang
约 1 分钟阅读
- sgl-project/sglangGitHubsgl-project/sglangsgl-project/sglang · Apache-2.0, Python, C++ · 参考 · https://github.com/sgl-project/sglang/tree/main/docs/supportedmodels笔记：sglang
- Apache-2.0, Python, C++
- 参考
uv pip install "sglang[all]>=0.4.9.post6" --prerelease=allow
uv run python3 -m sglang.launch_server \
--model-path Qwen/Qwen2.5-VL-7B-Instruct \
--host 0.0.0.0 \
--port 30000Server
FAQ
RuntimeError: SGLang only supports sm75 and above.
- sm75 是 CUDA 7.5 的简称
- 需要 CUDA 7.5+ / 2018 Nvidia Turing+ 架构
- 不会支持 70 sgl-project/sglang#931GitHub Issuesgl-project/sglang#931
FlashInfer
Python: 3.8, 3.9, 3.10, 3.11
PyTorch: 2.2/2.3/2.4 with CUDA 11.8/12.1/12.4 (only for torch 2.4)
Supported GPU architectures: sm75, sm80, sm86, sm89, sm90关联信息
反向链接、本文链接的其他页面和外部资料。
参考资料
GitHub3 条
- sgl-project/sglanggithub.com/sgl-project/sglang
- sgl-project/sglang/docs/supported_modelsgithub.com/sgl-project/sglang/tree/main/docs/supported_models
- sgl-project/sglang#931github.com/sgl-project/sglang/issues/931
其他外链1 条
- docs.sglang.ai/backend/server_arguments.htmldocs.sglang.ai/backend/server_arguments.html