单端口部署多模型最简单解决方案（vllm sglang 等均适用）
TLDR将不同模型部署在不同端口使用litellm的proxy功能统一管理转发步骤1本地部署好模型在若干端口上vllm serve Qwen/Qwen3.5-2B --port 8031 vllm serve Qwen/Qwen3.5-4B --port 8032 vllm serve Qwen/Qwen3.5-9B --port 8033步骤2安装配置litellm安装# 选项1使用uv安装 uv tool install litellm[proxy] # 选项2脚本直接安装 curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh配置litellm当前目录创建litellm_config.yaml写入以下内容model_list: - model_name: Qwen/Qwen3.5-4B litellm_params: model: openai/Qwen/Qwen3.5-4B # add openai/ prefix to route as OpenAI provider api_base: http://localhost:8031/v1 api_key: none - model_name: Qwen/Qwen3.5-2B litellm_params: model: openai/Qwen/Qwen3.5-2B # add openai/ prefix to route as OpenAI provider api_base: http://localhost:8032/v1 api_key: none - model_name: Qwen/Qwen3.5-9B litellm_params: model: openai/Qwen/Qwen3.5-9B # add openai/ prefix to route as OpenAI provider api_base: http://localhost:8033/v1 api_key: none步骤3用你需要的端口启动litellmlitellm --config ./litellm_config.yaml --port 8088然后如下之后直接使用8088端口就可以访问部署的三个模型了INFO: Started server process [214508] INFO: Waiting for application startup. ██╗ ██╗████████╗███████╗██╗ ██╗ ███╗ ███╗ ██║ ██║╚══██╔══╝██╔════╝██║ ██║ ████╗ ████║ ██║ ██║ ██║ █████╗ ██║ ██║ ██╔████╔██║ ██║ ██║ ██║ ██╔══╝ ██║ ██║ ██║╚██╔╝██║ ███████╗██║ ██║ ███████╗███████╗███████╗██║ ╚═╝ ██║ ╚══════╝╚═╝ ╚═╝ ╚══════╝╚══════╝╚══════╝╚═╝ ╚═╝ Thank you for using LiteLLM! - Krrish Ishaan LiteLLM: Proxy initialized with Config, Set models: Qwen/Qwen3.5-4B Qwen/Qwen3.5-2B Qwen/Qwen3.5-9B INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8088 (Press CTRLC to quit)
