[{"data":1,"prerenderedAt":50},["Reactive",2],{"news-story-news-1792384474854-r2jzu":3},{"id":4,"title_zh":5,"title_en":6,"summary_zh":7,"summary_en":8,"takeaways_zh":9,"takeaways_en":14,"content_zh":19,"content_en":20,"external_links":21,"source":28,"category":35,"impact_score":36,"views_count":37,"likes_count":38,"related_entities":39,"tags":43,"image_url":49},"news-1792384474854-r2jzu","vLLM 官方推出预编译字节码镜像与 Decision 3.0 服务：容器冷启动提速 18%，AMD MI325X 决策吞吐达 1.8 倍","vLLM Ships Precompiled Bytecode Images and Decision 3.0 Serving: Cold Start Accelerated by 18%, MI325X Decision Throughput Hits 1.8x","vLLM 官方推特宣布两项关键工程突破（X: 2111990039806914908 & 2112005673303339084）：在 v0.32.0 官方镜像中引入构建期 Python 字节码预编译（uv compile），使 H100 容器冷启动耗时缩短 14%~18%（省去约 18 秒且启动零重编译）；同时正式合入 Decision 3.0（D3）全系列 6 款结构化决策模型（0.6B 至 27B），通过 /v1/systemone 提供极速多模态选择与评分，在 AMD MI325X 单卡上支撑 64 并发的吞吐达原始基准的 1.8 倍。","The vLLM team announced two major milestones (X: 2111990039806914908 & 2112005673303339084): shipping precompiled Python bytecode via uv in v0.32.0 release images cuts cold-start container launch by 14–18% (~18s on NVIDIA H100) with zero runtime recompiles. Additionally, vLLM integrated the Decision 3.0 (D3) structured reasoning family (0.6B to 27B) into the /v1/systemone endpoint, ranking #1 on the Jev Decision Index and delivering 1.8x req/s on a single AMD MI325X under 64 concurrent clients.",[10,11,12,13],"vLLM 官方在发布镜像中引入构建期 Python 字节码预编译，使 H100 容器冷启动耗时缩短 14%~18%（省去约 18 秒）","正式合入 Decision 3.0（D3）全系列 6 款决策模型（0.6B 至 27B），通过 /v1/systemone 专有接口提供毫秒级多模态结构化判定","D3 模型在 Jev Decision Index 0.3.2 榜单上拿下文本 66.3 分与视觉 71.3 分双料第一","在单张 AMD Instinct MI325X 显卡上，面对 64 并发客户请求，D3 27B 的吞吐量达到原始服务的 1.8 倍",[15,16,17,18],"vLLM ships precompiled Python bytecode via uv in official images, reducing H100 cold starts by 14–18% (~18 seconds saved)","Natively serves the Decision 3.0 (D3) family (0.6B to 27B) via the /v1/systemone structured decision endpoint","D3 models rank #1 on the Jev Decision Index with 66.3 in text and 71.3 in vision","D3 27B achieves 1.8x the request throughput of standard checkpoint servers on a single AMD Instinct MI325X under 64 concurrent clients","### 核心背景与行业痛点\n在云原生无服务器（Serverless）GPU 扩缩容与高并发微调推理中，开发者面临两大落地痛点：第一，**容器冷启动时间严重滞后**——Python 动态解释器在每次 Pod 启动时必须实时遍历上千个库文件进行源码字节码编译（.pyc），在拥有庞大依赖树的 vLLM 中，冷启动开销高达数十秒，严重拉低弹性伸缩效率；第二，**轻量级路由与结构化决策算力浪费**——在智能体多路由（Agent Routing）、意图分类或打分场景中，开发者往往被迫调用百亿参数的通用对话模型生成冗长的 CoT 文本，导致极高的响应延迟与费用。vLLM 官方团队（`@vllm_project`）在此前发布的 **`v0.32.0`** 基础上，正式披露了字节码预编译与 **Decision 3.0（D3）** 结构化端点的最新生产实测。\n\n### 架构亮点与底层机制\n1. **构建期字节码预编译（Build-Time Python Bytecode Compilation）**：\n   - 在 Dockerfile 构建阶段引入单一 `uv` 标志（`uv compile`），将整个 Python 环境的所有模块与依赖提前预编译为平台原生的字节码缓存；在容器启动加载时实现**零源码重新编译（Zero recompiles at launch）**；\n2. **Decision 3.0（D3）结构化决策专用端点（`/v1/systemone`）**：\n   - vLLM Semantic Router 团队为 D3 全系 6 款模型（参数量跨越 0.6B 至 27B）合入了官方 Recipe。模型通过专门设计的 `/v1/systemone` 接口对外服务，直接基于文本、JSON、图像及视频的多模态输入输出选择概率（Choices）、布尔判断（Yes/No）与打分分布，跳过无谓的自回归字符生成解码，实现微秒级路由判别。\n\n### 权威 Benchmark 与实测跑分对比\n根据 vLLM 官方公布的基准测试数据：\n- **容器冷启动提速（H100 GPU）**：\n  - 引入预编译字节码后，容器整体启动耗时缩减 **14% 至 18%**，在 NVIDIA H100 实例上启动时间直接缩短 **约 18 秒**；\n- **D3 决策模型权威基准霸榜（Jev Decision Index 0.3.2）**：\n  - D3 模型在 Jev Decision Index 文本榜拿下 **66.3 分**，在视觉榜斩获 **71.3 分**，双双登顶榜首；\n- **AMD MI325X 生产吞吐实测**：\n  - 在单张 AMD Instinct MI325X 加速卡上，面对 64 客户端高并发混合负载，**D3 27B 的每秒请求数（req/s）达到了原始 Checkpoint Server 的 1.8 倍**。\n\n### 开发者实战落地与开箱指南\n- **即刻体验**：拉取最新的 vLLM 官方镜像或更新至 nightly：\n  ```bash\n  vllm serve vllm-sr/d3\n  ```\n- **调用示例**：直接向 `/v1/systemone` 发起结构化分类请求，获取意图类别与归一化置信度得分，可无缝替代原有基于提示词的沉重 Agent 路由中间件，并直接将路由推理成本压降至极限。","### 核心背景与行业痛点\nServerless GPU scaling and multi-agent systems face two persistent infrastructure bottlenecks: first, **slow container cold starts** caused by Python dynamically compiling source files into bytecode at launch, adding tens of seconds to initialization; second, **compute waste on simple routing tasks**, where developers invoke massive general-purpose LLMs to generate verbose strings just to determine a classification or routing decision. The vLLM team (`@vllm_project`) detailed production updates around precompiled bytecode containers and native serving of the **Decision 3.0 (D3)** structured model family.\n\n### 架构亮点与底层机制\n1. **Build-Time Python Bytecode Compilation**:\n   - Uses a single `uv compile` flag during Docker image creation to precompile all modules into bytecode, ensuring zero runtime recompilation when launching worker pods;\n2. **Decision 3.0 (D3) Serving via `/v1/systemone`**:\n   - Integrates the full D3 model family (0.6B to 27B) through vLLM's specialized `/v1/systemone` endpoint. Instead of generating verbose text tokens, models evaluate choices, booleans, and scalar scores with exact probabilities over text, JSON, images, and videos in a single forward pass.\n\n### 权威 Benchmark 与实测跑分对比\nOfficial benchmarks confirm:\n- **Container Cold-Start Gains**: Cuts vLLM startup time by **14–18% (~18 seconds saved on NVIDIA H100)**;\n- **Jev Decision Index 0.3.2 Leaderboard**: D3 models rank **#1 across both boards**, scoring **66.3 on text** and **71.3 on vision**;\n- **AMD MI325X Concurrency Performance**: On 1× AMD Instinct MI325X under 64 concurrent clients, D3 27B delivers **1.8x the requests per second** compared to standard checkpoint serving engines.\n\n### 开发者实战落地与开箱指南\n- **Deployment Command**:\n  ```bash\n  vllm serve vllm-sr/d3\n  ```\n- **Agent Architecture Advice**: Replace bulky prompt-engineered routing agents with lightweight D3 models on `/v1/systemone` to perform sub-10ms intent classification and guardrailing at negligible compute cost.",[22,25],{"title":23,"url":23,"type":24},"https://github.com/vllm-project/recipes/tree/main/models/vllm-sr","github",{"title":26,"url":26,"type":27},"https://docs.vllm.ai/en/latest/models/structured_decisions_models/","docs",{"platform":29,"author_name":30,"author_handle":31,"author_avatar":32,"tweet_url":33,"published_at":34},"twitter","vLLM Project","@vllm_project","https://unavatar.io/x/vllm_project","https://x.com/vllm_project/status/2111990039806914908","2026-10-19T04:34:34.854Z","Inference",92,40,0,{"models":40,"agents":41,"vendors":42},[],[],[],[44,45,46,47,48,35],"vLLM","Cold Start","Decision 3.0","AMD MI325X","System One","https://pbs.twimg.com/news_img/2111985693786521601/cVPraiE5?format=png&name=orig",1792934235717]