[{"data":1,"prerenderedAt":48},["Reactive",2],{"news-story-news-1792384474854-8dv0g":3},{"id":4,"title_zh":5,"title_en":6,"summary_zh":7,"summary_en":8,"takeaways_zh":9,"takeaways_en":14,"content_zh":19,"content_en":20,"external_links":21,"source":25,"category":32,"impact_score":33,"views_count":34,"likes_count":35,"related_entities":36,"tags":40,"image_url":47},"news-1792384474854-8dv0g","香港大学提出混合线性注意力投机解码 ReaDy：复用循环状态作为动态上下文，B200 解码吞吐达 302 TPS","HKU Introduces ReaDy for Hybrid Linear Attention: Recurrent States as Dynamic Context Accelerates B200 Decoding to 302 TPS","传统投机解码（Speculative Decoding）依赖目标模型的静态隐层特征，无法适应草稿模型随位置变化的动态上下文需求。香港大学团队发表 ReaDy（arXiv:2610.20503，HF 6 赞）。该框架首次为混合线性注意力模型（Hybrid Linear Attention，结合 Transformer 与 Mamba/RWKV 循环层）设计了以紧凑循环隐状态为动态上下文的投机草稿机制。在 NVIDIA B200 单卡上对 Qwen3.8-27B 展开评测，投机接受长度达 4.53，单卡解码吞吐飙升至 302.78 TPS，较标准自回归提速 2.4 倍。","Traditional speculative decoding relies on static target hidden features that fail to supply varying contextual needs across draft positions. HKU researchers introduced ReaDy (arXiv:2610.20503). Designed specifically for Hybrid Linear Attention (HLA) architectures combining Transformers with recurrent linear attention states, ReaDy leverages compact recurrent states as dynamic context for drafting. On a single NVIDIA B200 GPU serving Qwen3.8-27B, ReaDy achieves an acceptance length of 4.53 and 302.78 TPS, accelerating inference by 2.4x over standard autoregressive decoding.",[10,11,12,13],"香港大学提出首个专为混合线性注意力（HLA）架构打造的投机解码方法 ReaDy（arXiv:2610.20503）","创新性地复用 HLA 骨干中的紧凑循环隐状态作为动态上下文，解决传统草稿模型特征静态僵化的问题","在单张 NVIDIA B200 上对 Qwen3.8-27B 展开实测，解码吞吐达到 302.78 TPS，较标准自回归提速 2.4 倍，在 DSpark 基础上再提升 7.97%","仅需 Rank 8 超轻量读取器即可达成最佳性能，投机接受长度达到 4.53 个 Token",[15,16,17,18],"HKU introduced ReaDy (arXiv:2610.20503), the first speculative decoding framework customized for Hybrid Linear Attention (HLA) LLMs","Reuses compact recurrent states as dynamic context for drafting, overcoming the static context bottleneck of feature-based drafters","Achieves 302.78 TPS for Qwen3.8-27B on a single NVIDIA B200 GPU (2.40x over AR and +7.97% over SOTA DSpark)","Reaches an acceptance length of 4.53 tokens using an ultra-lightweight Rank-8 reader","### 核心背景与行业痛点\n投机解码（Speculative Decoding）是加速大模型自回归解码（AR）的主流手段。然而，现有的特征级草稿模型（Feature-Based Drafters，如 EAGLE、DSpark）普遍直接抓取目标大模型最后一层的顶层静态隐藏状态（Hidden State）作为草稿预测依据。这种做法存在内在缺陷：单一固定隐藏状态所包含的上下文侧重点，无法满足后续多个推测位置（Draft Positions）动态变化的语义需求。而近年来崛起的混合线性注意力架构（Hybrid Linear Attention, HLA，如融合了 Transformer 注意力与 Mamba/RWKV 线性状态的混合模型）在维持极低显存的同时维护了紧凑的**循环隐状态（Recurrent States）**，该状态天然积累了丰富的时序关联。香港大学团队在 **`arXiv:2610.20503`** 中首次利用该特性提出了全新的投机解码加速框架 **`ReaDy`**。\n\n### 架构亮点与底层机制\n1. **循环隐状态作为动态上下文（Recurrent States as Dynamic Context）**：\n   - `ReaDy` 创新性地将 HLA 骨干网络中固有的紧凑循环状态作为动态上下文信息源。通过可学习的低秩读取器（Low-Rank Reader，仅需 Rank 8~16）轻量提取历史语义记忆，与目标特征拼接共同输入草稿解码器；\n2. **解耦正交投影机制（Orthogonal Projection Decoupling）**：\n   - 在读取循环状态时，利用正交投影算子分离与当前查询重叠的已知特征与未探索的正交补空间（Orthogonal Complement），确保注入草稿网络的信息具备最大信息增益；\n3. **极简低参数草稿网络与跨层注入**：\n   - 经过系统消融，仅需在草稿网络最后一层后注意力位置注入第 30 层与第 62 层的循环状态，即可以极小的显存与计算开销达成最佳预测拟合。\n\n### 权威 Benchmark 与实测跑分对比\n在单张顶级 **NVIDIA B200 GPU** 上对 `Qwen3.8-27B` 进行的大规模投机解码基准实测（Table 1, 2, 3, 4）：\n- **端到端解码吞吐（Tokens Per Second, TPS）**：\n  - 纯自回归（AR）解码基线为 126.15 TPS；\n  - 先进投机解码基线 `DSpark` 达到 280.42 TPS（2.22x 加速，平均接受长度 $\\tau = 4.17$）；\n  - **`ReaDy` 吞吐飙升至 302.78 TPS（2.40x 加速，接受长度 $\\tau = 4.53$）**，在 DSpark 极高基准上进一步取得 **+7.97% 的纯净吞吐增益（+8.50% 接受长度提升）**；\n- **超低秩读取器效率（Table 3）**：\n  - 实测证实，读取器 Rank 仅设为 **8** 时即可达到 **302.78 TPS**，与全秩 128（302.31 TPS）完全持平，额外参数增加几乎可以忽略不计；\n- **跨任务稳定性**：\n  - 在 GSM8K、MATH500、MetaMathQA 与 Evol-CodeAlpaca 等代码与数学长推演任务上均取得一致的加速收益。\n\n### 开发者实战落地与开箱指南\n- **技术细节与消融实验**：架构推导与训练两阶段优化器参数详见 `arXiv:2610.20503`。\n- **工程部署指引**：正在研发或部署 Mamba/RWKV+Transformer 混合线性注意力大模型的团队，可借鉴 ReaDy 的循环状态抽取器，无需额外训练庞大的独立 Draft 模型，仅通过轻量级 Rank-8 读取器挂载即可在 B200/H100 实例上解锁 300+ TPS 的极速代码与推理输出。","### 核心背景与行业痛点\nSpeculative decoding accelerates autoregressive generation by drafting multiple speculative tokens in parallel. However, feature-based drafters (such as EAGLE or DSpark) condition predictions on target hidden features whose contextual emphasis is static and cannot align with the shifting semantic needs of subsequent draft positions. Recently emerging Hybrid Linear Attention (HLA) architectures combine Transformer self-attention with recurrent linear layers (like Mamba or RWKV), continually updating compact recurrent states that accumulate long-term historical context. Researchers from the University of Hong Kong (HKU) introduced **`ReaDy` (`arXiv:2610.20503`)**, the first speculative decoding framework specifically designed to harness HLA recurrent states.\n\n### 架构亮点与底层机制\n1. **Recurrent States as Dynamic Drafting Context**:\n   - `ReaDy` taps into the compact recurrent memory states maintained by HLA layers, utilizing a lightweight low-rank reader (Rank 8–16) to inject historical context into the drafting process;\n2. **Orthogonal Projection Decoupling**:\n   - Decomposes the recurrent state via orthogonal projection, separating features that span the current query subspace from complementary information in the null space to avoid redundant drafting signals;\n3. **Targeted Cross-Layer Injection**:\n   - Systematic layer-ablation reveals that tapping target recurrent states from layers 30 and 62 and injecting them post-attention into the final drafting layer delivers optimal speculative accuracy with virtually zero compute overhead.\n\n### 权威 Benchmark 与实测跑分对比\nEvaluated on a single **NVIDIA B200 GPU** running `Qwen3.8-27B` (Tables 1–4):\n- **End-to-End Decoding Speed (Tokens Per Second, TPS)**:\n  - Standard autoregressive baseline: 126.15 TPS;\n  - SOTA speculative decoding baseline `DSpark`: 280.42 TPS (2.22x speedup, acceptance length $\\tau = 4.17$);\n  - **`ReaDy` reaches 302.78 TPS (2.40x speedup over AR, $\\tau = 4.53$)**, gaining an additional **+7.97% TPS (+8.50% acceptance length)** over DSpark;\n- **Reader Rank Efficiency (Table 3)**:\n  - A reader rank of just **8** achieves **302.78 TPS**, fully matching full-rank 128 (302.31 TPS) while adding negligible parameter overhead;\n- **Generalization**: Demonstrates consistent speedups across complex mathematical reasoning (`GSM8K`, `MATH500`) and code synthesis (`Evol-CodeAlpaca`).\n\n### 开发者实战落地与开箱指南\n- **Research Reference**: Full architectural proofs and two-stage training configurations are detailed in `arXiv:2610.20503`.\n- **Deployment Advice**: Teams serving hybrid linear attention models can plug in ReaDy's low-rank recurrent state reader to break past 300 TPS on NVIDIA B200 accelerators without serving external draft checkpoints.",[22],{"title":23,"url":23,"type":24},"https://arxiv.org/abs/2610.20503","arxiv",{"platform":26,"author_name":27,"author_handle":28,"author_avatar":29,"tweet_url":30,"published_at":31},"twitter","University of Hong Kong (HKU)","@huggingface","https://unavatar.io/x/huggingface","","2026-10-19T04:34:34.854Z","Inference",92,35,0,{"models":37,"agents":38,"vendors":39},[],[],[],[41,42,43,44,45,46],"ReaDy","Speculative Decoding","HLA","NVIDIA B200","HKU","Throughput","https://arxiv.org/html/2610.20503v1/ready-architecture.png",1792890704078]