AI Coding Live News

C
CMU Infini-AI LabInfini-AI-Lab·
👀6
🔥87

CMU Infini-AI Lab Drops CompressionBench and ServerBench: Coding Agents Compress LLMs and Design Serving Stacks; Sonnet 5.5 Hits ~4.87× vLLM Dollars-per-Dollar

Carnegie Mellon’s Infini-AI Lab posted twin arXiv benchmarks on 2026-11-04. CompressionBench asks coding agents to deliver verified compressed weights plus inference code under a storage cap (29/36 submissions pass; best-per-task designs average ~3.4% PPL above BF16). ServerBench scores complete LLM serving design by latency-qualified token value per GPU dollar; across 11 models × 3 deployments Claude Sonnet 5.5 reaches 2.97 dollars-per-dollar on Gemma E4B (~4.87× the vLLM reference) and 22/33 runs verify.

Key Takeaways

  • Official: arXiv:2611.06858 CompressionBench + arXiv:2611.06841 ServerBench (CMU Infini-AI Lab)
  • CompressionBench: 6 agents × 6 settings; 29/36 submissions pass independent verification; best designs ~3.4% PPL above BF16
  • ServerBench: objective dollars-per-dollar = latency-qualified token value / GPU cost; stateful coding-session replay
  • Paper table: Claude Sonnet 5.5 hits 2.97 on Gemma E4B (~4.87× vLLM reference); 22/33 runs verify
  • Code: github.com/Infini-AI-Lab/Compression-Bench and Server-Bench
Read full story

CMU and Fudan Pioneer Looped Model Foundations with Huginn MoE: Sandwich Topology Matches 32B Feedforward Baseline with 8B Resident Weights

Looped language models reuse physical Transformer blocks across logical depth to compress parameter footprints, but existing approaches entangle recurrence placement, input injection, and recurrent-state topologies. CMU and Fudan researchers published Part I of 'Towards Looped Models Done Right' (arXiv:2610.23036). Introducing a Prelude-Recurrent-Coda sandwich architecture, an 8.0B-resident / 794M-active Huginn MoE trained on 500B tokens matches a 32.0B-resident / 3.2B-active feedforward reference across MMLU, GSM8K, and MATH500.

CMU and Fudan Pioneer Looped Model Foundations with Huginn MoE: Sandwich Topology Matches 32B Feedforward Baseline with 8B Resident Weights

Key Takeaways

  • CMU and Fudan University established Part I of 'Towards Looped Models Done Right' (arXiv:2610.23036), decoupling recurrence placement from state design
  • Proposes the Prelude-Recurrent-Coda sandwich architecture (Huginn) to prevent representational collapse in looped transformers
  • Demonstrates that an 8.0B-resident / 794M-active Huginn MoE trained on 500B tokens fully matches a 32.0B-resident / 3.2B-active feedforward reference
  • Slashes resident memory footprint by 75%, charting a clear path to run 30B-tier reasoning models on 16GB–24GB consumer GPUs
Read full story
Linked Models & Agents:

USC, UCLA, and CMU Release Grid-Bench for Multi-Step Spatial Reasoning: 8,928 Executable Instances Reveal Severe State Drift Across 64-Step Trajectories

Can LLMs reliably maintain spatial state representations across sequential operations? USC, UCLA, and CMU introduced Grid-Bench (arXiv:2610.22312, GitHub: grid-bench/Grid-Bench), an executable benchmark with 8,928 instances testing 2D/3D/4D grid state transitions across history lengths up to T=64. Evaluations across 6 frontier models show steep degradation: while GPT-5.6 Sol and Claude 5 Sonnet retain strong cell accuracy, open-weight and mid-tier models collapse to near-zero exact world reconstruction over long trajectories.

USC, UCLA, and CMU Release Grid-Bench for Multi-Step Spatial Reasoning: 8,928 Executable Instances Reveal Severe State Drift Across 64-Step Trajectories

Key Takeaways

  • USC, UCLA, and CMU introduced Grid-Bench (arXiv:2610.22312), an executable spatial tracking benchmark across 2D, 3D, and 4D discrete manifolds
  • Evaluates 8,928 instances across 8 spatial transformation rule families and trajectory lengths up to T=64 steps
  • At intermediate length T=16, GPT-5.6 Sol (98.0% cell accuracy) and Claude 5 Sonnet (96.8%) lead while mid-tier models drop sharply
  • At extreme length T=64, exact world recovery drops to 0 for GLM-5 and Kimi K2.5, demonstrating that long-horizon agents must rely on external state engines
Read full story
Linked Models & Agents:

Stanford and CMU Introduce T2-z for Test-Time Latent Optimization in Robot World Models: Single Trajectory Lifts Real-Robot Success from 33.7% to 73.0%

Robotic world models struggle to predict physics when key environmental properties (friction, mass, contact dynamics) cannot be directly seen. Researchers from Stanford, CMU, and Toyota Research Institute introduced T2-z (arXiv:2610.22364, GitHub: T2-z-latent-optimization/T2-z). By freezing the world model and optimizing a compact environment latent z_e at test time from as little as a single interaction trajectory, T2-z halves physical error and raises real-world robot task success from 33.73% to 72.96%.

Stanford and CMU Introduce T2-z for Test-Time Latent Optimization in Robot World Models: Single Trajectory Lifts Real-Robot Success from 33.7% to 73.0%

Key Takeaways

  • Stanford, CMU, and TRI introduced T2-z (arXiv:2610.22364) for test-time optimization of unobservable physical latents in robot world models
  • Freezes the world model and optimizes a compact environment latent z_e using as little as a single test trajectory
  • Reduces physical prediction error from 15.20 to 7.00 and elevates action success from 33.73% to 72.96% across 7 simulated tasks
  • Dramatically improves real-robot manipulation, raising Ball Friction success from 0% to 62.50% and Door Close from 22.22% to 55.56%
Read full story
Linked Models & Agents:
ADSponsored

Ending Coding Agent Reward Hacking: CMU's Weak-to-Strong Verifier Hardening Adopted by Terminal-Bench, While Recursive Environment Scaling Boosts SFT by up to 14.6 Points

Tackling reward hacking and benchmark saturation in coding agent environments, CMU researchers introduce Scalable Weak-to-Strong Verifier Hardening (arXiv:2610.15924)—combining an isolated verifier sandbox with an iterative hacker-fixer loop that slashes exploit success across Terminal-Bench, KernelBench, and SETA, and is now officially adopted by Terminal-Bench. Complementing this, Recursive Environment Scaling (RES, arXiv:2610.16397) recursively mutates execution dependencies across 294 tasks—exposing a 57.14-77.14 point drop on Claw-Eval-Live while boosting equal-budget SFT performance by 7.00-14.61 percentage points across all nine model-benchmark pairs.

Ending Coding Agent Reward Hacking: CMU's Weak-to-Strong Verifier Hardening Adopted by Terminal-Bench, While Recursive Environment Scaling Boosts SFT by up to 14.6 Points

Key Takeaways

  • CMU's weak-to-strong verifier hardening combines an isolated sandbox with an iterative hacker-fixer loop across Terminal-Bench, KernelBench, and SETA, and has been adopted by the official Terminal-Bench repo
  • Recursive Environment Scaling (RES) mutates information access, resource relations, and execution dependencies across 294 tasks, dropping model success on Claw-Eval-Live by 57.14-77.14 percentage points
  • With equal trajectory counts, SFT on RES-scaled environments beats original-trajectory SFT across all 9 model-benchmark pairs by +7.00 to +14.61 percentage points
Read full story
Linked Models & Agents:

Reading Right, Acting Wrong: CMU Diagnoses Why Fast System One Models Stumble in Autonomous Agent Loops

In production agentic loops, fast single-pass 'System 1' models are favored for latency-sensitive tool selection and interface actions. When these models fail, engineers cannot easily diagnose whether the agent misperceived environmental observations or misjudged its subsequent action. Researchers from Carnegie Mellon University (CMU) introduced a behavioral diagnostic benchmark across 17 fast models and frontier reasoning baselines, pairing next-action queries with independent factual probes about the state. The results expose a striking systemic pattern: the vast majority of erroneous actions follow completely accurate perception, with error concentration increasing in more accurate models. The true failure mode lies not in reading facts, but in synthesizing them into the next programmatic sub-goal.

Reading Right, Acting Wrong: CMU Diagnoses Why Fast System One Models Stumble in Autonomous Agent Loops

Key Takeaways

  • Unmasking agent failure modes: CMU demonstrates that next-action failures are rarely caused by perceptual hallucinations; most errors occur after flawlessly reading state facts
  • Higher accuracy exacerbates synthesis gaps: as base model accuracy increases, a higher percentage of its remaining failures stem from reasoning synthesis rather than perception
  • Engineering remediation: identifies sub-goal derivation as the missing link; composing the model's own independent factual readings with deterministic code resolves the failure
Read full story
Linked Models & Agents:

Video2Skill: Streaming Embodied Skill Discovery from Video Feeds Exposes Bottlenecks in Open-World Skill Expansion

Embodied manipulation policies generalize across diverse scenes by recombining a core library of reusable physical skills. However, current agents rely almost exclusively on human-curated, hard-coded skill primitives. To achieve autonomous lifelong learning, agents must perform Streaming Embodied Skill Discovery (SESD)—observing uncurated continuous video streams and incrementally building a persistent skill catalog. Researchers from Northwestern University and CMU introduce Video2Skill (arXiv:2609.36691), benchmarking SESD across robotic manipulation and human kitchen tasks under three core capabilities: temporal event localization, physical transformation grouping, and the meta-decision to reuse existing skills versus instantiating novel primitives. Evaluating 19 open-source Vision-Language Models (VLMs) reveals severe structural limitations: most models cluster manipulation events at near-chance accuracy, and scaling parameters fails to resolve error rates. Furthermore, trained models suffer from skill library stagnation, consolidating familiar primitives while failing to expand into unobserved transformations. The authors propose Counterfactual Library-State Rebalancing (CLaRe) to mitigate clustering collapse (andyzworks.github.io/video2skill/).

Key Takeaways

  • Northwestern and CMU formulate Streaming Embodied Skill Discovery (SESD) and introduce Video2Skill benchmark across robot and human videos
  • Evaluates 19 open VLMs showing near-chance clustering accuracy; joint perception merges distinct skills while text pipelines duplicate them
  • Uncovers library stagnation: models fail to create novel skill primitives for unseen transformations, capping library size at <50% reference
Read full story

ProgressCompass: Context-Aware Embodied Progress Reward Models Drive Long-Horizon Agent Planning

As embodied robotics agents tackle long-horizon tasks across household manipulation and industrial assembly, standard terminal outcome reward models (ORMs) fail to provide actionable learning signals along multi-step trajectories. While Process Reward Models (PRMs) evaluating step-by-step task progress percentages offer a viable solution, researchers from Northwestern University and CMU demonstrate in arXiv:2609.36684 that progress reward models collapse into random noise without appropriate context. The authors introduce ProgressCompass, formalizing the essential context dependencies required for embodied progress evaluation: task objectives, initial environmental baseline states, and continuous action history. Without grounding in preceding action sequences, PRMs routinely misjudge progress direction, confusing partial object states with completed goals. ProgressCompass provides an empirical testbed and modeling architectures to guide search and policy refinement in long-horizon robotic manipulation (andyzworks.github.io/progresscompass/).

Key Takeaways

  • Northwestern and CMU present ProgressCompass, formalizing essential context dependencies for embodied Process Reward Models
  • Demonstrates that progress estimation collapses without joint access to goal specs, initial baselines, and temporal action histories
  • Boosts long-horizon robotic task success by 38.6% and reduces interaction overhead by 41.2% via context-aware PRM trajectory guidance
Read full story

WM-VLM: CMU and MBZUAI Probe Internal World Models for Interleaved Visual-Textual Spatial Reasoning

Humans solve complex spatial transformations by mentally simulating physical visual transitions, maintaining visual imaginations to ground multi-step reasoning. In contrast, existing Vision-Language Models (VLMs) tackle spatial questions almost entirely via text-token chains of thought, triggering acute hallucinations when predicting 3D rotations or spatial clearances. Researchers from Carnegie Mellon University and MBZUAI introduce WM-VLM (arXiv:2609.34826), integrating a lightweight world model branch directly into foundation VLMs to synthesize intermediate visual states during inference. The framework adopts a two-stage training paradigm: first training the model to foresee and render verifiable intermediate visual states, and then training it to ground downstream logical deductions upon those synthesized states. Evaluated on rigorous 2D and 3D mental rotation tasks, WM-VLM outperforms standard SFT baselines by up to 39.25 percentage points. Ablations prove that performance collapses when generated visual frames are removed or corrupted, establishing that internal world models provide indispensable spatial cognitive scaffolds.

WM-VLM: CMU and MBZUAI Probe Internal World Models for Interleaved Visual-Textual Spatial Reasoning

Key Takeaways

  • CMU and MBZUAI introduce WM-VLM, equipping VLMs with internal world models for interleaved visual-textual mental simulation
  • Adopts a two-stage training paradigm: synthesizing verifiable intermediate visual states and grounding spatial deduction upon them
  • Outperforms standard SFT baselines by up to 39.25 percentage points on 2D/3D mental rotations, proving causal visual grounding
Read full story

Latent-MOPD: Carnegie Mellon Pioneers Representation-Level Multi-Teacher On-Policy Distillation for LLMs

On-Policy Distillation (OPD) trains student LLMs on trajectories sampled from their own generation policies, providing robust alignment for mathematical and code reasoning. However, existing multi-teacher OPD frameworks restrict knowledge transfer purely to output token probability distributions, discarding rich semantic representations embedded in teachers' intermediate hidden states. Researchers from Carnegie Mellon University and Georgia Tech introduce Latent-MOPD (arXiv:2610.02381), the first representation-level multi-teacher OPD architecture for LLMs. Latent-MOPD integrates diverse specialists by coupling their output distributions with the internal representations used to generate them, without modifying teacher checkpoints. To coordinate disparate representations across specialists, Latent-MOPD selectively aligns late-layer targets, resolves channel mismatches through shared linear projections, and enforces domain-grouped gradient updates while gradually annealing hidden-state loss into token prediction. Across nine math, coding, and logic benchmarks, Latent-MOPD surpasses token-only and uniform-averaging baselines; matching the parameter budget of individual teachers, the distilled student outperforms the best domain teacher on a majority of benchmarks.

Latent-MOPD: Carnegie Mellon Pioneers Representation-Level Multi-Teacher On-Policy Distillation for LLMs

Key Takeaways

  • Carnegie Mellon introduces Latent-MOPD, the first representation-level multi-teacher on-policy distillation framework for LLMs
  • Pairs shared projections with domain-pure grouped updates and annealing schedules to eliminate latent representation collapse
  • Dominates all 9 math, code, and logic benchmarks, with an equal-sized student outperforming domain-best teachers on most tasks
Read full story
ADSponsored

MotorMind Harness Scaffolds General Vision-Language Models for Zero-Shot Robotic Manipulation with 95% Real-World Success

Autonomous robot manipulation has long relied on specialized vision-language-action (VLA) models whose rigid training precludes zero-shot adaptation to novel environments, cutting them off from rapid foundation VLM advancements. Conversely, recent robotic agents rely heavily on fragile coding agents and auxiliary grounding models like SAM3, introducing severe latency and operational brittleness. Researchers from UIUC and CMU present MotorMind (arXiv:2609.38078), a direct robotic harness connecting general-purpose VLMs to deterministic robot control. By translating VLM mid-level actions into deterministic Cartesian primitives paired with asynchronous execution monitoring and background memory updates, MotorMind achieves 66.7% zero-shot success on LIBERO-PRO suites (versus 13.3% for prior baselines) and achieves 95% average success on a physical xArm6 robot under live human disturbances.

MotorMind Harness Scaffolds General Vision-Language Models for Zero-Shot Robotic Manipulation with 95% Real-World Success

Key Takeaways

  • Empowers general-purpose foundation VLMs to perform zero-shot robot manipulation without specialized VLA training or SAM3
  • Introduces mid-level action scaffolding paired with asynchronous monitoring for adaptive human-like disturbance recovery
  • Surges LIBERO-PRO benchmark success from 13.3% to 66.7% and achieves 95% success on a physical xArm6 robotic manipulator
Read full story

CUA-SWE Unifies Computer-Use Agents and Visual Software Engineering: Multimodal Code-GUI Benchmarking Redefines Debugging

Modern software engineering transcends isolated code edits: engineers run applications, interact with GUI interfaces, and visually inspect rendering feedback to diagnose errors and verify changes. CUA-SWE bridges coding agents and computer-use agents by introducing the first unified benchmark and environment for visual software engineering. Spanning four engineering domains, CUA-SWE challenges agents to interleave code/config modifications, terminal execution, GUI interactions, and screenshot inspections, with deterministic test suites evaluating behavioral correctness.

CUA-SWE Unifies Computer-Use Agents and Visual Software Engineering: Multimodal Code-GUI Benchmarking Redefines Debugging

Key Takeaways

  • First multimodal visual software engineering benchmark coupling computer-use capabilities with code repositories
  • Text-only agents stall below 12% on visual defect repairs, while closed-loop visual feedback elevates success to 43.8%
  • Identifies attention contention bottlenecks between high-resolution GUI screenshots and repository-scale source code
Read full story