CMU Infini-AI Lab Drops CompressionBench and ServerBench: Coding Agents Compress LLMs and Design Serving Stacks; Sonnet 5.5 Hits ~4.87× vLLM Dollars-per-Dollar
Carnegie Mellon’s Infini-AI Lab posted twin arXiv benchmarks on 2026-11-04. CompressionBench asks coding agents to deliver verified compressed weights plus inference code under a storage cap (29/36 submissions pass; best-per-task designs average ~3.4% PPL above BF16). ServerBench scores complete LLM serving design by latency-qualified token value per GPU dollar; across 11 models × 3 deployments Claude Sonnet 5.5 reaches 2.97 dollars-per-dollar on Gemma E4B (~4.87× the vLLM reference) and 22/33 runs verify.
Key Takeaways
- Official: arXiv:2611.06858 CompressionBench + arXiv:2611.06841 ServerBench (CMU Infini-AI Lab)
- CompressionBench: 6 agents × 6 settings; 29/36 submissions pass independent verification; best designs ~3.4% PPL above BF16
- ServerBench: objective dollars-per-dollar = latency-qualified token value / GPU cost; stateful coding-session replay
- Paper table: Claude Sonnet 5.5 hits 2.97 on Gemma E4B (~4.87× vLLM reference); 22/33 runs verify
- Code: github.com/Infini-AI-Lab/Compression-Bench and Server-Bench








