MMPostTrainBench & MMResearch: Benchmarking Autonomous Agents in Multimodal Post-Training and Mitigating Capability Regressions
Autonomous research agents hold the promise of sustaining AI self-improvement by iteratively proposing hypotheses, tuning models, and refining post-training pipelines. While agents demonstrate competence on text-only code tasks, their capacity to execute multimodal post-training across images, audio, and video remains uncharted. Researchers introduce MMPostTrainBench (arXiv:2610.05398), the first benchmark designed to evaluate autonomous research agents conducting multimodal post-training across eight tasks spanning image, audio, video, joint audiovisual comprehension, and image-grounded software repair. Agents operate from a common base model under fixed budgets with development feedback before independent evaluation. The audit reveals sobering realities: across all tasks, 52.1% of agent-submitted models fall below the baseline, exhibiting severe non-target capability regressions, and agents fail to reliably select the best candidate (trailing top models by up to 5.38 percentage points). To resolve multimodal research drift, the authors introduce MMResearch, a framework integrating media-grounded evidence and hierarchical memory. Integrated into coding agent runtimes, MMResearch improves submitted model accuracy by up to 7.75 percentage points for Claude Code (Claude Opus 4.8) and 2.33 points for Codex (GPT-5.6).
Key Takeaways
- MMPostTrainBench benchmarks autonomous research agents conducting multimodal post-training across eight core tasks
- Reveals that 52.1% of unguided agent post-training runs degrade baseline performance, suffering a 5.38 pp selection gap
- Introduces MMResearch scaffolding, elevating Claude Code model accuracy by +7.75 pp and slashing regressions by 64%