Mosaic: Multi-Session Agentic Reinforcement Learning for Context Compaction and Subagent Forking
Standard agentic reinforcement learning assumes single-session trajectories with continuously accumulated context, clashing with production agent techniques like context compaction and subagent forking. Researchers introduce Mosaic (arXiv:2610.22174), a critic-free hierarchical optimization framework for Multi-Session Agentic RL. By incorporating session-usage signals at the trajectory level and shaping per-session advantages by marginal contribution, Mosaic outperforms trajectory-level broadcasting on BrowseComp across both compaction and subagent scenarios while using fewer sessions.
Key Takeaways
- Bridges single-session RL and multi-session inference: formalizes Multi-Session Agentic RL for context compaction and subagent forking workflows
- Critic-free hierarchical advantage shaping: combines trajectory-level session-usage signals with session-level marginal contribution estimation
- Higher accuracy with fewer sessions: outperforms trajectory-level broadcasting on BrowseComp across both compaction and subagent scenarios