CompoWorld: Compositional Environment Scaling with 10,130 Tools and 448 Services Outperforms Frontier Agents

Background and the Problem

/ Background & Pain Points Autonomous agents excel at isolated, single-domain tool calls but collapse in enterprise production workflows where decisions require orchestrating state and data flows across disparate third-party services. Existing synthetic agent environments generate tasks within isolated silos, while manually designing multi-service end-to-end integration datasets remains prohibitively expensive and unscalable.

Architecture and How It Works

/ Architectural Highlights Researchers from Nanjing University and collaborating labs formulated CompoWorld:

1. Compositional Service Scaling: Employs coding agents to instantiate 448 typed microservices exposing 10,130 discrete tools with uniform state interfaces, augmented by world-model fallbacks for non-deterministic web services;
2. Random-Walk Dependency Workflows: Chains disparate tools via directed dependency graphs, synthesizing verifiable long-horizon scenarios where prerequisite information is generated by upstream services and consumed downstream;
3. Completion-Focused Rubric Reward: Directs policy gradient optimization in RL by dynamically prioritizing failure-prone transition criteria within rollout groups to enforce true end-to-end completion.

Benchmarks and Measured Results

/ Benchmark & Evaluation Fine-tuning Qwen3.6-35B-A3B with just 3K SFT trajectories and 1K RL tasks yielded striking empirical milestones:

• Cross-Benchmark Gains: Achieves a +9.17 point mean gain across eight standard tool-use benchmarks over the base checkpoint;
• Surpassing Frontier Models: On AutomationBench, CompoWorld outperforms Claude Opus 4.6 and leads all open and specialized 35B-scale agent architectures;
• Sample Efficiency: Validates that combinatorial graph-driven task generation unlocks generalization without massive pre-training data volumes.

Getting Started for Developers

/ Developer Practical Guide

• Technical Reference: Detailed algorithms are presented in arXiv preprint 2609.33665;
• Open Assets: Service schemas, dependency generators, and evaluation rubrics are accessible via Hugging Face Papers;
• Enterprise Takeaway: To train reliable integration agents, engineering teams should model cross-service API contracts and dependency trees rather than fine-tuning on disconnected single-API prompt pairs.