AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

👀47

🔥81

Kaggle and IBM Research launch the ScarfBench Java migration leaderboard: Gemini 4 Argon with Antigravity leads at 75.2%, and about 72% of failures happen at runtime

Kaggle announced ScarfBench with IBM Research, a 204-task benchmark that tests whether coding agents can migrate enterprise Java apps between Jakarta EE, Quarkus and Spring without changing behavior. IBM's October 27 technical note shows Gemini 4 Argon with Antigravity first on the smoke score at 75.2%, Claude Opus 5.5 with Claude Code highest on all-tests-pass at 72.1%, and GPT-6.1 Sol with Codex cheapest at $1.25 per solved task.

Key Takeaways

204 migration tasks and 10 model-plus-harness configurations; top smoke scores are Gemini 4 Argon + Antigravity 75.2%, Claude Opus 5.5 + Claude Code 70.5% and Claude Opus 5 + Claude Code 69.8% (IBM figures)
Claude Opus 5.5 + Claude Code has the highest all-tests-pass rate at 72.1%, just ahead of Gemini 4 Argon at 71.1%
Compiling is mostly solved: 8 of 10 configurations compiled at least 97% of solvable tasks, yet smoke scores spanned more than 50 points
535 of 745 classified failing runs (about 72%) were runtime behavioral failures, including 210 lost-route/404 failures
Cost per solved task: $1.25 for GPT-6.1 Sol + Codex, $1.59 for Gemini 3.8 Flash + mini-swe and $10.10 for Claude Opus 5.5 + Claude Code; public submissions are not open yet

Read full story

#ScarfBench#Kaggle#IBM Research#Benchmark#coding agents#Java#Spring#Quarkus#Jakarta EE#Migration

Hugging Face Daily Papers@HuggingFace·Collected 19d ago

👀89

🔥96

MLExpert: Open 35B MoE Agentic Model Lifts MLE-Bench Lite Medal Rate from 37.9% to 72.7%, Matching Kimi-K3 and GLM-5.3

Advancing autonomous AI-for-AI (AI4AI) engineering, researchers from Shanghai AI Lab (InternScience) release MLExpert (arXiv:2610.22344), a 35B Mixture-of-Experts agentic model for autonomous machine learning engineering. Powered by the Kaggle-MLE Factory, hierarchical agentic SFT, and Windowed Interaction RL (WIRL), MLExpert boosts average medal rates on MLE-Bench Lite from 37.9% to 72.7%—matching frontier giants like Kimi-K3 and GLM-5.3 while generalizing to agent scaffold optimization.

Key Takeaways

72.7% medal rate on MLE-Bench Lite: boosts base model performance from 37.9% to 72.7%, matching frontier models Kimi-K3 and GLM-5.3 with a 35B MoE backbone
Windowed Interaction RL (WIRL): assigns shared outcome rewards to multi-turn windows between successive experiment results with asynchronous policy updates
Cross-domain AI4AI generalization: transfers optimization skills directly to NatureBench Lite, agent scaffold optimization, and algorithm engineering

Read full story

Linked Models & Agents:

⚡ MLExpert-35B-MoE⚡ Kimi K3 ⚡ GLM-5.3 🤖 MLExpert 🤖 AutoML Agents 🌐 Shanghai AI Lab 🌐 InternScience

#MLExpert#Machine Learning Engineering#WIRL#MLE-Bench#Kimi-K3#GLM-5.3#AI4AI

Google DeepMind@GoogleDeepMind·Collected 38d ago

👀87

🔥96

Google DeepMind & Kaggle Launch Game Arena: Dynamic Competitive Benchmarks for LLM Strategic Evaluation

Addressing the rapid saturation and benchmark overfitting of static LLM evaluations, Google DeepMind and Kaggle unveiled Game Arena. By orchestrating head-to-head live agent matchups with dynamic Elo rating evolution, Game Arena prevents performance saturation across three pilot environments: Chess (perfect information), Texas Hold'em Poker (imperfect information), and Werewolf (multiplayer social deduction and deception), systematically benchmarking long-horizon strategic reasoning.

Key Takeaways

Dynamic Head-to-Head Evaluation: Replaces static, easily saturated benchmarks with real-time model-versus-model matchups and dynamic Elo tracking that scales naturally as LLM reasoning capabilities evolve.
Comprehensive Game Theory Spectrum: Spans perfect information (Chess), imperfect information with strategic betting (Poker), and multi-turn multi-agent social deduction and deception (Werewolf).
Reproducible Sandboxed Infrastructure: Jointly open-sourced by DeepMind and Kaggle, providing standardized Python tournament APIs, replay verification pipelines, and sandboxed anti-cheat guardrails.

Read full story

Linked Models & Agents:

#Game Arena#Multi-Agent#DeepMind#Kaggle#LLM Benchmark#Strategic Reasoning

LlamaIndex@llama_index·Collected 63d ago

👀58

🔥86

View on 𝕏 ↗

LlamaIndex and Kaggle launch ExtractBench: 370-doc extraction leaderboard for agents

LlamaIndex (@llama_index) and Kaggle launched ExtractBench, a schema-guided document-extraction leaderboard for downstream agents. It covers the documents most likely to break workflows—long record lists, noisy scans, handwriting, and complex tables—across 370 enterprise documents, 8 business domains, and 67 document types, comparing LlamaParse, Codex, Claude Code, and more. The leaderboard is live on Kaggle.

Key Takeaways

ExtractBench is live on Kaggle for schema-guided enterprise document extraction.
370 documents across 8 domains and 67 types, including scans, handwriting, long lists, and complex tables.
Compares LlamaParse, Codex, Claude Code, and more for downstream agent workflows.

Read full story

Linked Models & Agents:

⚡ GPT-5.5 ⚡ Claude Opus 4.8 🤖 Claude Code (Anthropic)🌐 OpenAI 🌐 Anthropic

#ExtractBench#LlamaIndex#Kaggle#Claude Code#Codex#document extraction