AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

👀47

🔥81

Kaggle and IBM Research launch the ScarfBench Java migration leaderboard: Gemini 4 Argon with Antigravity leads at 75.2%, and about 72% of failures happen at runtime

Kaggle announced ScarfBench with IBM Research, a 204-task benchmark that tests whether coding agents can migrate enterprise Java apps between Jakarta EE, Quarkus and Spring without changing behavior. IBM's October 27 technical note shows Gemini 4 Argon with Antigravity first on the smoke score at 75.2%, Claude Opus 5.5 with Claude Code highest on all-tests-pass at 72.1%, and GPT-6.1 Sol with Codex cheapest at $1.25 per solved task.

Key Takeaways

204 migration tasks and 10 model-plus-harness configurations; top smoke scores are Gemini 4 Argon + Antigravity 75.2%, Claude Opus 5.5 + Claude Code 70.5% and Claude Opus 5 + Claude Code 69.8% (IBM figures)
Claude Opus 5.5 + Claude Code has the highest all-tests-pass rate at 72.1%, just ahead of Gemini 4 Argon at 71.1%
Compiling is mostly solved: 8 of 10 configurations compiled at least 97% of solvable tasks, yet smoke scores spanned more than 50 points
535 of 745 classified failing runs (about 72%) were runtime behavioral failures, including 210 lost-route/404 failures
Cost per solved task: $1.25 for GPT-6.1 Sol + Codex, $1.59 for Gemini 3.8 Flash + mini-swe and $10.10 for Claude Opus 5.5 + Claude Code; public submissions are not open yet

Read full story

#ScarfBench#Kaggle#IBM Research#Benchmark#coding agents#Java#Spring#Quarkus#Jakarta EE#Migration

Factory@FactoryAI·Collected 6d ago

👀83

🔥82

View on 𝕏 ↗

Factory launches Droid 1.0: programmatic tool calling by default, hundreds of tool calls per model request, leads native harnesses on its own Terminal-Bench 4 runs

AI coding agent company Factory announced Droid 1.0 on X and its blog at 22:31 UTC on Oct 29, calling it the largest update yet to its agent harness. The key change makes programmatic tool calling (PTC) the default: the model writes a JavaScript program that runs up to hundreds of tool calls in a sandbox in one model request. In Factorys own Terminal-Bench 4 runs, Droid scored 57.6% with GPT-6.1 Sol ($0.79 per passed task) and 65.2% with Claude Opus 5.5 ($4.73), ahead of Codex, Claude Code, Cursor, OpenCode and Devin on the same models.

Key Takeaways

Programmatic tool calling is now the default: the model writes JavaScript that runs in a sandbox with no file, network or system access of its own, making up to hundreds of tool calls per request, with independent calls run concurrently
Factory example: moving and verifying 151 Linear issues took 312 tool calls; direct calls needed about 36 requests and 3.1M input tokens, a Script needed 2 requests and 61K tokens
Terminal-Bench 4 (Factory runs, Oct 23-28, three trials per task): with GPT-6.1 Sol, Droid 57.6% vs Codex 54.0%, OpenCode 51.0%, Devin 45.0%; with Opus 5.5, Droid 65.2% vs Cursor and OpenCode 62.1%, Devin 59.6%, Claude Code 57.6%
The first request shrank from 20,887 to 13,055 tokens (-37%) and up-front tool definitions went from 11 to 2; newly loaded MCP tools are appended to keep the prompt cache intact, and a typical production request reads 98% of input from cache
Available now as a macOS and Windows desktop app, a terminal UI and the Droid SDK (@factory/droid-sdk); the benchmarks are vendor-run with no third-party reproduction yet

Read full story

Linked Models & Agents:

⚡ GPT-6.1 Sol ⚡ Claude Opus 5.5 🤖 Droid 🤖 Codex 🤖 Claude Code 🤖 Cursor 🤖 OpenCode 🤖 Devin 🌐 Factory

#Coding agent#Agent harness#Programmatic tool calling#Terminal bench#SDK

ClaudeDevsClaudeDevs·Collected 14d ago

👀81

🔥86

View on 𝕏 ↗

Claude API ships version 2026-10-21, its first since 2023: sampling parameters removed, new major SDKs across languages

On Oct 21 Anthropic released Claude API version 2026-10-21, its first official version since 2023. It removes the temperature, top_p and top_k sampling parameters and drops old Admin API field names, while new SDK majors (Python 2.0, TypeScript 1.0, Go 2.0, Java 3.0 and more) default to it. Version 2023-06-01 is not deprecated.

Key Takeaways

Version: 2026-10-21 is the Claude API's first official version since 2023; 2023-06-01 is not deprecated and more frequent versions are planned
Breaking: sending temperature, top_p or top_k returns a 400 on every model; Claude 4.7+ models behave as before
SDKs: Python 2.0.0, TypeScript 1.0.0, Go 2.0.0 (/v2 path), Java 3.0.0, C# 13.0.0, Ruby 2.0.0, PHP 1.0.0, ant CLI 2.0.0
Auth: SDKs send the API key as Authorization: Bearer and hold 1 credential per client; SDK license moves to Apache 2.0
Platforms: Claude API, Claude Platform on AWS, Microsoft Foundry, Google Cloud; Bedrock only for Claude 4.7+, see the migration guide

Read full story

Linked Models & Agents:

⚡ Claude Opus 5.5 ⚡ Claude Sonnet 4.6 🤖 Claude API 🤖 ant CLI 🌐 Anthropic

#Claude API#API versioning#SDK#breaking changes

OpenAI@OpenAI·Collected 16d ago

👀113

🔥88

View on 𝕏 ↗

OpenAI open-sources MultiLangBench and reports low-resource language gains: GPT-6 Astra averages 87.4% building SQLite across 20 languages, GPT-6.1 Sol reaches up to 97.9% on COBOL migration

On October 19 OpenAI said its latest models are much better at low-resource languages such as COBOL, Fortran, Visual Basic and OCaml, and open-sourced MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch in each of 20 languages. GPT-6 Astra averages an 87.4% test pass rate (GPT-5.6 Sol: 26.8%), above 83% in 19 of 20 languages, with COBOL hardest at 72.1%. On a separate COBOL migration eval requiring byte-for-byte output parity after porting to C++, Java, Kotlin, Python or Rust, GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol: 5.4%) and 97.9% at higher effort. All figures are OpenAI's own.

Key Takeaways

MultiLangBench: GPT-6 Astra (max) averages 87.4% across 20 languages vs 26.8% for GPT-5.6 Sol, above 83% in 19 of them (OpenAI-reported).
Per language: COBOL 10.9% to 72.1%, OCaml 25.6% to 85.8%, Fortran 21.4% to 83.5%, Visual Basic 27.5% to 89.4%.
Budget tier: GPT-6 Luna averages 40.3% at about $0.77 per sample vs $3.48 for GPT-5.6 Sol, roughly 78% cheaper.
COBOL migration: GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol 5.4%) at about 61% of the API cost; 97.9% at higher effort, near saturation.
The eval setup, test suite and per-language results are on GitHub openai/multilangbench; the COBOL migration eval is not public.

Read full story

Linked Models & Agents:

⚡ GPT-6 Astra ⚡ GPT-6.1 Sol ⚡ GPT-6 Luna ⚡ GPT-5.6 Sol 🌐 OpenAI

#COBOL#Fortran#MultiLangBench#Benchmark

Google DeepMind@GoogleDeepMind·Collected 19d ago

👀463

🔥99

View on 𝕏 ↗

Google DeepMind: Four Gemini 4 Argon Agents Autonomously Beat Minecraft from Scratch in 27 Hours by Writing 2,783 Programs

Google DeepMind announced that a team of four Gemini 4 Argon agents completed Minecraft from scratch in 27 hours and 14 minutes without human assistance. Operating in a sandboxed survival world using only standard Mineflayer APIs, code execution, and text chat, the agents autonomously divided roles, recovered teammates' items across 605 deaths, and wrote and debugged 2,783 JavaScript programs to defeat the Ender Dragon.

Key Takeaways

First-ever end-to-end autonomous Minecraft completion: four Gemini 4 Argon agents defeated the Ender Dragon from a random survival seed in 27 hours and 14 minutes
2,783 self-written programs with zero human help: coordinated via in-game chat and shared notes, debugging code and recovering items across 605 deaths
WANDR record & Perplexity launch: Gemini 4 Argon simultaneously set a new record on the WANDR benchmark and rolled out to Perplexity Pro and Max users

Read full story

Linked Models & Agents:

⚡ Gemini 4 Argon 🤖 Minecraft Multi-Agent Team 🤖 Perplexity Pro 🌐 Google DeepMind 🌐 Perplexity

#Gemini 4 Argon#Google DeepMind#Minecraft#Multi-Agent Coordination#Long-Horizon Agents#Mineflayer

NUS School of Computing@NUSingapore·Collected 22d ago

👀115

🔥97

Last Secure Code Benchmark: Vibe Coding Under Fire as Top Agents Produce Secure Code on Only 25.1% of Tasks

While AI coding agents and 'vibe coding' workflows synthesize an unprecedented volume of production software, code security remains highly fragile. Researchers from NUS and Monash University introduced the Last Secure Code Benchmark, evaluating 450 tasks derived from 150 real CVE vulnerabilities across 7 languages (Java, C, C++, JS, Python, PHP, Go) and 24 CWE categories (including 21 of the 2025 CWE Top 25 and 8 of the Top 10 KEVs). While frontier agents achieve 73.6% to 91.3% functional completion rates, even the strongest model (Claude Fable 5) produces code that is both functional and secure on only 25.1% of tasks, exposing acute supply-chain risks in autonomous vibe coding.

Key Takeaways

First comprehensive vibe-coding security benchmark: 450 tasks based on 150 CVEs spanning 7 languages, covering 21 of the 2025 CWE Top 25 and 8 Top 10 KEVs
The functionality illusion: while agents pass functional specs at 73.6%–91.3%, dual functional-and-secure pass rates collapse to 16.4%–25.1%
Vulnerability defense crisis: across full-repository and file generation scenarios, top models resolve only 12% of vulnerability environments securely

Read full story

Linked Models & Agents:

⚡ Claude Fable 5⚡ GPT-5⚡ Claude 3.5 Sonnet ⚡ DeepSeek-Coder-V2 🤖 Coding Agents 🤖 Autonomous Software Engineering 🌐 National University of Singapore 🌐 Monash University

#Last Secure Code#Vibe Coding#Coding Agents#Cybersecurity#CWE#Software Vulnerability

Hugging Face Daily Papers@HuggingFace·Collected 28d ago

👀93

🔥99

CheckerBench: ECNU and Peking University Introduce the First Executable Benchmark for Coding Agents Synthesizing Static-Analysis Checkers

Prevailing coding agent benchmarks evaluate isolated patch synthesis (e.g. SWE-bench) or vulnerability detection, neglecting whether agents can construct reusable, production-grade static-analysis checkers from scratch. Developing static analyzers requires interpreting defect specifications, traversing multi-file AST semantics, writing engine-specific logic, and refining implementations through iterative compilation and diagnostic feedback. Researchers from East China Normal University, SJTU, and Peking University unveil CheckerBench (arXiv:2610.07557), an executable suite comprising 300 tasks derived from 297 CVEs across 167 repositories, 85 CWEs, and five programming languages. Powered by the CheckerLab harness, evaluation across 21 model-harness configurations reveals a mean Pass@1 rate of only 32.30% (peak 45.33%), demonstrating that end-to-end program analysis tool synthesis represents a profound new frontier for software engineering agents.

Key Takeaways

ECNU, SJTU, and Peking University launch CheckerBench, the first executable benchmark for coding agents synthesizing static analysis checkers
Features 300 tasks spanning 297 CVEs and 85 CWEs across C/C++, Go, Java, Python, and Rust with the CheckerLab framework
21 model-harness setups achieve a mean Pass@1 of only 32.30% (peak 45.33%), exposing gaps in compiler-in-the-loop tool synthesis

Read full story

Linked Models & Agents:

⚡ CheckerBench⚡ CheckerLab⚡ SWE-bench⚡ Coding-Agents🤖 Static-Analysis-Agent 🤖 Software-Security-Agent 🤖 Checker-Synthesizer 🌐 East China Normal University 🌐 Peking University 🌐 Hugging Face

#CheckerBench#Coding Agents#Static Analysis#Vulnerability Detection#Compiler In The Loop#AI Benchmarks

Hugging Face Daily Papers@HuggingFace·Collected 29d ago

👀130

🔥99

IGMWorld: Benchmarking Coding Agents on Executable World Editing across Minecraft and Terraria via Industry-Grade Modding

Interactive world models are progressing beyond passive simulation toward active world generation and agency. However, deliberately editing an existing, running executable world while preserving unedited systemic invariants remains an underexplored frontier. An international research consortium introduces IGMWorld and IGMBench (arXiv:2610.02331), framing world editing through industry-grade game modding across Minecraft (Java) and Terraria (C#/.NET). Spanning 110 tasks evaluated against 1,100+ deterministic executable criteria across property, entity, dynamics, and system-level intervention depths, IGMBench rigorously assesses coding agent precision. Frontier agents achieve 78.2% task-level success and 94.8% criterion-level pass rates. Crucially, failure rates scale monotonically with intervention depth—most failed edits build and load without syntax errors but diverge in runtime execution logic. Project assets are accessible at vinesmsuic.github.io/IGMWorld.

Key Takeaways

Introduces IGMWorld and IGMBench, the first benchmark for executable world editing using Minecraft and Terraria game modding
Defines four hierarchical intervention depths (property, entity, dynamics, system) with 110 tasks and 1,100+ deterministic executable criteria
Frontier coding agents achieve 78.2% task-level and 94.8% criterion-level accuracy, identifying sharp drops in deep system interventions

Read full story

Linked Models & Agents:

⚡ IGMWorld⚡ Minecraft Modding⚡ Terraria🤖 World-Editing-Agent 🤖 Game-Modding-Agent 🤖 Coding-Agent 🌐 University of Hong Kong 🌐 National Taiwan University 🌐 Hugging Face

#IGMWorld#World Editing#Interactive World Models#Game Modding#Coding Agents#Benchmark

LangChain4j@langchain4j·Collected 39d ago

👀92

🔥91

LangChain4j 1.20.1: solidifies non-blocking AI Services (CompletableFuture/Flow.Publisher), opt-in Jackson 3, MCP Supplier with params

LangChain4j 1.20.1 landed 2026-09-25 atop the 1.20.0 line. AI Services now choose sync, TokenStream, CompletableFuture/CompletionStage, or Flow.Publisher by return type; non-blocking reaches tools, memory, guardrails, and the retrieval graph (@Experimental). Opt-in langchain4j-jackson3 routes JSON through Jackson 3 with typed JsonRead/WriteException. 1.20.1 lets @McpClientSupplier methods also take supplier parameters for per-request MCP clients. Aimed at Quarkus/Vert.x/WebFlux teams needing concurrent agents.

Key Takeaways

Shipped: 1.20.1 on Central; headline features from 1.20.0 non-blocking line
Pick CompletableFuture/Flow.Publisher return types; async tool defaults differ from sync
Docs: Non-blocking and Reactive tutorial
Opt-in langchain4j-jackson3 for Jackson 3 + typed JSON exceptions
1.20.1: @McpClientSupplier methods may take supplier parameters

Read full story

Linked Models & Agents:

🤖 LangChain4j-AI-Service 🤖 MCP-Client 🌐 LangChain4j 🌐 Quarkus 🌐 Spring-WebFlux

#LangChain4j#Java#Non blocking#Reactive#MCP#Jackson3#AI-Service#RAG

GitHub@github·Collected 60d ago

👀46

🔥88

View on 𝕏 ↗

GitHub: Copilot Canvases give agentic workflows a durable shared surface for runs, diffs, and review

GitHub’s official post on Copilot Canvases: a durable shared surface instead of endless agent chat scrolls, tracking what ran, what changed, and what still needs review. The write-up stresses visible, steerable, cost-efficient agentic workflows and points to Java Modernization Studio, Site Studio in awesome-copilot, plus /create-canvas.

Key Takeaways

Product: Canvases replace endless agent chats with a durable shared surface.
Value: track runs, changes, and review queue with steerable, cost-aware workflows.
Ecosystem: awesome-copilot ships Java Modernization Studio and Site Studio; build more with /create-canvas.

Read full story

Linked Models & Agents:

🤖 GitHub Copilot (Agent Mode / Codex)🌐 github

#GitHub#Copilot#Canvases#agentic workflows#Awesome copilot

L@lanreadelowo·Collected 61d ago

👀59

🔥91

View on 𝕏 ↗

Spotify: Portal shunt plugin offloads Claude Code I/O to cheap models — ~90% token savings

Spotify Engineering describes Portal AiKA Modes plus a Claude Code plugin called shunt: PreToolUse hooks intercept large-file Read/bash I/O and route work through Portal CLI bulk-reader / code-writer modes (examples use Gemini 2.5 Flash), keeping frontier Claude for real reasoning. Across four Java monorepo scenarios, mean bulk-read savings were about 90% Claude tokens. Install from the spotify/portal-ai-plugins marketplace.

Key Takeaways

shunt hooks block large Read/bash I/O and force Portal delegation.
bulk-reader / code-writer modes run cheap workers (e.g. Gemini 2.5 Flash) with no infra to manage.
Mean ~90% Claude token savings on bulk-read in a Java monorepo; marketplace: spotify/portal-ai-plugins.

Read full story

Linked Models & Agents:

⚡ Gemini 2.5 Flash ⚡ Claude Fable 5.1 🤖 Claude Code (Anthropic)🌐 Anthropic

#Spotify#Portal#Shunt#Claude Code#token savings#AiKA Modes#Gemini 2.5 Flash#MCP