Benchmark6d ago
Artificial Analysis releases AA-Omniscience v1.1: 3,000 refreshed questions and a calibrated 300-question public set, with Gemini 4 Carbon leading at a Net Score of 54
On October 31, independent evaluator Artificial Analysis released AA-Omniscience v1.1, its knowledge and hallucination benchmark and part of Intelligence Index v5. The question set shrinks from 6,000 to 3,000, dropping previously public and saturated questions, fixing errors and ambiguities, and adding new ones, and a new 300-question public set on Hugging Face is calibrated to the private set's difficulty, with 0.99 Pearson and Spearman correlation on Net Score. Gemini 4 Carbon (high) leads with a Net Score of 54, ahead of Claude Opus 5.5 at 48 and Claude Fable 5.1 at 46; Fable 5.1 has the highest accuracy (68%) and Gemini 4 Argon the lowest hallucination rate (4%). Models score about 1.9 points lower than on v1 on average, and MiniMax M3.1 Flash drops 22 points. All figures are as reported by Artificial Analysis.
Benchmark60d ago
Artificial Analysis Intelligence Index v4.3: Terminal-Bench 4.0 + AutomationBench-AA
Artificial Analysis (@ArtificialAnlys) shipped Intelligence Index v4.3: Terminal-Bench upgraded 2.1→4.0 (now mini-SWE-agent harness) and τ³-Banking replaced by AutomationBench-AA (657 private Zapier business workflows). Claude Fable 5.1 and GPT-6 Astra tie at 53; OpenAI holds most of the intelligence-vs-cost frontier across Astra reasoning efforts.
Benchmark62d ago
Artificial Analysis Intelligence Index v4.2: Fable 5.1 leads, Astra second
Artificial Analysis shipped Intelligence Index v4.2: adds private agentic knowledge-work eval AA-Briefcase and long-document GDP.pdf, doubles held-out weighting, upgrades grading, and drops saturated GPQA Diamond. Claude Fable 5.1 leads; GPT-6 Astra is second with ~4pt gain over GPT-5.6 Sol and strong output-token Pareto efficiency. A major interim benchmark refresh for model selection and cost/quality tradeoffs.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.