Perplexity's Q2D-Web Benchmark Tests AI Search on 190M Real Web Documents
Perplexity released Q2D-Web, a large-scale benchmark with 190M documents and 70K agent-reformulated queries for evaluating retrieval in agentic RAG systems.
// Intelligence
Benchmarks
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
Microsoft's ThinkingBox Catches AI Agents Lying About Database Changes
Microsoft open-sourced a benchmark harness that checks whether agents actually change state, not just claim they did, across 507 real business tasks.
UT Austin Finds Fractal Chaos Explains Why AI Reasoning Costs Spike 10x
A new paper shows recurrent-depth reasoning models behave like chaotic dynamical systems, where hard problems create fractal basins that trap thinking near wrong answers.
OpenBMB's MiniCPM5-2B Tops Sub-4B Charts While Burning 3x Fewer Tokens
OpenBMB's 2.6B dense reasoning model tops the sub-4B open weights leaderboard, punching well above its weight on agentic benchmarks while staying token-efficient.
NVIDIA's Nemotron Beats the Best Human at the Coding Olympics
NVIDIA's fine-tuned Nemotron system scored 535.4 out of 600 at IOI 2026, topping the best human contestant under identical contest conditions.
Artificial Analysis Rebuilds Its AI Leaderboard to Stop Models From Gaming It
Artificial Analysis pushes Intelligence Index v4.2 with private test sets, a 4,592-page PDF reasoning benchmark, and a new agentic knowledge-work suite.
LlamaIndex's ExtractBench Catches GPT-5 and Gemini-3 Hitting 0% on Wide Schemas
LlamaIndex and Kaggle launch a schema-guided document extraction benchmark that grades models on missing fields, source grounding, and repeated records across 370 enterprise files.
Anthropic's Claude Formally Proved Fermat's Last Theorem in 11 Days
Claude autonomously wrote a 13 million line Lean proof of Fermat's Last Theorem in 11 days, verifying 29,500 supporting theorems along the way.
GitHub's HydraFusion Beats Claude Opus 5 at 67% Lower Cost
GitHub's new research preview routes each coding task through single, cascade, or critique workflows across multiple models, beating Opus 5 at a fraction of the cost.
Warp's WarpBench Slashes Coding Agent Costs by 63% Using Your Own Codebase
Warp launches Factory Benchmarks, letting teams replay real agent runs to pick the best model per task and cut cost-per-PR from $80 to $30.
Microsoft's MAI-Transcribe-2 Beats ElevenLabs on Accuracy at 54% Lower Price
Microsoft AI's new speech recognition model hits 2.0% word error rate at 411x real time speed for $1.67 per 1,000 minutes.
STAIR Beats Every RAG Baseline by Reading Documents Like a Table of Contents
A new generative retriever leverages the table of contents to hit 82.6% Recall@1 on a fresh 18-book benchmark, with hallucinations under 0.05%.
Qwen's E-Commerce Bench Exposes How Badly AI Agents Fail at Running a Business
A new 365-day simulation drops LLM agents into a deterministic marketplace with ¥100,000, real supplier data, and fraudsters, and no model wins everything.
Artificial Analysis Rebuilds Its Image Editing Arena Across 17 Specialized Categories
Artificial Analysis rebuilt its Image Editing Arena with a two-axis taxonomy of editing actions and use cases, revealing which model wins each specific task.
Multiverse Computing's Quasar 438B Tops Europe's AI Leaderboard While Beating Bigger Models
Multiverse Computing's Quasar 438B tops European AI rankings with a 43 Intelligence Index score and 15.3-second reasoning responses.
Anthropic's Claude Fable 5.1 Tops Every Frontier Model but Costs 20% More
Independent evaluations put Anthropic's new flagship at the top of the intelligence charts, but the win comes with a token-count tax that eats the cache savings.
Lovable's Fable 5.1 Fixes Live Apps 17% Better at 31% Lower Cost
Lovable's new default coding model gains 17% on the hardest fix-and-iterate tasks while cutting cost by 31%, with sharper self-verification and visual taste.
Meta's Muse Voice Transcribe Beats Every Rival With 3.1% Error Rate
Meta Superintelligence Labs' new real-time speech model tops streaming ASR and diarization benchmarks, handles 20+ speakers, and ships at $0.18 per hour.
Google's TimesFM-3 Tops Three Forecasting Leaderboards With 330M Parameters
Google's new 330M-parameter time series foundation model handles multivariate forecasting zero-shot in a single forward pass, topping Gift-Eval, FEV-Bench and Time.
Z.ai's GLM-5.3 Beats GPT-5.6 on Coding Without Touching Its Base Weights
Z.ai shipped a frontier coding model without touching the base weights, matching GPT-5.6 Sol and Claude Fable 5 on agentic tasks through scaled post-training alone.
Apodex 1.1 Beats DeepSeek on Real-World Agentic Work With a 35B Open Model
Apodex 1.1 lands with an Elo of 1348 on GDPval-AA v2, beating DeepSeek V4 Pro and Kimi K2.6 on real-world agentic work.
Harvard and MIT's Science Sandbox Catches Claude Beating Humans While Missing the Point
Broad Institute researchers unveil a framework that separates score-chasing from real scientific reasoning, exposing where Claude, GPT, and Gemini break down.
Perplexity's Search API Tops Agent Leaderboard, Beating Rivals at $5 Flat
Perplexity's new Search API takes the top three spots on the Artificial Analysis Search Index, extending the quality-cost Pareto frontier for agentic search.
Epoch's EBR-Bench Reveals Humans Learn While Frontier AI Stays Stuck
Epoch AI's EBR-bench human baseline shows people quickly outclass frontier models at Earthborne Rangers, exposing a real learning gap.
No stories match these filters.