Keenable Beats Every Search Tool on Vals' Finance and Legal AI Index
Keenable's search API paired with GPT-5.6 Sol takes the top spot on the Vals Web Search Index at 49.1%, leading finance research.
// Intelligence
Benchmarks
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
Microsoft Solves a 31-Year-Old Math Problem Using AI Agents
Microsoft researchers used human-directed AI agents to prove a 31-year-old geometry conjecture at the heart of randomized sampling algorithms.
FrontierCS 2 Challenges AI Agents to Rebuild Missing Research Code
A new benchmark strips the key contribution from 48 research papers and asks AI agents to rebuild it well enough to beat the authors' own code.
Artificial Analysis Finds Claude Sonnet 5.5 and GPT-6 Astra Fail Differently
A refreshed medical record review leaderboard puts Claude Sonnet 5.5 and GPT-6 Astra in a tie, with no model clearing 42%.
Mistral and NVIDIA Point Mistral Large 4 at Industrial Engineering Simulation
Mistral and NVIDIA are teaming up to accelerate physics simulation, publish open benchmarks for CAE and EDA, and validate agentic AI on real engineering tasks.
Zed Beats 25 AI Search Methods With Simple Weighted BM25 for MCP Tools
Zed benchmarked 25 MCP tool search methods across 150 queries and 536 tools, then shipped weighted BM25 with normalization rules, running fully local.
Vals AI's ClaimBench Shows Perplexity's Decider V1.2 Beats Finance Verification at $0.10
Vals AI's new ClaimBench pits 15 models on 1,960 financial claims paired with 10-K excerpts, and small decision models win on cost.
Krea Launches Krea 3 to Edit 2K Images 71% Cheaper Than GPT
Krea 3 lands in the top ranks of image arenas with 2K edits under 8 seconds and $0.06 per image, roughly a third of competitor pricing.
Meta's Muse Spark Open Leads US Labs Storming the Open-Weights Frontier
Three US labs shipped open-weights models in a single week, reshaping the Pareto frontier of intelligence versus parameter count.
Rehearsals' Engram Beats GPT and Claude at Predicting Human Survey Behavior
Kaggle and Rehearsals launched a benchmark testing whether language models can predict real human choices across 1,199 survey tasks grounded in 1,244 US adults.
Sakana AI's QD-Rec Builds Smarter Shortlists by Evolving Competing Hypotheses
Sakana AI's QD-Rec applies Quality-Diversity evolutionary search to LLM-powered recommendations, growing archives of varied, well-justified picks instead of ranked lists.
DeepSeek V4.1 Flash Beats GPT-6 Sol in a Simulated Cyber Attack
A preliminary Vals AI test shows an open-weight DeepSeek model outperforming OpenAI's gated cyber tier on a multi-step network attack scenario at a fraction of the cost.
KAIST's Chain of Retrieval Beats Standard Paper Search by 7.82%
A new framework retrieves full scientific papers by decomposing them into aspect-specific queries and iteratively expanding the search as a tree.
Vals AI's Vals Index v2.2 Reshuffles Leaders With Stricter Grading
Vals AI shipped v2.2 of its GDP-weighted agent benchmark, tightening four component tests and reshuffling the frontier model leaderboard.
Ant Group's Ming-3.0 Quietly Tops Open-Source Speech Benchmarks at 93.2%
Ant Group quietly dropped an MIT-licensed 0.5B TTS model that ties Fish Audio for #1 pronunciation accuracy and leads all open weights in voice cloning.
Artificial Analysis' HealthBench Rankings Show Verbose AI Medical Answers Lose Big
Artificial Analysis launched independent leaderboards for OpenAI's two health conversation benchmarks, surfacing sharp tradeoffs between answer length, cost, and clinical quality.
Black Forest Labs' FLUX 3 Max Generates 10-Second Videos 22x Faster
Black Forest Labs and fal shipped a post-trained FLUX 3 variant that renders a 10-second clip in 12 seconds, 22x faster than Seedance 2.5 at matched quality.
Artificial Analysis Ranks 36 AI Models by Writing Style and Personality
Artificial Analysis launched AA-PersonalityBench, scoring 36 models on 30 personality traits, 8 indexes and 21 programmatic communication metrics using judged pairwise comparisons.
Google DeepMind's XYEval Reveals One Bad Hint Tanks AI Agents by 46.7%
A new meta-evaluation framework shows frontier AI agents cave to confident but wrong user hints, with task scores falling as much as 46.7 percent.
Inception's Mercury 3 Hits 1,068 Tokens per Second, Beating GPT-6 Luna
Inception's new diffusion LLM hits 1,068 tokens/sec, beats GPT-6 Luna and Claude Haiku 5.5, and proves scaling laws apply beyond autoregressive models.
DeepSeek's V4.1 Flash 1017 Boosts Agent Scores 150 Elo at No Extra Cost
DeepSeek's updated Flash checkpoint climbs to joint #2 open-weights, cuts hallucinations from 96% to 67%, and keeps the same $0.30/$1.20 pricing.
ScoringBench Reshuffles AI Leaderboards by Grading Full Prediction Uncertainty
A new open benchmark reveals that tabular foundation models rank very differently when judged on full probability distributions instead of RMSE.
Apple's DFN5B-CLIP Hits 84% ImageNet Accuracy by Teaching AI to Pick Its Own Data
Apple's DFN5B-CLIP-ViT-H-14-378 keeps climbing Hugging Face charts, hitting 84.2% zero-shot ImageNet accuracy by rethinking how training data is filtered.
Shanghai Jiao Tong Builds AirQA to Stress-Test AI Research Agents on 14,000 Papers
A new benchmark of 1,246 questions over 13,956 AI papers exposes how badly current LLM agents handle real literature-review workflows.
No stories match these filters.