Tamara Tran's fast-jev-compaction Stops Claude Code From Forgetting Critical Tool Output
A new open-source Claude Code plugin uses TypeSafe's Jev decision model to prune agent context per tool call instead of summarizing it away.
// Intelligence
Infra
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
Perplexity Computer Adds Four Effort Presets to Control AI Task Depth
Perplexity Computer now lets users pick effort levels from Light to Ultra, and the orchestrator picks the right model and reasoning depth automatically.
vLLM Boosts Kimi K3 Throughput by 2.8× With Smarter Scheduling
vLLM's latest optimizations push Kimi K3 serving to 2.2 to 2.8x higher throughput on B300 GPUs, with TTFT cut by up to 85%.
Epoch Maps 44% of Global AI Compute Across 86 Data Centers
Epoch AI's data center explorer now maps 86 sites covering an estimated 44% of global AI compute, with satellite imagery and detailed hardware specs.
Cohere's Model Vault Now Encrypts AI Prompts Even From Cloud Admins
Cohere is adding hardware-backed confidential computing to Model Vault, encrypting prompts and model activations inside NVIDIA GPUs during inference itself.
Bojie Li's Open Textbook Teaches AI Infrastructure Through Hardware Limits
Bojie Li open-sourced a Chinese-language AI infra textbook that derives inference and training system design from first-principles hardware constraints, with 3.4k stars in days.
Mia AI Lab Runs GLM-5.3-Flash Across Two DGX Sparks at 146 tok/s
A hobbyist lab shipped a two-node vLLM stack that runs GLM-5.3-Flash at 4bpw across a pair of DGX Sparks with 900k context.
Perplexity's CobbleDB Cuts Search Storage Latency by 82% Over DynamoDB
Perplexity rebuilt its search hot store from scratch, cutting batch-read latency 5x and slashing costs 20% versus DynamoDB.
Google's Retrieve-for-Train Slashes AI Search Latency by 20x
Google Research swaps expensive chain-of-thought fan-out for a 53.9M-parameter diffusion retriever, delivering 12-20x faster expert-level search slates.
Google's Firebase Spend Caps Auto-Pause Runaway AI Services Before Bills Explode
Firebase now lets you cap spending per service and auto-pause workloads at 100% of budget, protecting against runaway Gemini API and Cloud Functions bills.
What DeepSeek-V4.1-Flash teaches us about efficient AI
A 552B model with 890-byte KV cache, 8B active on input, and an Artificial Analysis 40 versus Gemini 3.8 Flash High at 41
Together AI's ThunderKittens Hits 22.4 PFLOPS on NVIDIA's Rubin
Together AI ported its ThunderKittens kernel framework to NVIDIA's Vera Rubin NVL72, hitting 22.4 PFLOPS on NVFP4 GEMMs and rivaling cuBLAS.
Edge0 Runs a 35B AI Model on a Mac mini Using SSD
Edge0 is an open source framework that streams MoE experts from SSD, letting a 35B parameter model run on a 24GB Mac mini with under 3GB active memory.
Claude Prompt Caching: 6 Tricks That Cut Your API Bill
Breakpoints, TTL math, Batch stacking, and a two-session split. Measured on OpenRouter, $0.003 vs $0.022 per turn
Intelligent Internet's Meta-Zenith Agent Rewrote vLLM Kernels for a 4x Speedup
An autonomous agent ran 111 trials to optimize vLLM for Qwen3.8-27B on a single RTX 5090, hitting 4.1x throughput at 65K context.
NVIDIA's CUDA Python 1.0 Makes Python a First-Class GPU Citizen
NVIDIA ships CUDA Python 1.0 with semantic versioning, a shared cuda.core foundation, and PyTorch and CuPy already building on it.
Cohere's Open-Source Megakernel Beats vLLM by 1.58x on H100
Cohere open-sourced a serving engine that runs the entire LLM decode step as one persistent CUDA kernel, hitting 1.58x vLLM throughput on H100.
vLLM's Hybrid HiSparse Triples Concurrent Requests on Million-Token Contexts
vLLM's new Hybrid HiSparse keeps long-context requests decoding when the KV cache overflows HBM, tripling concurrency on GLM 5.3 at 1M context.
Experiential Gives AI Agents One Open-Source Gateway to Rule All Models
Experiential is an open source, OpenAI-compatible gateway that routes across hosted, BYOK, and local models with zero markup and spend controls.
Perplexity's ROSE Serving Stack Beats vLLM on Speed and Latency
Perplexity open-sources details of Ivy, Tulip, and ROSE, its Rust plus Python embedding stack that beats vLLM on latency and throughput.
Prime Intellect Slashes GLM-5.2 Weight Transfer by 22x With RDMA
Prime Intellect rebuilt weight sync on RDMA and vLLM tracing, dropping GLM-5.2's 1.6TB policy transfer from 86 seconds to under 4.
Nous Research's Hermes Desktop Picks and Runs Local AI Models in One Click
Hermes Desktop now auto-detects your hardware, picks a fitting local model, downloads it, and configures the runtime without any manual setup.
FlashAttention-4's Direct-P Finally Unlocks 2.13x FP4 Speedup on NVIDIA GB200
A new attention kernel unlocks Blackwell's FP4 tensor cores for inference, hitting 2.13x BF16 forward throughput on GB200 while training stays partly in FP8.
Google DeepMind's WeatherNext 3 Ditches Physics Simulations for 60% Sharper Forecasts
Google DeepMind's new weather AI trains on raw satellite feeds and station data, delivering hourly 5km forecasts with up to 50% better rain accuracy.
No stories match these filters.