Liquid AI's RIPL Engine Beats vLLM by 28x on Real-Time Inference
Liquid AI's new RIPL inference engine runs small Liquid Foundation Models in under 50ms end-to-end, targeting search, ads, fraud, and payments.
// Intelligence
Infra
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
NVIDIA's Flare Ditches Python at Runtime to Serve AI Faster on Blackwell
NVIDIA's new Rust-based inference server targets DGX Spark and RTX Blackwell with ahead-of-time planning, FP8 KV cache, and no Python in the serving loop.
vLLM Matches DeepSeek's DeepInfer Within 6% on DeepSeek-V4.1-Flash
Open-source vLLM reproduces DeepSeek's own DeepInfer engine within 6% on decode throughput, hitting roughly $39/GPU/hour at the official API prices.
EAGLE-4 Triples Qwen3 Inference Speed by Adapting to Serving Load
A single adaptive drafter adjusts speculative decoding to batch size and hardware, posting up to 2.99x throughput on vLLM and 3x single-request speedups.
OpenAI Extends Prompt Caching to 12 Hours for Long-Running Agents
OpenAI added a 12-hour time-to-live option to API prompt caching, letting persistent agents keep shared instructions, tools, and context cached across the full workday.
Google and Meta Release TorchTPU, Letting PyTorch Run on 8,192 TPU Chips
Google and Meta ship a native PyTorch backend for TPUs with true eager execution, torch.compile, vLLM and SGLang support, and near-linear scaling to full pods.
Databricks' MemCurrent Triples AI Agent Success While Using 12x Fewer Tokens
Databricks unveils MemCurrent, a persistent memory system that triples agent pass rates while cutting tokens 31% and latency 26% in production.
New AI Architecture Challenges Token-by-Token Generation
Rough concept first, then the words. 10x to 31x faster than strong diffusion models, on 4x to 5x fewer training tokens
DeepSeek Opens DeepInfer, Exposing the Engine Running V4.1-Flash
DeepSeek open-sourced DeepInfer and deepseek-recipe, the production stack powering DeepSeek-V4.1-Flash, bringing their internal serving optimizations and API parity to anyone.
SGLang's Low-Latency Mode Boosts DeepSeek-V4.1-Flash Speed by 1.56x on AMD
SGLang's new plugin framework lets NVIDIA and AMD ship specialized low-latency kernels, cutting DeepSeek-V4.1-Flash decoder time by up to 1.56x.
SGS-SLAM Bakes Semantic Labels Directly Into 3D Gaussian Maps
A trending open-source SLAM system fuses 3D Gaussian Splatting with semantic segmentation to deliver object-aware dense reconstruction in real time.
Ephemeral AI Lab's LayerFS Gives Parallel AI Agents Forkable Isolated Filesystems
A Rust-based filesystem gives each AI agent a disposable, zero-copy fork of a shared base, enabling cheap branching, rollback, and parallel rollouts.
vLLM's Kimi K3 Backend Hits 842 Tokens per Second on NVIDIA Blackwell GPUs
A unified CuTe DSL decode path on GB300 pushes Kimi K3 to 842 tokens/s at batch 1, up to 2x faster than stock vLLM.
KiToke Keeps 89% Video AI Accuracy While Ditching 99% of Tokens
A training-free token compression method keeps Video LLMs accurate even when 99% of visual tokens are thrown away, with a 12.5x prefill speedup.
FlashMap Drafts 16 Dependent Tokens at Once, Hitting 6.1x Faster Decoding
A flow-map drafter conditions on hidden states to generate entire token blocks in one shot, hitting 6.1x lossless speedups on Qwen3-8B.
Replit's BYOC Lets Enterprises Run Its AI Coding Agent in Their own Cloud
Replit now lets enterprises run its agent platform and published apps inside their own AWS, Azure, or Google Cloud tenants with full data isolation.
Perplexity Opens SPACE, Its Internal Sandbox for Running AI Agent Code
Perplexity open-sourced SPACE, a Rust-based platform that runs untrusted agent code inside Firecracker microVMs with an E2B-compatible HTTP API.
Anthropic's Claude Mythos 5 Cuts Quantum Circuit Gates by 17x
Anthropic's STEM Fellows gave Claude Mythos 5 plain-language briefs for eight quantum circuits, and it beat the best published designs, sometimes by over 10x.
Factory's Droid Now Takes Coding Tasks Directly From Slack and Jira
Factory makes Delegations generally available, letting teams hand coding tasks to its Droid agent directly from Slack, Linear, Jira, and Microsoft Teams.
Fish Audio's Drama 3 Beats ElevenLabs on Pronunciation With 35B Open Model
Fish Audio's new 35B mixture-of-experts TTS model takes bracketed stage directions, tops open-weight leaderboards, and ships free under a revenue cap.
Cognition's Devin Oncall Sends an AI Agent to Fix Production Alerts
Cognition's new SRE agent plugs into Datadog, Sentry and Dynatrace to triage alerts, run investigations and open pull requests for fixes.
vLLM-Omni Now Serves Fish Audio's Drama 3 at 128 Concurrent Streams
vLLM-Omni now serves Fish Audio's 35B directable TTS model with 128 concurrent streams on one H200 and 255 ms median time to first audio.
Baidu's ProxyAttn Cuts Long-Context AI Processing 10x Without Retraining
A training-free sparse attention method uses a handful of representative attention heads as proxies, delivering 10.3x attention speedup on long contexts.
KV cache techniques for faster, cheaper multi-model AI agents
Nvidia's Qwen3 14B to 32B mapper is 25.1x vs re-prefill at 32,768 tokens. Cache-to-Cache reports about 2.5x against text
No stories match these filters.