ISTA-DASLab Squeezes DeepSeek's 763B Model onto a 12 GB GPU
ISTA-DASLab shrinks DeepSeek-V4.1-Flash-1017 to under 2 bits per weight, letting a 763B MoE run locally on a single 12GB consumer GPU.
// Intelligence
Gpus
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
gpu-unlock Gives Nvidia RTX 3090 Ti Nearly 80% More AI Throughput on Linux
An open source Linux kernel module removes the FP16 and BF16 throttle on Ampere GeForce cards, roughly doubling tensor throughput on an RTX 3090.
humans& Beats FlashInfer's Best MoE Kernel by 25% on Blackwell GPUs
Humans& rebuilt the NVFP4 decode kernel on Blackwell by keeping weights at 4 bits while promoting activations to BF16, gaining both speed and accuracy.
Alibaba's Qwen2.5-3B Shrinks to 2.5GB so Anyone Can Run AI Locally
A 4-bit quantized build of Qwen2.5-3B-Instruct runs chat, coding and long-context tasks on roughly 2.5 GB of VRAM.
Abiray Shrinks Minimax-H3 Singularity V2 From 34 GB to 12.5 GB
A community pruning of the Minimax H3 Singularity V2 fine-tune shrinks the 34GB checkpoint to 12.5GB while keeping INT8 quality intact.
Google Completes Open-Source Coral NPU Stack for Always-On Edge AI
Google completes its open 32-bit Coral NPU RISC-V design, ships a v1.0 MLIR compiler, and lands commercial Synaptics silicon hitting 1 TOPS under 10 mW.
PyTorch's FlyDSL Boosts AMD MI350 GPU Speed by 3x With Low-Precision Math
AMD and Meta's new FlyDSL backend plugs into PyTorch on MI350-series GPUs, delivering up to 3.01x end-to-end speedup for vLLM inference workloads.
Flapping Airplanes Raises $1B to Collapse Entire AI Training Into One GPU Kernel
A year-old research lab led by the creators of ThunderKittens just raised $1B at a $6B valuation to rebuild how GPUs run AI workloads.
AgentionAI's Gyro-Coder Squeezes a 125B Coding Model onto a 24GB GPU
A 125B mixture-of-experts coding model squeezed onto a single 24GB consumer GPU through evolutionary expert pruning and 2.125-bit trellis quantization.
Ant Group's Humming Open-Sources Faster AI Inference, Hitting 5x Gains
Ant Group and vLLM open-source Humming, a unified JIT kernel library delivering up to 5x speedups and 14-43% end-to-end MoE throughput gains.
Google's EmbeddingGemma 2 Brings Multimodal Search to Your Device
Google's new 740M multimodal embedding model runs locally across text, images, and audio, powering a Mac meeting companion and mobile search demos.
Google and Meta Release TorchTPU, Letting PyTorch Run on 8,192 TPU Chips
Google and Meta ship a native PyTorch backend for TPUs with true eager execution, torch.compile, vLLM and SGLang support, and near-linear scaling to full pods.
Perplexity's Computer Agent Now Runs Fully Private on Apple Silicon Macs
Perplexity's agent now runs the model, planner, and tool execution entirely on Apple silicon, keeping private files and credits off the cloud entirely.
Kindling AI Shrinks GLM-5.3's 744B Expert Weights to Fit Memory-Constrained Clusters
Kindling AI uploaded a preview EXLR8 v2 quantization of Z.ai's 744B GLM-5.3 MoE, tuned for NVIDIA GB10 unified-memory boxes.
SGLang's Low-Latency Mode Boosts DeepSeek-V4.1-Flash Speed by 1.56x on AMD
SGLang's new plugin framework lets NVIDIA and AMD ship specialized low-latency kernels, cutting DeepSeek-V4.1-Flash decoder time by up to 1.56x.
Liquid AI's LFM2.5-24B Runs 184 Tokens per Second for Local Agents
Liquid AI's new 24B mixture-of-experts activates only 2B parameters per token, delivers 184 tokens/sec on an M5 Max, and ships in GGUF for llama.cpp.
vLLM's Kimi K3 Backend Hits 842 Tokens per Second on NVIDIA Blackwell GPUs
A unified CuTe DSL decode path on GB300 pushes Kimi K3 to 842 tokens/s at batch 1, up to 2x faster than stock vLLM.
ReadyArt's Wicked-Tutu-31B Brings Private Mature Roleplay to Local Hardware
ReadyArt drops a 31B uncensored Gemma-based roleplay model in GGUF format, built for explicit fiction, horror, and violence on local hardware.
NVIDIA's Vera Rubin Delivers 5.7x More AI Agent Power per Megawatt
NVIDIA's Vera Rubin NVL72 posts up to 5.7x the throughput per megawatt of GB300 on realistic agentic coding benchmarks.
OpenBMB's MiniCPM-V 4.7 Runs Full Vision AI on Your Phone at 1B Parameters
OpenBMB's new 1B vision language model pushes the performance-efficiency frontier for on-device multimodal AI, beating Qwen3.5 0.8B while running on phones.
Offrail 27B Squeezes a Full AI Model Into an 8GB Gaming GPU
A 27B uncensored model with multi-token prediction runs at 50 tok/s on an RTX 3060, fits on 8GB cards, and refuses nothing.
QuantTrio Shrinks Alibaba's Qwen3.6 From 70 GB to 24 GB for Workstation Deployment
QuantTrio shipped a 4-bit AWQ quantization of Qwen3.6-35B-A3B that compresses the MoE down to roughly 24GiB for vLLM deployment.
Google DeepMind Shrinks Gemma 4 E4B to 5GB Without Losing Reasoning Power
Google released quantization-aware trained Q4_0 GGUF checkpoints of Gemma 4 E4B, delivering near-bfloat16 quality in a 5.15 GB on-device package.
Prime Intellect Buys Moondream to Double Vision AI Throughput
The neocloud is folding Moondream's open vision models, Photon inference engine, and megakernel compiler into its stack to chase cheaper multimodal inference.
No stories match these filters.