Thinking Machines' Inkling 2 Small Trains a 594B Model in 68 Hours
Thinking Machines open-sourced Inkling 2 Small , a 594B MoE with 17B active params, under Apache 2.0. Pretrained end-to-end natively in NVFP4 on GB300 NVL72 racks in just 68 hours over 8.4T tokens. Scores 31 on Artificial Analysis Intelligence Index, up from 13 on Inkling Small.
Takeaways
- Thinking Machines open-sourced Inkling 2 Small , a 594B MoE with 17B active params, under Apache 2.0.
- Pretrained end-to-end natively in NVFP4 on GB300 NVL72 racks in just 68 hours over 8.4T tokens.
- Scores 31 on Artificial Analysis Intelligence Index, up from 13 on Inkling Small.
- Leads open-weights peers on PerceptionBench, ScreenSpot Pro, ARC-AGI-2, BrowseComp; trails on coding.
- Runs on 2x B300 or 4x H200 at 500GB VRAM minimum; supports 1M context and image input.
- Priced at $0.30 in / $1.00 out per million tokens on Tinker ; works in vLLM and SGLang.
Thinking Machines Lab has released the model weights for Inkling 2 Small under Apache 2.0. The 594-billion-parameter mixture-of-experts model was pretrained with native NVFP4 arithmetic on NVIDIA Blackwell hardware, processing 8.4 trillion tokens in 68 hours. It activates 17 billion parameters per token and scores 31 on the Artificial Analysis Intelligence Index, up from 13 for the original Inkling Small.
The technical significance comes from using four-bit floating-point operations during pretraining instead of converting a higher-precision checkpoint after training. The release notes provide evidence that low-precision training can support a large sparse model while reducing memory traffic and increasing tensor-core throughput on compatible hardware. Actual cost savings depend on cluster size, utilization, communication overhead and the higher-precision states retained by the training recipe.
594B weights, 17B at work
Inkling 2 Small combines sparse expert routing, hybrid attention, image input and an advertised one-million-token context window. Its DSpark speculator accelerates generation by drafting tokens for the main model to verify.
| Component | Specification |
|---|---|
| Architecture | 48-layer, decoder-only transformer with sparse mixture-of-experts feed-forward blocks |
| Parameters | 594B total, 17B active per token |
| Routing | Each token uses 8 of 384 experts |
| Attention | Hybrid local and global attention |
| Modalities | Text and images in, text out; a vision transformer projects images into the shared hidden space |
| Context | Up to 1M tokens for inference; reinforcement learning used a 256K training context with compaction for longer tasks |
| Checkpoints | NVFP4 and MXFP8, with NVFP4 recommended by the lab |
| License | Apache 2.0 |
A mixture-of-experts model loads its full checkpoint across the available devices while routing each token through a small subset of experts. Inkling therefore has the storage and communication demands of a 594B checkpoint, even though its per-token expert computation is closer to a 17B model.
FP4 enters pretraining
Many low-bit releases begin as BF16 checkpoints and are quantized after training. Thinking Machines instead used NVIDIA’s NVFP4 format during pretraining, including the model’s principal forward and backward matrix operations.
NVFP4 represents values with four-bit floating-point numbers and shared scales for small groups of values. Those scales preserve more useful range than plain four-bit integers. Compared with FP8, four-bit operands can halve storage for quantized tensors and double peak tensor-core throughput for supported Blackwell operations.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
