Prism ML's Ternary-Bonsai-2 Shrinks a 9B AI Model to 2.42 GB

Takeaways

Prism ML released Ternary-Bonsai-2-9B-gguf , a 9B VLM with ternary weights at ~2.42 GB.
Compresses roughly 7.5x from FP16 using {-1, 0, +1} weights with FP16 group scales.
Inherits Qwen3.5-9B hybrid attention and a 262,144-token context window.
Ships three GGUF packings: PTQ1_0 (2.00 GB), PQ2_0 (2.42 GB default), Q2_0 (2.56 GB).
Needs the PrismML llama.cpp fork for ternary kernels; Q2_0 runs upstream.
Includes built-in MTP speculative decoding and optional DFlash2 drafter for faster inference.

Prism ML has released Ternary-Bonsai-2-9B, a quantized vision-language model derived from Qwen3.5-9B. Its default GGUF compresses the roughly 8.95-billion-parameter language backbone from about 17.9 GB at FP16 to 2.42 GB, while the vision components ship separately. The model repository showed more than 896,000 downloads and 3,700 likes at the time of writing, though Hugging Face totals can include repeated and automated downloads.

The smaller weight files expand the hardware available for local inference, including laptop GPUs, Apple silicon and CPU-only systems. Several constraints shape deployment: the smallest formats require Prism ML’s llama.cpp fork, runtime memory exceeds the GGUF file size, and using the full 262,144-token context window adds substantial memory and latency.

How 9B parameters fit into 2.42 GB

The language model stores its major weight matrices with ternary values from {-1, 0, +1} , paired with a shared FP16 scale for each group. The scheme covers embeddings, attention projections, MLP projections and the language-model head. Custom kernels read the packed representation during inference, avoiding a full FP16 copy of the weights.

A blockwise Hadamard rotation is folded into the stored weights, and the runtime applies the corresponding transform to activations. The rotation spreads large outlier values across dimensions, reducing the quantization error caused by mapping weights into three values.

Packing	File size	Layout	Runtime support
PTQ1_0	2.00 GB	Dense trits; smallest file	Prism ML fork
PQ2_0	2.42 GB	Two-bit slots; default and fastest to unpack	Prism ML fork
Q2_0	2.56 GB	Upstream-compatible layout	Prism ML fork and upstream llama.cpp

The default PQ2_0 file is about 7.4 times smaller than the 17.9 GB FP16 language model. That ratio applies to the language weights on disk; it excludes the optional vision package, context state, compute buffers and runtime overhead.