A Developer Compiled a Full RISC-V CPU Into Qwen3.5's Weights
Qwenputer compiles a 32-bit RISC-V CPU directly into Qwen3.5-4B's weights , no external runtime. Gated DeltaNet layers become addressable RAM via one-hot keys with gate and write strength pinned to 1. Benchmarks: 100% on NUPA versus 48.6% base Qwen; 97.5% on DeepMind mul_big versus 55% base.
Takeaways
- Qwenputer compiles a 32-bit RISC-V CPU directly into Qwen3.5-4B's weights , no external runtime.
- Gated DeltaNet layers become addressable RAM via one-hot keys with gate and write strength pinned to 1.
- Benchmarks: 100% on NUPA versus 48.6% base Qwen; 97.5% on DeepMind mul_big versus 55% base.
- CPU passes official riscv-tests, matches qemu, ran 18M instructions free-running with zero errors.
- A filesystem variant stores 512 files in state and survives across sessions.
- Code open-sourced on GitHub ; 18.8 GB checkpoint, slow at ~3,200 CPU tokens/sec on L4.
Divyansh Agrawal’s Qwenputer embeds a compiled 32-bit RISC-V processor in a modified Qwen3.5-4B checkpoint. Eleven compiler-generated transformer layers sit before Qwen’s original 32 layers and encode the processor in roughly 1 million nonzero weights. Most remaining positions in the added tensors contain zero padding, and the base model’s weights remain unchanged.
Programs written in C or MicroPython execute through the network’s repeated forward passes during autoregressive decoding. Qwenputer stores registers and RAM in Gated DeltaNet state, emits tokens that advance the processor, and relies on a custom driver to load firmware and remove execution traces from visible output. Arithmetic and processor state transitions occur in the compiled transformer layers, without a sidecar Python interpreter or sandbox performing the calculation.
The project provides a concrete implementation of deterministic in-model computation and persistent recurrent memory without additional training. Reported gains concentrate on arithmetic tasks. The prototype also requires a custom runtime, tokenized CPU traces, a large checkpoint, and explicit state management.
Eleven layers turn recurrent state into RAM
Qwen3.5 uses Gated DeltaNet in three of every four layers. Gated DeltaNet is a linear-attention mechanism with a fixed-size recurrent state matrix. A learned gate controls how much old state remains, while a write-strength value called beta controls each update.
Agrawal fixes the gate and beta at 1, then supplies one-hot keys that select individual rows. The resulting memory supports exact overwrite and read operations at specific addresses. Its dimensions remain fixed as the sequence grows.
The inserted layers implement an RV32IM processor with Zba instructions and a custom big-number unit. RV32IM combines RISC-V’s 32-bit base integer instruction set with multiplication and division. Zba adds address-generation operations, while the big-number unit supports long-integer arithmetic. Registers, memory, and the program counter occupy rows in the DeltaNet state.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
