The voice assistant that runs entirely on your laptop.
It hears you, reasons about your code, and answers in a voice that belongs to nobody. No cloud, no API calls, no audio leaves the machine. The codec, the kernels and the voice were built from scratch on a base M4.
Most voice assistants are a microphone attached to somebody else's datacentre. This one runs on the machine in front of you: speech recognition, language model and speech synthesis, through about thirty-five hand-written Metal kernels, at conversational speed.
It started as a question: can this run on consumer hardware? It can. The harder problem is where the compute goes. Speech carries about five decisions a second and neural codecs run at twelve and a half frames a second, so most of the budget goes on regenerating texture that nothing depends on.
Writing
28 July 2026
From 22.8 to 42.8 tokens a second without touching a matrix multiply
A speculative-decoding plan killed by the criterion written before it started, and the profiler it left behind, which found one serial norm kernel costing every token forty percent of its time.
22 July 2026
Rendering speech with 36 million parameters instead of 600 million, and it comes out clearer
The speech decoder was a 600M model working through fifteen codebooks a frame. A 36M head that renders continuous latents from the frozen backbone's hidden state does the job, and out-decodes its teacher.
13 July 2026
I taught a locked text-to-speech model to speak in a voice that doesn't exist
The model conditions its voice on a tensor from an encoder that was never released, so there is no supported way to make a new voice. The tensor is numbers, and numbers can be optimised.
- Hardware
- MacBook Pro, base M4, 16 GB
- Hearing
- Kyutai STT, reimplemented in Rust
- Reasoning
- Qwen3 4B Instruct, int4, about 34 tokens a second
- Speaking
- Kyutai TTS 1.6B with an inverted voice
- Codec
- Mimi at 24 kHz, 80 ms frames
- Cloud services
- None