A voice assistant that runs entirely on one laptop.

It hears you, reasons about your code, and answers in a voice that belongs to nobody. No cloud, no API calls, no audio leaving the machine. The codec, the kernels and the voice were all built from scratch on a base M4.

Most voice assistants are a microphone attached to somebody else's datacentre. This one is not. Speech recognition, language model and speech synthesis all run on the machine in front of you, through around thirty-five hand-written Metal kernels, at conversational speed.

It started as a question about whether that was possible on consumer hardware, and turned into a longer argument about spending compute where the decisions are. Speech carries about five decisions a second and neural codecs run at twelve and a half frames a second, so most of the budget goes on regenerating texture that nothing depends on.

Writing