Qwen: a self-hosted voice assistant
A voice-first assistant I run on my own hardware. It listens, thinks with a local language model, speaks back, and can use tools like search, notes, reminders and my homelab's status.
- Stack
- Node.js · TypeScript · WebSocket · SQLite · Docker
- Speech
- Silero voice-activity detection, local speech-to-text and text-to-speech servers
- Brain
- A Qwen-family model served by llama.cpp on my GPU nodes
- Version
- 0.2.0, running continuously on the home server
- Access
- Login required. Not public, no download
This is the orchestrator behind my home voice assistant. A browser page streams microphone audio over a WebSocket. The server detects when I stop talking, sends the text to a local language model, streams the answer back, and speaks it with a local text-to-speech server. Nothing goes to a cloud AI service.
What it does
- Voice conversation with turn detection (Silero VAD running in ONNX), so it can tell when I’m done speaking.
- Tool use. The model can search the web and read pages, check the time and weather, do math, save and recall notes, set reminders, and report on the homelab’s services.
- File uploads, including Word and Excel documents, that it can read and discuss.
- Saved conversations in SQLite, with the ability to edit a message and re-run the answer.
- Login with lockout after repeated failures.
Where it stands
It works and I use it, but it’s a personal tool. It runs behind a login and relies on model servers spread across my own machines, so there’s no public demo or download. A 3D animated face for the page is an experiment that isn’t finished.
Related reading: the model bake-off posts show how I pick which models go on which GPU.