I built a sub-500ms latency voice agent from scratch
AI
voice agent
SHOW HN2026
Legend
I built a sub-500ms latency voice agent from scratch
A sub-500ms latency voice agent from scratch
ntik.meHow HN took it
18 positive, 31 neutral or asking, 1 mixed, 10 critical, out of 60 comments about the project. Thread score +0.18.
Idea: 10 for, 6 againstPerformance: 2 for, 3 againstDocs: 3 for, 1 againstDesign: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Clear writeup and explanation of turn-taking and latency
From 3 comments
This is an outstanding write up, thank you! Regarding LLM latency, OpenAI introduced web sockets in their Responses client recently so it should be a bit faster. An alternative…
armcat on HNThis is a really solid writeup. The streaming pipeline architecture, the detailed latency breakdown per stage are genuinely useful. Building the core turn-taking loop from…
stonelazy on HNPretty exciting breakthrough. This actually mirrors the early days of game engine netcode evolution. Since latency is an orchestration problem (not a model problem) you can beat…
NickNaraghi on HN
Interruption handling feels natural
From 1 comment
The voice samples sound fantastic The interruption handling is amazing. I felt you were talking to an actual person. It might have helped that he had a British accent :)
aanet on HN
Objections
Latency around 0.4-0.5 seconds still feels slow in conversation
From 1 comment
Thats half a second delay. 0.4 to 0.5 seconds. Thats the same as the delay in a GEO orbit satellite mediated phone conversation. Perhaps I'm in an older cohort, but I remember…
ggm on HN
Post reads as AI-generated
From 1 comment
This post is AI except you manually replaced the em dashes with hyphens
eudamoniac on HN
The 2x Vapi comparison ignores tool calls and platform work
From 1 comment
This is a really solid writeup. The streaming pipeline architecture, the detailed latency breakdown per stage are genuinely useful. Building the core turn-taking loop from…
stonelazy on HN
Requests
Open-source end-to-end or offline version
From 2 comments
Does anyone know about a fully offline, open-source project like this voice agent (i.e. STT -> LLM -> TTS)?
docheinestages on HNIMO STT -> LLM -> TTS is a dead end. The future is end-to-end. I played with this two years ago and even made a demo you can install locally on a gaming GPU:…
modeless on HN
Caching common phrases for lower latency
From 2 comments
Depending on the TTS model being used latency can be reduced further yet with an LRU cache, fetching common phrases from cache instead of generating fresh with TTS. However the…
hosaka on HN100% - I thought about that shortly after writing this up. One way to make this work is to have a tiny, lower latency model generate that first reply out of a set of options,…
nicktikhonov on HN
Questions
How is semantic end-of-turn detection done and at what cost
From 3 comments
How are you doing semantic end-of-turn detection without adding latency to the critical path? Is it a separate lightweight model or integrated into the LLM stream?
swaminarayan on HNThe insight about TTFT dominating everything resonates. We're seeing the same pattern in CLI tools — the perceived speed of AI features comes down to how fast you get the first…
recognity on HNWhat about the costs?
nthypes on HN
How long-term memory and context are handled
From 1 comment
This is awesome! Exactly the kind of low-latency agent tooling I've been looking for. How are you handling long-term memory/context between calls?
mjbonanno on HN
Alternatives named
Pipecat and LiveKit Agents
From 1 comment
Nice write-up, thanks for sharing. How does your hand-vibed python program compare to frameworks like pipecat or livekit agents? Both are also written in python.
loevborg on HN
Soniox real-time STT with endpoint detection
From 1 comment
Or you could use Soniox Real-time (supports 60 languages) which natively supports endpoint detection - the model is trained to figure out when a user's turn ended. This always…
lukax on HN
NVIDIA PersonaPlex
From 1 comment
I didn't try Soniox, but I made a note to check it out! I chose Flux because I was already using Deepgram for STT and just happened to discover it when I was doing research. It…
nicktikhonov on HN
Dograh open source voice agent
From 1 comment
Great writeup. The speaking vs listening framing is underrated. TTFT with Groq and colocation are both real wins that don't get talked about enough. For anyone wanting this…
pritesh1908 on HN