The Bigger Model Answered Faster: Latency Lessons from a Real-Time Voice Agent

Tue, 30 Jun 2026 00:00:00 +0000

This is Part 3 of Field Notes from a Korean Phone Voice Agent, a seven-part series about a project I led: a real-time Korean phone voice agent (STT → LLM → TTS) for a public-service call line. Part 1 covered how my STT evaluation misled me, and Part 2 where rules beat models. This part is about latency: where the seconds went, and why the fixes that mattered were almost never “use a smaller model.”

Tensorrt on Juntak Noh — AI Notes

The Bigger Model Answered Faster: Latency Lessons from a Real-Time Voice Agent