Real-Time Voice Agent & LangGraph Orchestration
The headline
Fifteen minutes of spoken practice costs about $7.5 with a human tutor at market rate. The same fifteen minutes on this pipeline costs cents, and 1,000 learners can run at once, because the ceiling is servers and not people.
$7.5 = $30 an hour ÷ 4, using the market rate for a human tutor. The cents figure is streaming-ASR unit price × minutes + LLM token price × duration, with the unit prices still to be filled in from a real run. The distance between the two is about two orders of magnitude.
What you are buying
You are buying a real-time voice agent and the LangGraph state machine that drives it. The serial chain — record, transcribe, think, synthesise — is replaced by a full-duplex WebSocket with streaming VAD, so both sides can speak at once. The model is not allowed to answer on its own: brain_node rules first (PASS, FIX_NEEDED, HELP_REQUEST) and action_node writes the feedback only after that ruling, which is why every line the coach speaks is short and on topic. The aim is to move a demo that can chat into a system your staff practise against every day.
What this replaces
- The money leaks at the hiring step. A human tutor or corporate trainer bills $30+ an hour at market rate, so fifteen minutes is about $7.5. Put 50 new salespeople through script practice and the first thing that has to say yes is the budget sheet, not the technical review.
- The seconds leak inside the product. A half-duplex chain waits for the learner to stop, then records, transcribes, sends the text to the LLM and synthesises speech — 3 to 5 seconds on every turn, the industry norm for that architecture rather than our measurement. Spoken practice depends on being able to interrupt and be interrupted, and those seconds are what remove it. A general-purpose model cast as a coach also writes essays, where the learner needs a micro-lesson and a line to shadow.
- The bill leaks wherever engineering takes the shortcut. Streaming-voice API keys get hardcoded into the mobile client to shave the first response, and one reverse-engineered packet capture lifts that key out of the app. After that the spend is no longer yours to control.
Architecture
Three stages left to right: the on-device audio gateway, the LangGraph state machine, and the session-asset sink. The token-auth checkpoint sits on the WebSocket line between the first two, and a dotted loop runs back from the brain to the gateway so context survives a reconnect.
Guardrails
- Short-lived token auth instead of a key inside the app. iOS calls /session/start first; the backend debits the account’s compute credits, and only then asks the streaming-voice vendor for a temporary token — 15 minutes of life, usage:write scope — which the client uses to open the WebSocket. The vendor API key never enters the mobile binary.
- Immunity to interruption and to dropped sockets. The AVAudioSession lifecycle observers are rewritten at the low level, so an incoming call, a move to the background or a cell-tower handover that kills the socket enters exponential backoff. When the app returns to the foreground, reconnectSession rebuilds the audio chain within 1 second, and the training context is still there.
- A typed floor under the report. The async report depends on a high-complexity prompt, so when the JSON breaks or a key field is missing, Pydantic on the backend and init(from decoder:) on iOS both drop into type fallback and fill N/A. The learner’s chart does not break.
What you end up holding
- iOS native streaming audio engine — the Swift / AVFoundation duplex transport and UI components, including StoreKit 2 credit-pack management and the SwiftUI radar chart.
- Multi-agent orchestration gateway — the stateless FastAPI backend, carrying the full LangGraph workflow routing network and the Vercel Blob asset pipeline.
- Commercial-grade storage and permissions architecture — the Supabase RLS data model, real-time user credential distribution and the account security system.
4 to 6 weeks, from business-logic abstraction and LangGraph node orchestration down to the iOS audio-layer coupling. A planning range assessed by complexity, not a contract commitment.
Stack
- Orchestration
- LangGraph · OpenAI GPT-4o · Qwen (DashScope API)
- Voice streaming
- Deepgram (Aura TTS / Nova-2 STT) · Hume EVI WebSocket
- Client & data
- SwiftUI · AVAudioEngine · Supabase (Auth / RPC / DB) · FastAPI (httpx)
The figures, with their arithmetic
- Target: under 500 ms end to end
- A target, not a measurement. It is still to be instrumented, and the architecture it rests on is a full-duplex WebSocket with streaming VAD rather than a serial chain.
- 3–5 seconds a turn on the serial architecture
- The industry norm for half-duplex design, not our measurement. The figure this pipeline is built against is the sub-500 ms target above.
- $30+ an hour for a human tutor
- An industry quote, neither our data nor our price. Fifteen minutes of it is about $7.5, at $30/hr ÷ 4.
- 15 minutes of practice: cents
- An arithmetic projection, not yet instrumented: streaming-ASR unit price × minutes + LLM token price × duration, with the unit prices still to be filled in from a real run. Against the tutor’s $7.5 that is about two orders of magnitude.
- 1,000 concurrent sessions
- A server-side orchestration capacity, not a staffing figure and not a load test. A human tutor takes one learner at a time.
The same shape, three other problems
This is the part that matters if the featured use case is not yours: the same architecture pointed at three unrelated situations, none of them a variation on the case above, and none of them requiring a rebuild. These are transfer directions rather than delivered engagements — the system above is what we have run, and the three below are what its shape solves next. We label the difference rather than blur it.
Enterprise sales and support enablement
load the standard script SOP library, and the pipeline generates a hard-to-please customer agent from it. 1,000 reps practise at once, and the session captures pause latency and off-script lines and scores them by RAG.
Clinical CBT support
24-hour low-latency voice companionship. brain_node watches for high-risk emotional language in real time; crossing the red line leaves the small talk and hands over to the emergency response chain.
Executive media rehearsal
a cloned voice bank poses long, pointed questions. The trainee has to keep their logic intact under pressure and sudden interruption, and answer with a BLUF — bottom line up front.
Have a system that needs building?
Write down the situation. You get a written, specific response: no calendar, no discovery call, no fifteen minutes of introductions.
Status
Built in-house. This is an architecture asset we built for ourselves to prove our own real-time voice and state-machine orchestration — not a client engagement, and no external organisation runs it. It runs in our own stack.
Built in-house. Running in our own stack.