Know It, Comply, Evaluate: Voice Compliance and Evaluation
Prerequisite: Lesson 08's handoff envelope and its transfer gate, plus lesson 06's progressive control — the deterministic gate the LLM doesn't get a vote on. This lesson assumes you already accept that some seams need a hard gate instead of a prompt instruction. After this lesson, you can: name the two regulatory regimes that govern a voice agent's disclosure, explain why disclosure has to be a gate rather than a prompt line, and pick the voice-specific metrics that actually predict abandonment instead of borrowing a chat agent's eval suite.
Six failures, six fixes. But shipping a voice agent in 2026 means two more things beyond getting the orchestration right: knowing the law that governs the call, and knowing whether your agent is actually working before a caller finds out for you.
Know it, comply: disclosure is a gate, not a sentence
Two regimes matter here, and both have teeth.
| Regime | Requirement | Enforcement |
|---|---|---|
| EU AI Act, Article 50 | Every EU caller told, at call start, in their own language, that they're talking to AI; the synthetic voice must be machine-readable as AI-generated | Begins August 2, 2026, fines up to €15M or 3% of global turnover |
| US FCC / TCPA | AI-generated voices ruled "artificial" (Feb 2024), bringing them under consent-and-disclosure requirements | Lingo fined $1M; the consultant behind the fake-Biden robocalls fined $6M+ |
If your stack is stitched together from multiple vendors, there's a second EU problem underneath the first: the AI-generated watermark has to survive a chain of custody across the SIP boundary, and a multi-vendor stack is exactly where that chain gets weakest. Neither set of numbers is theoretical; both regimes have already fined real operators.
Pindrop, measuring across 1.2 billion calls, found deepfake voice-fraud attempts rose more than 1,300% in 2024. That's the reason voice biometrics gets treated as one signal among several today, never the sole authentication factor: the same synthetic-voice capability that makes your agent sound natural is available to whoever's trying to impersonate your customer.
Why disclosure has to be a gate
Here's the connection worth making explicit: disclosure is not a line you add to the system prompt and hope the model reads at the right moment. It's the same shape as lesson 06's progressive-control gate: agentic by default, with a deterministic hard checkpoint at the seam that matters. The disclosure has to fire before the LLM gets the mic, every time, on every call, regardless of what the model decides is a natural place to mention it. A soft instruction that fails roughly one time in five (lesson 06's own measured failure rate for exactly this class of instruction) is not a compliance posture with an enforcement date attached to it. The gate doesn't ask the model's opinion; it runs first.
Evaluate: chat metrics don't see what voice needs
The other half of shipping responsibly is knowing whether the agent is actually working, and most teams reach for the eval suite they already know: word error rate, task completion percentage, and someone occasionally listening to a handful of calls. None of it catches the failures this course has spent eight lessons on. WER doesn't tell you the agent cut someone off. Task completion doesn't tell you a reply took 1.6 seconds too long. Spot-checking transcripts doesn't catch a wrong number spoken with total confidence.
Word error rate, task completion rate, and manual transcript spot-checks. These work reasonably well for a chat agent, where a reader has time to notice something is off and there's no clock running on the interaction itself.
None of them are timing-aware or interaction-aware, which is exactly what a phone call is.
The mechanism worth naming is bot-to-bot simulation, and it runs in three steps, in order:
A persona agent plays the caller: cooperative, rambling, adversarial, whatever profile the test needs. Golden personas form your regression set, run on every commit. Adversarial personas are your red team, run against every change that could plausibly break turn-taking or interrupt handling.
Your voice agent answers exactly as it would in production, no shortcuts, no mocked audio.
A judge model scores the resulting conversation for task completion, turn-taking, and repairs, producing a pass/fail per call rather than a transcript someone has to read.
A reference demo built for this course runs a cooperative persona and a rambling persona against the same agent. The judge scores task completion, turn-taking, and repairs per call, and it catches a turn-taking regression on the rambling persona that a transcript-only check would never surface, because the transcript still reads fine even when the timing was wrong. In production, this same pattern runs as real audio calls through simulation vendors in this category: Cekura, Coval, Hamming, and Future AGI are the ones worth knowing by name, cited here as the category this pattern belongs to, not as an endorsement of any one of them.
The takeaway
It isn't testing your voice agent. It's testing a chat agent that happens to have a TTS layer bolted on. And if your disclosure line lives in a system prompt instead of a gate, you're one long call and one distracted model away from an enforcement date that doesn't care about your intent.
The capstone: six failures collapse into five patterns, and every one of them is something the engineer building the agent already controls.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
