Voice agents can't spell your name

Let's imagine that your name is Camila Kowalski and you have to call customer support. The person who answers isn't sure whether your name starts with a C or a K, and so they ask you to spell it before they type it into their computer. Anybody working in customer support will do this; it's not in the training manual, it's just common sense.

We expect voice agents to be able to do it too, but we found out that they don't. This blog post talks about the problem and also how the newest VAmoS candidate - Dialt - does on it.

Putting voice agents to the test

We ran nineteen voice agents through VAmoS Pro Bench, our benchmark of simulated customer calls to a utility company. Every agent gets the same calls, from the same simulated callers. Before the agent can open account info, it has to verify the caller with (among other things) the customer's name.

All of the agents had access to the same traditional disambiguator for names: Soundex, an old phonetic code that ignores vowels and doubled letters, so “Hadad” passes for Haddad. But it keeps the first letter as written, so “Kamila” does not pass for Camila. Nevertheless, agents still manage to get your name wrong more often than you'd expect:

Share of calls where the first name the agent sent to the account check did not match the account, on calls without a TV in the room.

The first problem: speech-to-text

All of these voice agents, at some point in this task, need to convert speech-to-text. Cascaded agents always do it with a native speech-to-text model, but even speech-to-speech models need to type text into their tools. So long as databases expect text and human speech is audio, the problem is not going away.

Here is how often native speech-to-text got the name right in cascaded stacks.

Cascaded agents only: the share of calls where speech-to-text wrote down the caller's name well enough to pass the check, the first time the caller said it.

When the name came through wrong, it was usually something plausible: “Alice Chaudhary” for Elise Choudhury, “Farrakowski” for Farah Kowalski, “Elise had that” for Elise Haddad.

The second problem: trusting speech-to-text

Next, the agent has to decide whether to trust the speech-to-text or not. Humans do this all the time. When we hear somebody say their name, we know how confident we are in our spelling. And even if a human only saw the name spelled out and entered it into their system, when the system rejected it, they would know some names are more likely to get misspelled, so they'd ask again.

Voice agents mostly just trust it. When the text held a wrong name, the agent sent it to the check 85% of the time:

When the agent had a wrong version of the name, how often did they try to enter it in the computer?

dialt smart is the exception. It almost never sent a misheard name. Its replies suggest it gets more than one version of what the caller said, and asks when they disagree:

“I heard two versions of the first name: Zamina, with Z as in Zebra, or Amina, with A as in Alpha. Which is on the account?”

“I heard the name as ‘Farrakowski,’ but it also came through as ‘Kowalski.’ Could you repeat the full name on the account?”

This clearly paid off.

Agent asks again, but hears the same thing

A rejection from the computer is the clearest signal that the spelling was wrong. Most agents ask the caller to say their name again, but it rarely helps. Ask a caller to say “Camila” again, and the speech-to-text returns “Kamila” again.

The speech-to-speech models do this too. gpt-live 1 asked its callers to try again, heard the same thing, and sent “Kamila Kowalski,” “Kamila Kowalski,” “Kamila Kowalski.” After three failures the agent has to stop.

Which agents ask to spell

Half the stacks never asked anyone to spell their name, including both Gemini Live models, grok voice and both OpenAI Realtime models. dialt smart asked far more than anyone else:

Share of calls in which the agent explicitly double-checked the spelling.

“Is your first name spelled Farrah with two R's, or Farah with one?”

dialt fast asked less often, but after a rejection it produced the most human line we found. Its recognizer had transcribed the name as “Kovalski”, but the computer rejected it. So dialt asked whether the V should really be a W:

“Sorry about that — I just want to get the spelling right. Is it Kovalski, or Kowalski with a W?”

The caller spelled it out, and the next try went through.

If you're building a voice agent, keep these things in mind. The agent shouldn't trust a name just because the speech-to-text produced it. And when in doubt, it should just ask the human to spell it.