Accents and speech recognition: what research actually measures, and what to plan for on the phone

Allison Koenecke and her co-authors ran interviews with 42 white speakers and 73 Black speakers through five speech recognition systems developed by Amazon, Apple, Google, IBM and Microsoft. The corpus covers five cities or communities in the United States and totals 19.8 hours of audio, matched by the speakers’ age and gender. The recordings come from two sociolinguistic corpora: CORAAL, devoted to African American English, and Voices of California.

The metric used is the word error rate: the number of words substituted, deleted or added by the machine, divided by the number of words in the reference human transcription. The result: all five systems show substantial disparities, with an average error rate of 0.35 for Black speakers versus 0.19 for white speakers.

The authors attribute the gap to the acoustic models, that is, the part of the system that maps sounds to words: the gap remains just as large on a subset of identical phrases spoken by Black and white speakers. They recommend more diverse training data, including African American Vernacular English, and suggest that other varieties of English, including regional and non-native accents, could face similar difficulties. This is a hypothesis, not a measurement.

The scope must be stated exactly: American English, commercial systems as they existed at the time of the study, interviews recorded between 2004 and 2017. The study says nothing about the current versions of these systems or about French. What it does show is that a system can post good overall performance while serving some of its users less well.

The studies we found concern varieties other than the accents of mainland France. In 2023, Lucas Maison and Yannick Estève used recordings representing four French accents to fine-tune recognition models: by mixing several accents in training, they reduced errors by up to 25% in relative terms on African and Belgian accents compared with training on a single domain, while maintaining good performance on standard French. Their starting point is explicit: systems remain not very robust to variations in accent.

In Quebec, the CEREALES corpus, presented at the Interspeech 2025 conference, brings together more than 600 hours of spontaneous Quebec French speech recorded during a commission of inquiry, to make up for the shortage of accented data. Another team compared large multilingual models on Quebec conversations: according to its authors, the results published on well-known benchmarks such as FLEURS or Common Voice are poor predictors of the performance they observe on their corpus of conversations recorded during Quebec commissions of inquiry.

For the regional accents of mainland France, we had not found, at the time of writing, any comparable public measurement. So be wary of any figure put forward without a method.

A word misrecognised in the middle of a sentence is often recovered from context. A surname, a phone number, a street name or the name of a hamlet is much harder to guess: and that is precisely the information you will need to call back. The constraints specific to the phone line, already covered in our article on speech recognition, come on top of the variety of accents.

Three steps reduce the risk, whether the call is handled by a person or an automated agent. Rephrase the key information before hanging up: the name, the number, the address. Have proper names spelt out and numbers confirmed digit by digit. And when the conversation stalls, offer a hand-off rather than making the caller repeat themselves over and over: a complete message or a transfer to a human.

Then reread the transcripts of your calls to spot recurring errors, for example the name of a neighbouring town that is consistently mangled, and decide which instruction to apply: systematic rephrasing, a request to spell, a transfer.

Then test in your own conditions: ask friends or colleagues with different accents to call, from a mobile, in a noisy environment, using proper names from your area, then compare what they said with what was noted down.

Alyas answers based on the information you give it, states at the start of the call that it is an AI and that the call is recorded, transfers the call to your number if the caller asks, and places a transcript and a summary of each call in your Alyas dashboard. The 10-day free trial, with no credit card required, is there precisely so you can check how well it understands your callers.