Speech analytics software converts recorded calls into text and then searches, categorises and scores that text: which calls mention a fee, which customers sound dissatisfied, which bankers skipped a disclosure. For a bank, everything the software finds depends on that first step. A notice of error the recogniser mishears is a notice nobody flags, and in a peer-reviewed test of five commercial speech recognisers the average word error rate was 0.35 for Black speakers and 0.19 for white speakers. Before comparing dashboards, a bank should test transcripts on its own calls, broken down by speaker group, because a monitoring program that misses more for some customers than for others creates a fair-treatment problem of its own. Botminds' Call Compliance Monitoring transcribes every call with a Listening agent, and the same test applies to it.
The research figures on this page come from the study, linked where they appear. The call examples come from a fictional sample bank, Corvane National Bank, and its sample data.
Word error rate by speaker group: almost twice as high for Black speakers on every system tested
What is speech analytics software, and what does a bank use it for?
Speech analytics software is a pipeline: record the call, recognise the speech, separate the speakers, then search and classify the text, and in some products score the call. A bank uses it for quality assurance, for finding complaints and dissatisfaction, for checking required statements and disclosures, for spotting trends in why customers call, and increasingly for compliance monitoring on every call rather than a sample.
Regulators have noticed the last use, in one case approvingly. Reviewing how UK banks and building societies handled bereavement and power of attorney cases, the Financial Conduct Authority in 2025 recorded as good practice that "Some firms used artificial intelligence (AI), including speech analytics that – either in real-time or through an automatic review after the customer interaction - highlighted when a customer had said something that could indicate potential characteristics of vulnerability" (FCA multi-firm review, 2025). That is a UK finding under UK rules, and it shows the direction of travel: speech analytics treated as a control, which means it will be judged as one.
Does speech recognition work equally for all accents? Not in the published test
Not in the published evidence. A peer-reviewed test published in PNAS in 2020 ran five commercial recognisers on interviews with 42 white and 73 Black speakers and found that "all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers" (Koenecke et al., PNAS, 2020). The gap held on every system: "the error rates for black speakers are nearly twice as large in every case."
The detail matters for a bank deciding what to test. The gap was largest for Black men, 0.41 against 0.30 for Black women, while white men and women were closer, 0.21 and 0.17. Vocabulary was not the cause: 98 to 99% of the words spoken were in the systems' vocabularies for both groups. On 206 matched phrases with identical words, error rates were still about twice as high, which led the authors to trace the disparity to the acoustic models and conclude that "The likely cause of this shortcoming is insufficient audio data from black speakers when training the models."
The study has limits the authors state themselves. The white and Black speakers came from different regions, so some of the gap may be regional rather than ethnic, although error rates tracked how strongly speakers used features of African American Vernacular English. The interviews were recorded between 2004 and 2017, and the systems were tested before the paper was submitted in October 2019. Recognisers have improved since. What has not changed is the lesson: an average accuracy figure can hide a group for whom the system works far worse.
What is word error rate?
Word error rate is the standard measure of transcription accuracy: the number of words the recogniser substituted, deleted or inserted, divided by the number of words actually spoken. The PNAS study defines it as (S + D + I) ÷ N, "where S, D, and I denote the number of word substitutions, deletions, and insertions between the machine and ground-truth transcriptions, respectively, and N is the total number of words in the ground truth." A word error rate of 0.19 means roughly one word in five is wrong, missing or extra.
For compliance monitoring the rate is necessary but not enough, because errors are not equal. Take the sentence a customer at the sample bank said at 3:12 into a call: "There's a charge for $412.60 at Brightline Electronics on the 17th. I never made that. I don't even shop there." Twenty words. Two transcripts with similar error rates can carry very different consequences.
Two transcripts of one sentence: similar error rates, different consequences
| Transcript | Errors | Word error rate | Is the notice of error still there? |
|---|---|---|---|
| "…charge for 412 60 at Brightline Electronics on the 17th. I never made that…" | "$412.60" heard as two words: 1 substitution, 1 insertion | 2 ÷ 20 = 0.10 | Yes: the amount is recoverable from the card record, and "never made that" survives |
| "…charge for $412.60 at bright line electronics on the 17th. I ever made that…" | "Brightline" split: 1 substitution, 1 insertion; "never" heard as "ever": 1 substitution | 3 ÷ 20 = 0.15 | No: the one word that made it a dispute is gone |
How accurate is speech analytics? Every flag inherits the transcript
Speech analytics is as accurate as its transcripts for the calls that matter, and the published evidence says that accuracy varies most at the bottom: for some speakers a usable transcript is the exception. The PNAS study looked at the share of snippets bad enough to be useless: "If we assume that a WER >0.5 implies a transcript is unusable, then 23% of audio snippets of black speakers result in unusable transcripts, whereas only 1.6% of audio snippets of white speakers result in unusable transcripts."
Where the averages hide the problem: transcripts too poor to use
23%
of snippets of Black speakers with a word error rate above 0.5
1.6%
of snippets of white speakers above the same threshold
0.41
average word error rate for Black men across the five systems
98–99%
of spoken words were in the systems' vocabularies for both groups
Every downstream step inherits those errors. A rule that looks for a customer saying a charge is wrong cannot fire on a transcript that lost "never". A complaint classifier cannot see dissatisfaction in a sentence the recogniser garbled. And a reviewer who reads only the transcript has no way to know what was lost. At Corvane, the Listening agent separates speakers and transcribes every call, and the reviewer sees the transcript excerpt next to the waveform with a button to play the audio from the second the finding was pinned, so the words can be checked against the voice.
The transcript beside the audio, so a reviewer can hear what the text says
Why transcript errors become a fair-treatment risk in call monitoring
Transcript errors become a fair-treatment risk when they fall unevenly, because a monitoring program that misses more findings for one group of customers leaves that group's disputes and complaints unworked more often. The obligation does not change with the customer's voice. Under Regulation E a spoken notice of error starts the bank's clock whether or not any software heard it (12 CFR 1005.11), and the missed notice is the bank's problem either way.
The pattern would be hard to see from inside. A program that misses a third more notices for some speakers produces no error message; it produces fewer findings for those customers, which can look like better service. Only a test that knows the right answer, seeded calls or human reference transcripts, broken down by speaker group, can show it. That is why the order matters: test the transcript first, then the findings built on it.
How do you test speech analytics accuracy? A test on your own calls
Test speech analytics accuracy by comparing its transcripts with careful human transcripts of the same calls, word by word and group by group, and then by seeding calls with known findings and counting what it catches. The steps below are ours, adapted from the PNAS study's method to a bank's calls.
A transcript test for speech analytics, before any finding is trusted
- Choose calls across speaker groups
Groups your bank can define lawfully and that matter in your customer base: regions, age bands, gender, language preference.
- Match the groups
Similar call types and lengths in each group, as the study matched on age, gender and snippet length.
- Make human reference transcripts
Two transcribers per call, differences resolved, card data redacted before anyone outside the bank sees it.
- Compute word error rate per group
Substitutions, deletions and insertions over words spoken; report the distribution, not only the average.
- Count the critical words
Negations, amounts, dates, merchant names, "complaint", "cancel": the words a finding depends on.
- Seed findings and count catches
Notices of error and complaints in ordinary words, at least 60 per group you want to measure.
- Repeat after every model change
A new recogniser or language model can close one gap and open another.
- Set the threshold before you look
Decide what gap between groups fails the test, and who signs off if one is found.
The seeded part of the test is the same one a bank should run before buying any automated review; piloting on last month's calls sets out how many seeds to plant and why 60 caught without a miss supports a miss rate under 5%. Run it per group, not only overall, and calibrate the people who will judge the results first, as call center calibration describes.
Can speech analytics detect complaints?
Speech analytics can detect complaints if it looks for dissatisfaction as the bank's policy defines it rather than for the word "complaint", and only as reliably as the transcript lets it. The CFPB itself treats a complaint as a submission expressing dissatisfaction with, or suspicion of wrongful conduct by, a company, and a bank's own policy, such as the sample bank's, may count any expression of dissatisfaction. Customers describe the problem; they rarely name the category.
The CFPB's 2025 complaint report gives a sense of what that sounds like on checking accounts: consumers "encountered rude and dismissive customer service representatives", "experienced long hold times on the phone, dropped calls with no return call, and transfers to wrong departments" (CFPB Consumer Response Annual Report, 2025). None of those sentences contains the word "complaint". At Corvane, the Complaint agent applies the bank's complaint policy to every call and chat and checks the complaint system for a log; a person confirms before anything is logged. Complaint vs inquiry works through where that line falls.
Does speech analytics need a human review?
Yes, for anything that opens a case or reaches a customer. A transcript is a model's best guess at what was said, and for some speakers it is wrong often enough to change the meaning. A person who can hear the audio, see the excerpt and read the rule is the check on that guess, and the decision on what the bank owes the customer belongs to that person.
Human review also has to be designed so that it catches transcript errors rather than rubber-stamping them. At Corvane a second agent, on a different model family, re-reads each flag against the whole call before a person sees it, and removes flags with a written reason when the evidence does not support them. The removed flags stay on the call, and a QA analyst can restore any of them.
A flag the verifier removed, with the transcript and the reason beside it
What to look for in speech analytics software for a bank
The best speech analytics software for a bank is the one whose transcripts have been tested on the bank's own customers, group by group, and whose findings can be traced to the audio and the rule. Features that do not change what is found, such as dashboards and sentiment scores, come second.
General speech analytics, and what compliance monitoring at a bank needs from it
General speech analytics: One word error rate on a benchmark.
What a bank's compliance monitoring needs: Word error rate and critical-word recall on your calls, by speaker group.
General speech analytics: Keywords, topics, sentiment.
What a bank's compliance monitoring needs: Findings against rules: notices of error, complaints, misstatements, required statements.
General speech analytics: The transcript.
What a bank's compliance monitoring needs: The transcript plus the fee schedule, hold notices and account and case records.
General speech analytics: Often redacted in text only.
What a bank's compliance monitoring needs: Redacted from the transcript and the audio reviewers hear, with the original untouched.
General speech analytics: Scores are the output.
What a bank's compliance monitoring needs: A person confirms each finding; nothing opens a case before that.
General speech analytics: Often a separate product.
What a bank's compliance monitoring needs: The same rules on chats, which need no speech recognition but carry the same obligations.
Where Call Compliance Monitoring fits: it is compliance monitoring for bank calls and chats, built on the bank's own rule library and records, with a person deciding every finding. It is not a general analytics suite for marketing insight or agent sentiment scoring. Its Listening agent should be held to the same test as any other recogniser: run on your calls, by speaker group, before go-live and after every model change. The agents and their limits are on the Call Compliance Monitoring page.
Questions people ask
What word error rate is good enough for compliance monitoring?
There is no regulatory standard. The PNAS study treated a word error rate above 0.5 as unusable, but a transcript can pass that bar and still lose the one word that made a sentence a dispute, so measure recall on critical words and on seeded findings, by speaker group, alongside the overall rate.
Does speech analytics work on chats?
Chats need no speech recognition, so they avoid transcript errors entirely, but they carry the same obligations: a chat message can be a notice of error or a complaint. A monitoring program should apply the same rules to both channels and report them separately.
Why do speech recognition systems make more errors for some speakers?
In the PNAS study the cause was the acoustic models, not vocabulary or grammar: error rates were about twice as high even on identical phrases, and the authors pointed to "insufficient audio data from black speakers when training the models." More diverse training data is their proposed fix; testing is the buyer's.
Is speech analytics the same as call recording?
No. Call recording captures the audio; speech analytics turns it into text and analyses it. Whether a call may be recorded at all is set by federal and state consent laws, and the analysis inherits whatever consent the recording had.
Should a reviewer read the transcript or listen to the call?
Read the excerpt, and listen when the finding depends on a single word, a number or a name. At the sample bank the reviewer can play the audio from the pinned second, which takes seconds and catches the transcript errors the PNAS study shows are common for some speakers.