Earnings call sentiment analysis: a valid score is not a verdict

Earnings call sentiment analysis measures the tone of what is said on a results call, usually sentence by sentence, as positive, negative or uncertain, using a finance-specific word list, a trained language model or, in some research, the audio of the speakers' voices. Peer-reviewed studies find that tone on calls carries information beyond the numbers, and that general-purpose methods misread financial language badly. Even a well-built score has a limit, though: it measures how a call sounded, not whether any part of your investment thesis changed. Botminds' Earnings Call Analysis, software that reads each earnings call against a buy-side analyst's thesis, uses line-by-line tone to show where a call turned cautious, then requires a cited line that meets a pillar's own rule before anything moves. This guide sets out what the research supports, where scores go wrong, and how to test one.

What is earnings call sentiment analysis?

It is the measurement of tone in a call's transcript or audio, turned into a score that can be compared across lines, speakers, calls and quarters. Four families of method are in use. Dictionary methods count words from a list of positive, negative or uncertain terms. Machine-learning classifiers are trained on labelled sentences. Transformer language models, such as FinBERT, read each word in context. And vocal analysis measures stress or emotion in the speakers' voices from the audio.

The output is usually a number per sentence, rolled up per speaker or per section, and compared with the previous call. That makes tone easy to chart and hard to interpret, because the same cautious sentence can mean a guidance cut or a lawyer's standard warning. Loughran and McDonald, who built the field's standard finance word lists, put it bluntly in their survey of the literature: "Relative to quantitative methods traditionally used in accounting and finance, textual analysis is substantially less precise" (Journal of Accounting Research, 2016).

Why do general sentiment dictionaries fail in finance?

Because ordinary accounting words carry a negative meaning in everyday English. Loughran and McDonald found that in 10-K filings from 1994 to 2008, "almost three-fourths of the words identified as negative by the widely used Harvard Dictionary are words typically not considered negative in financial contexts" (Journal of Finance, 2011). The paper's own title makes the point: "When Is a Liability Not a Liability?" In a balance sheet, almost always.

A call is worse ground than a 10-K for a general dictionary. Chief financial officers talk about costs, charges, taxes, depreciation and risks as a matter of routine, and a tone score that counts those words reports a gloomy call every quarter. A tool that cannot say which word list it uses, or that uses a general one, should be treated as a measure of vocabulary rather than of tone.

What is the Loughran-McDonald dictionary?

It is the set of finance-specific word lists Loughran and McDonald published in 2011: an alternative negative word list "along with five other word lists, that better reflect tone in financial text", which they linked to filing-date returns, trading volume, volatility, fraud, material weakness and unexpected earnings. Because it was built from financial documents rather than general English, it is the benchmark newer methods are measured against; FinBERT's authors, below, tested their model against it.

Its strength is transparency: every scored word can be shown. Its weakness is that it counts words without reading them. "Not a material decline" and "a material decline" contain the same words, and a list cannot tell a quoted risk factor from a live warning.

From word lists to language models

Language models read words in context, and on financial text they do measurably better. Huang, Wang and Yang built FinBERT, a model adapted to finance, and found it "substantially outperforms the Loughran and McDonald dictionary and other machine learning algorithms" in classifying the sentiment of sentences from analyst reports, especially those other methods mislabel as neutral. Applied to calls, they found "other approaches underestimate the textual informativeness of earnings conference calls by at least 18% compared to FinBERT" (Contemporary Accounting Research, 2023).

Large general models have been tested on financial news too, with a caution attached. Lopez-Lira and Tang report that GPT-4 "captures initial market responses, achieving approximately 90% portfolio-day hit rates for the non-tradable initial reaction", and that "strategy returns decline as LLM adoption rises" (arXiv 2304.07619, version of Oct. 28, 2025; not peer reviewed). Read together, the two findings say the reaction a model can see is the reaction everyone can see, and it is quickly priced.

Can tone predict stock returns?

Tone is associated with returns, but the association often reverses, and the direction depends on who is speaking and why. That is the finding to hold on to before buying a signal.

The managed-tone result is the one a buy-side analyst should remember on a call. Management knows its tone is being measured, and Huang, Teoh and Zhang found abnormal optimism was associated with just meeting thresholds, later restatements and equity issues: in their words, "consistent with managers using strategic tone management to mislead investors about firm fundamentals". A rising tone score can be the company's message rather than the business's.

What are vocal cues in earnings calls?

Vocal cues are measurable features of the speakers' voices, such as stress, emotion and hesitation, extracted from the call's audio rather than its transcript. Mayew and Venkatachalam analysed call audio with vocal emotion software and found that "positive and negative affects displayed by managers are informative about the firm's financial future. Analysts do not incorporate this information when forecasting near-term earnings" (Journal of Finance, 2012). When analysts changed recommendations, they reacted to positive affect but not to negative.

That is a real signal and a narrow one. It came from the Q&A, "when managers are scrutinized by analysts", and from the audio, which a transcript-based score never sees. It is also a measure of the speaker, not of a pillar: a nervous chief financial officer may be nervous about a margin, a currency or the question itself.

Can AI detect deception in earnings calls?

Somewhat better than chance, and no better than the accounts. Larcker and Zakolyukina labelled calls as deceptive using later restatements and found that models built on executives' word choices did "significantly better than a random guess by 6–16%", and "at least equivalent to models based on financial and accounting variables" (Journal of Accounting Research, 2012). Hobson, Mayew and Venkatachalam found vocal dissonance markers in chief executives' speech flagged irregularity restatements with accuracy "11% better than chance", similar to models based on accounting data alone (Journal of Accounting Research, 2012).

Those are useful screening results across thousands of firms. They are not grounds for calling one executive a liar on one call, and a research note that did so would put its author in a difficult position. Deception scores belong, if anywhere, in a risk screen across a portfolio, not in a thesis.

Valid is not accurate: why a sound score can still be wrong about your thesis

A sentiment score can be valid, measuring tone correctly by every test, and still be wrong about the thesis, because the thesis is not about tone. On the fictional Hoshida Robotics call that runs through Earnings Call Analysis, the tone turned cautious through the stretch where the chief financial officer cut margin guidance and reported China orders down 27%. A tone-led reading would mark the China pillar broken. But the decisive lines came later and sounded cautious too: the chief executive's expectation of a fourth-quarter recovery as large electric-vehicle projects restart, with two projects in final specification. Measured by tone, those lines add to the gloom. Measured against the pillar's break rule, China orders below −20% for two quarters or backlog under ¥400 billion, they are counter-evidence, and the pillar stopped at Weakened.

In Earnings Call Analysis, the tone ribbon shows where the call turned; the pillars show what moved

The test of a sentiment tool is therefore not "is the score right?" but "does each scored line connect to something I decide?". A score that cannot point to its lines, its word list or model, and the pillar it bears on is a mood ring. One that can is a map of the call, and a useful one.

How Earnings Call Analysis uses tone

The solution's listener agent scores the tone of each line as the call runs, in the language spoken, and draws it as a ribbon that marks where the call turned cautious and where each guidance statement sits. Tone is how the analyst finds the stretch worth replaying. It is not how a pillar moves. A pillar moves to Weakened or Broken only when tagged lines meet the pillar's rules, after the comparer has checked them against last quarter and the model and the skeptic has searched the same call for the strongest case against the move. Any line the listener scores under 80% confidence is marked unclear and never used until a person checks it; the threshold is the Director of Research's setting, from 60% to 95%.

Asking the agents for every line on one topic, with timestamps

That design follows the research rather than arguing with it. Tone is informative, especially in answers and in the voice, and the solution shows it. But a buy-side analyst is paid for a view on the business, and a thesis written with break rules is what turns a cautious call into a decision. The comparison of summaries with thesis testing goes through the same call another way, and the Earnings Call Analysis page shows the rest of the evening.

Questions people ask

How accurate is sentiment analysis on earnings calls?

It depends on the method and the text. A finance-adapted model like FinBERT substantially outperformed the Loughran-McDonald dictionary and standard machine-learning methods on analyst-report sentences, but even the authors of the standard word lists call textual analysis "substantially less precise" than quantitative methods. Accuracy at labelling sentences is also not the same as usefulness for a thesis.

Is sentiment analysis on earnings calls done on the transcript or the audio?

Mostly the transcript, because it is easier to obtain and process. Research on the audio finds information the words do not carry: managers' vocal affect predicted the firm's financial future in a 2012 study, and analysts did not incorporate the negative part of it.

Can I build earnings call sentiment analysis in Python?

Yes: researchers have built tone measures on the Loughran-McDonald word lists and on finance-adapted models such as FinBERT, both designed for financial text. The hard parts are not the code but the inputs and the test: clean speaker turns, separate script and Q&A, the right language, and a back-test on calls where you know the outcome.

Why do analysts and investors react differently to tone?

Partly because they are doing different jobs. Mayew and Venkatachalam found analysts incorporated managers' positive vocal affect into recommendation changes but not the negative, and did not use either in near-term earnings forecasts, even though both were informative about the firm's future.

Does a positive earnings call mean the stock will go up?

Not reliably. In Hong Kong, positive call tone was followed by higher returns, but research on US earnings releases found that abnormally positive tone predicted weaker future earnings and a delayed negative price reaction, consistent with managers using tone to shape perceptions.