Earnings call summary AI vs reading vs thesis testing, on one call

Earnings call summary AI condenses a results call into a few bullet points or paragraphs of what management said, usually within minutes of the transcript being available, and it is good at that: summarisation and extraction are the uses financial firms report most. But a summary answers one question, "what was said?", and a buy-side analyst needs two more: "what changed since last quarter?" and "what does it do to my thesis?". Botminds' Earnings Call Analysis is software that reads each call against an analyst's own investment thesis, pillar by pillar, and drafts a cited note the analyst decides and signs; it produces a summary along the way but is built for the third question. This guide compares the three approaches on one call, sets out what the research says about summary accuracy, and gives a test any team can run on its own past calls.

The worked call is a fictional one: Hoshida Robotics' second-quarter results call, held in Japanese on October 29, 2026, as it runs through the solution for the fictional Quillfield Capital. All figures about it are sample data.

What earnings call summary AI does well

It is fast, it scales across a coverage list, and it is the use case firms trust most. FINRA's 2026 oversight report ranks summarisation and information extraction as the most common generative AI use among its member firms (FINRA, December 9, 2025); what that means for the person who checks the output is the subject of AI earnings call analysis and the maker-checker split. For an analyst with several calls on the same afternoon, a good summary is a triage tool: it says which call to open first and which can wait until morning.

It is also, by design, neutral. A summary reports management's words without weighing them, which is what makes it safe to share widely and what limits it. The moment a summary starts to say "this is bad for the stock", it has become an analysis, and an analysis needs a basis, a source for each claim and a person who stands behind it.

Can ChatGPT summarize an earnings call?

Yes, if you give it the transcript, and with three cautions. The first is the data: a firm's own notes or models pasted into a public chatbot leave the firm's control, and ESMA has said firms remain responsible for staff use of third-party tools such as ChatGPT, whether or not senior management knew of it (ESMA, May 30, 2024). The second is length: a full call transcript is long, and a model that silently truncates it summarises the part it kept. The third is citations: a general chatbot does not, by default, tie each sentence to a timestamped line, so checking it means reading the transcript anyway.

None of that makes a chatbot useless for a first look. It makes the output a draft that nobody has checked, which is a fine thing to read and a poor thing to forward.

Are AI earnings call summaries accurate?

On wording, often; on numbers and coverage, less reliably, and the research shows where to look. ECTSum, an academic benchmark for summarising calls published at EMNLP 2022, pairs 2,425 call transcripts with expert-written bullet summaries of about 50 words, from documents of about 2,900 words. Its authors write that "quantity hallucination is a known problem in abstractive summaries", and they measure the fraction of numbers in a generated summary that actually appear in the source (Mukherjee et al., EMNLP 2022; doi).

The same benchmark shows the coverage problem. To build it, the authors state: "We only consider the unstructured text corresponding to the Prepared Remarks section to form the source documents", and calls that began directly with questions were excluded. A benchmark built to train and score call summarisers leaves out the Q&A, which is where research finds most of a call's new information. Matsumoto, Pronk and Roelofsen found that "discussion periods are relatively more informative than presentation periods", in more than 10,000 transcripts (The Accounting Review, 2011).

On questions about filings rather than calls, the evidence is blunter. In the FinanceBench test of 10,231 questions, GPT-4-Turbo used with retrieval got 81% of a 150-case sample wrong or gave no answer (Islam et al., arXiv 2311.11944, November 2023; not peer reviewed); what the benchmark measures, and what it leaves out, is set out in FinanceBench and the defensible answer. Models have improved since, and the test is a minimum bar rather than a ceiling. The lesson that survives is about checking: a summary is accurate when each figure can be traced to its line, not when it reads well.

Five ways a call summary goes wrong

Most summary errors fall into five patterns, and each can be checked for. Truncation: a long transcript cut to fit a model's input, so the summary describes the part it kept. Numbers: a figure rounded, attributed to the wrong period or simply invented, the "quantity hallucination" the ECTSum authors measure. Script bias: the prepared remarks are orderly and quotable, the Q&A is not, so summaries lean on the script. Translation: a summary of an English transcript of a Japanese call inherits every error in the translation, and a figure misheard once is repeated with confidence.

The fifth is the hardest to see: what was not said. Gow, Larcker and Zakolyukina found that about 11% of analyst questions draw non-answers, and that questions "with a negative tone, greater uncertainty, greater complexity, or requests for greater detail are more likely to trigger non-answers"; performance questions drew more non-answers when the news was unfavourable (Journal of Accounting Research, 2021). A summary reports the answers it was given. It has no line for the question that was dodged, which is often the most useful thing an analyst writes down on a call.

What is the difference between a summary and an analysis?

A summary restates; an analysis judges, and a judgement needs a basis and an owner. CFA Institute Standard V(B) requires members to keep fact and opinion visibly apart when they present analysis and recommendations (CFA Institute Standards of Practice, V(B), updated January 2024), the same rule that shapes a thesis you can test every quarter. A summary is all fact, if it is accurate. An analysis mixes the two, which is why it must label which is which.

A thesis test is a structured analysis. It starts from the analyst's own claims, each with the evidence that would confirm or break it, and asks of every line on the call whether it meets one of those rules. The output names a pillar, a direction, the lines behind it, the counter-evidence weighed and the person who decided. That is what lets a portfolio manager act on it, and what lets a reviewer, or an examiner years later, see how the view was formed. How to write an investment thesis you can test every quarter covers the pillars.

Asking the agents in Earnings Call Analysis about one topic: the lines, not a paragraph

What do analysts use to analyze earnings calls?

Five inputs, more or less, whatever the tool: the call itself, live or as a transcript; the results release and last quarter's guidance; consensus and the analyst's own model; the written thesis; and the analyst's notes from the call. Summary tools work on the first. A careful read uses all five but keeps the connections in the analyst's head. A thesis test makes the connections explicit, so each claim in the note points back to a line, a figure or a model cell.

The order of work matters as much as the inputs. A summary-first workflow reads the call, then decides what matters. A thesis-first workflow decides in advance what would matter, then reads the call for it. The second is faster on a crowded evening and harder to fool, because the analyst is not deciding what is important while the company is telling them.

How do I summarize an earnings call?

If you write the summary yourself, five lines are enough, in this order: the quarter against consensus and your model; each guidance line that changed, with old and new; what the Q&A added that the script did not; what management would not answer; and which pillar of your thesis each of those bears on. Cite the minute and second of every line you quote. That last line is the one most summaries leave out, and it is the one that turns a summary into the start of a note.

How to test an earnings call summary AI on your own calls

Run any tool on ten past calls where you already know what happened next, and score it on what you need rather than on how it reads. The checklist below can be run in an afternoon and works the same for a summariser, a general chatbot or Earnings Call Analysis.

Where each approach fits

All three have a place, and a research team uses them for different readers. Summaries suit triage and broad distribution; a careful read suits the names that matter most; a thesis test suits every name the team holds a view on, because it is the only one that turns a call into a recorded decision.

What the portfolio manager sees: signed notes, with each pillar's state

The practical answer for most teams is not to choose one. Use a summary to decide what to read first, and a thesis test for every name you hold a view on, with the summary as a by-product. Why the reader is never the checker explains the checks that sit behind the test, and earnings call sentiment analysis covers the tone layer. The full evening is on the Earnings Call Analysis page.

Questions people ask

What is the best AI for earnings call summaries?

The one that passes a test on your own past calls: every figure traceable to the transcript, the Q&A covered, each claim linked to a line you can play, and calls in other languages handled with figures checked in both. Model names matter less than whether the output can be checked in minutes.

Are AI earnings call summaries good enough to send to portfolio managers?

As a first look, perhaps; as research, no, because nobody stands behind them. CFA Institute Standard V(B) asks analysts to distinguish fact from opinion, and a portfolio manager acting on a call needs the opinion, with its basis, from the analyst who owns the name.

Why do AI summaries miss the Q&A?

Partly length, because the Q&A is the longer part of most calls, and partly design. The ECTSum benchmark, a published dataset for call summarisation research, builds its source documents from prepared remarks only, while peer-reviewed studies find the discussion is the more informative part.

How long does an AI summary of an earnings call take?

Minutes, once a transcript exists. The bigger delay is the transcript itself for calls in other languages; in the sample above the vendor's English transcript arrived 43 minutes after the call ended, while the solution had read the call live from the firm's own recording.

Can an AI summary replace listening to the call?

Not for names you hold a view on. Research finds information in managers' voices and in which questions go unanswered, and a written summary carries neither. For names you only monitor, a cited summary is a reasonable substitute.