AI commentary generator: why the writer must never be the checker

An AI commentary generator for fund commentary should be judged less on how well it writes than on how it proves what it wrote. Any capable language model can turn an attribution table into fluent sentences; the risk is a fluent sentence with the wrong figure, and research on large language models finds they struggle to correct their own reasoning without external feedback, sometimes getting worse when they try (Huang and colleagues, arXiv preprint, ICLR 2024 paper). So the generator worth buying separates the writer from the checker: one agent drafts, a second with different instructions re-derives every figure from the source, and a person decides whenever they disagree. That is how Botminds' Portfolio Commentary Generation is built, and this guide sets out the design, the evidence behind it and the questions to ask any vendor, including us.

The figures that show the design at work come from a fictional manager, Quillfield Capital, whose third-quarter 2026 cycle of 214 strategies was run through the solution on sample data. They are not a measured result at any firm.

What is an AI commentary generator, and what should one do for fund commentary?

An AI commentary generator turns data into narrative: sports play-by-play, market summaries, business-intelligence chart notes, or the performance commentary an asset manager publishes every month or quarter. For fund commentary, the job has four parts, and generation is only the first. The tool must draft from the right data, check every figure and claim against that data, route anything it cannot prove to a person, and keep a record of what it did.

The second and third parts are where tools differ. A generator that drafts well and checks nothing moves the work to the reviewer, who now reads fluent text with no idea which sentences were derived and which were guessed. Commentary is dense with figures (the sample's ten-sentence Global Core Equity draft carries sixteen), and each must match its source in value, unit, sign, period and type. One figure's journey through nine versions shows how many places it can go wrong.

Can an LLM check its own work?

Not reliably, on the evidence so far. A 2024 conference paper examined "intrinsic self-correction", where a model reviews its own answer without outside feedback, and found that on reasoning tasks models rarely fixed their own mistakes without feedback from outside, and sometimes did worse after trying (Huang and colleagues, Large Language Models Cannot Self-Correct Reasoning Yet). The paper is about reasoning benchmarks, not fund commentary, but the mechanism carries over: a model asked to check its own draft brings the same assumptions that produced the error.

What works is external feedback, and in commentary the external feedback is the source file. A checker that re-derives a figure from the attribution cell is not asking whether the sentence seems right; it is asking whether the cell says what the sentence says. That is why the checker in a maker-checker design must have its own instructions and must not see the drafter's reasoning: if it inherits the reasoning, it inherits the error.

"These records should encompass aspects of AI deployment, including the decision-making processes, data sources used, algorithms implemented, and any modifications made over time."

ESMA, public statement on AI and investment services, 30 May 2024, ¶24

Regulators have arrived at the same design from the other end. ESMA expects firms using AI in client-facing information to put controls in place before the information is sent, to check accuracy afterwards as well, and to keep a record of how the AI was put to work, quoted above (ESMA public statement, May 2024). A drafting agent that cites its sources and a checking agent that re-derives each figure from them is what those expectations look like in a commentary process.

What the checker caught in a 214-strategy cycle

In the sample cycle, the Fact-check agent checked 18,940 sentences and corrected 286 of them. The two agents settled 241 corrections between themselves; 45 went to writers with both positions shown. Every one of the 286 is visible to the writer of that commentary.

Two of the corrections, both in the Global Core Equity draft, show what kind of errors a checker finds. The draft said the yen rose 6.1% against the dollar; the closing rates the checker pulled, 158.40 and 152.15, give 4.1%. And it credited the Financials overweight with 12 basis points, which is the sector's total effect; the allocation effect alone was 9. Both sentences were fluent, both used real numbers, and both would have passed a reader who checked only that the figures looked plausible.

How the agents wrote it: every step, every source, and the two sentences sent back

Why two agents, and not one model asked twice?

Asking the same model to check its draft is self-correction by another name, and the research above says that is where models struggle. Independence has to be designed in, in three ways. The checker gets different instructions: it is told to re-derive, not to review. It gets no access to the drafter's reasoning, so it cannot be talked into the drafter's assumption. And it checks against sources, not against the draft: the attribution cell, the returns file, the closing rates, the issuer's release.

Agreement then means something. When two agents with different instructions and different paths reach the same figure from the same cell, both being wrong in the same way is much less likely than one being wrong, as long as their errors are not correlated, which is what the separate instructions and sources are for. When they disagree, the disagreement is information, and it should go to a person with both positions rather than be settled by whichever agent speaks last. The sample settings cap the loop at two rounds, on the view that a further round between the same two agents adds repetition rather than new evidence.

The threshold for applying a correction without a person is a governance choice. The sample manager accepts an agent correction automatically only at 97% confidence or above and only when both agents agree, and the writer still sees every one. A firm that wants more eyes can lower the bar for escalation; a firm that finds its writers rubber-stamping trivial corrections can raise it. Either way the setting is versioned and approved by a second administrator, so the audit trail shows what the threshold was on the day a commentary went out.

Test it before you buy: seed the errors

The fairest test of any AI commentary generator is one where you know the answers. Take a commentary you have already published, with the attribution, returns and notes it was written from, and plant errors of known types in a copy: a basis-point figure written as percent, a total effect labelled as allocation, a negative effect described as a contribution, a gross return set against a net index, two transposed digits in a translated version, a forecast in the PM's outlook, and a sizeable detractor left out.

Then run the generator's checker on the seeded copy and count what it catches, what it misses and what it flags that was not wrong. A checker that catches the unit and digit errors but not the mislabelled effect is checking that numbers appear in the source, not that they mean what the sentence says. One that misses the omitted detractor is not checking balance at all. And one that flags many correct sentences will train your writers to click through its warnings, which is worse than no checker.

Ask the vendor to run the same test live, on your files, and to show the evidence for each catch. The seeded test takes an afternoon to prepare and tells you more than a month of watching polished drafts.

Is AI-generated commentary compliant?

It can be, on the same terms as any other commentary, because the rules do not change with the author. FINRA's 2024 notice on generative AI says plainly that "The rules apply when member firms use AI, including Gen AI or similar technologies" (FINRA Regulatory Notice 24-09), and the content standards of its communications rule apply to the text whether a person or a tool produced it. The SEC's Division of Examinations names "generative SI tools" used for drafting and client communications among the technologies it is examining, in its own term for them, "super intelligence (SI)" (SEC Division of Examinations, FY2027 priorities).

Three obligations bite hardest on an AI commentary generator. Where commentary is used as an advertisement, an adviser may not include a material statement of fact it cannot substantiate on demand (17 CFR 275.206(4)-1(a)(2)), so a generator must leave a source behind every figure. A broker-dealer's supervisory system must cover how it uses such tools. And a generator that sends client account data to an outside service brings that service into the firm's data protection programme: under the amended Regulation S-P, a covered firm's policies must require its service providers to tell it of a breach within 72 hours of finding it (17 CFR 248.30(a)(5)).

Free AI commentary generators: what they can and cannot do

Most tools that come up for "AI commentary generator free" are built for sports, video or social posts, and the general-purpose chat assistants are the realistic free option for a fund. They can draft a readable paragraph from an attribution table pasted into them. They cannot read your performance system, cannot tell whether the table was final, cannot check their own figures against the source, and leave no record a compliance officer can use.

How do you check AI-written commentary? Ten questions for a demonstration

The demonstration that tells you most is one run on a quarter you have already published, with its real attribution and the commentary that actually went out. Put the questions below to every vendor, and ask to see each answer on screen rather than hear it described.

Where Portfolio Commentary Generation fits, and where it does not

It is built for a manager that publishes periodic commentary for many strategies from a performance system that produces signed attribution, and that cares more about every figure being right than about the first draft being fast. In the sample cycle the Drafting agent wrote 207 of 214 drafts within an hour of each strategy's attribution being signed final, but the reason to buy is the 286 corrections, not the speed.

Thresholds and writing rules are settings, approved by a second administrator

It does not fit everywhere, and a fair evaluation should say where. It does not calculate attribution; it reads the attribution your performance team signs, and checking the attribution itself is a different solution's job. It is not built for intraday market notes or social posts, where there is no signed source to tie to. It does not translate; it checks the translation vendor's versions against the master, in their own language. And it never clears a compliance flag or publishes on its own: if the goal is to remove people from approval, this is the wrong tool. To see the maker-checker loop on a quarter you have already published, start from Portfolio Commentary Generation; the four numbers that prove it pays sets out how to measure the result.

Questions people ask

Is there a free AI commentary generator for fund commentary?

General-purpose chat assistants can draft fund commentary from a pasted attribution table at no cost, and some are free to use. None of them checks figures against your performance system, waits for final numbers or keeps a record your compliance team can rely on, so the checking cost moves to the reviewer. For published commentary, the free part is the smallest part of the work.

Can ChatGPT write fund commentary?

It can write a fluent draft, and a randomised experiment found ChatGPT cut professionals' time on writing tasks by 40%. It cannot know whether the numbers it was given are final, and research on self-correction finds models struggle to fix their own reasoning without outside feedback. Used for fund commentary, it needs a separate check against the source for every figure.

Do clients have to be told that AI drafted the commentary?

The US rules cited here hold the firm to the same content standards whoever drafts, and do not themselves require a label on AI-drafted text. ESMA expects firms to tell clients when they interact with AI tools such as chatbots, and to present any information about their use of AI in a clear, fair and not misleading way. A firm's own policy may go further.

How accurate is AI-written commentary?

No regulator or independent study has measured it for fund commentary. Research on long-form AI text found one widely used chat model's biographies supported on only 58% of their atomic facts, which is why figure-by-figure checking against a source matters more than fluency. The right measure for your firm is the correction rate on a quarter you have already published.

What does an AI commentary generator cost?

Vendors of purpose-built commentary systems rarely publish prices, and prices depend on strategies, versions and languages. The better question is what it saves and what it prevents: writer hours per strategy, days from final attribution to approved text, and corrections caught before the PM reads the draft.