Evaluate · the seeded-error test

Fund factsheet example: plant 12 errors, count what review catches

Take one fund factsheet example, plant in it the twelve errors European supervisors actually report, send it through your normal review and count what comes back. The catch rate by error type, not the number of reviews, is the control. The same test is how to judge Factsheet Automation or any tool that claims to check factsheets.

Updated · 14 min read

The most useful fund factsheet example is one you have broken on purpose. Take a factsheet you released last month, make a copy, plant the errors that European supervisors report most often (a stale date, the wrong benchmark, cumulative returns without calendar years, the management fee where the KID shows ongoing charges, a risk class that no longer matches the KID, a missing language statement) and send it through your normal review without telling the reviewers. Then count what they catch, error by error. That catch rate is the measure of your control; the fact that every factsheet was "reviewed" is not. Botminds' Factsheet Automation was built to raise that rate on every factsheet rather than a sample, with agents checking each field against its source and holding what they cannot prove for a compliance reviewer, and the same seeded test is the fairest way to judge it.

Why a test is needed at all comes down to arithmetic. A reviewer who opens a random tenth of a wave sees an error only if it sits in one of the factsheets opened. An error confined to a single factsheet is opened about one time in ten; an error copied across a whole fund is almost always opened. Review catches template errors and misses single-factsheet errors, and supervisors' findings are full of the second kind.

How often a 10% random sample opens a factsheet carrying the error

Error on 1 factsheet (one class, one language)
10%
On 3 factsheets (one class in three languages)
28%
On 8 factsheets (a restated benchmark, 4 classes × 2 languages)
58%
On 19 factsheets (a top-ten table after a stock split)
87%
On 36 factsheets (a template error across one fund)
98%
Our arithmetic: the chance that a random sample of 42 out of 415 factsheets includes at least one of the affected factsheets (hypergeometric). Opening it is not the same as noticing the error, so real catch rates are lower. The wave size is from Factsheet Automation's fictional sample cycle.

Which errors do supervisors actually find in factsheets?

They find consistency errors far more often than drafting errors: a factsheet that disagrees with the fund's KID or prospectus, or shows performance on the wrong basis. ESMA's report of 6 January 2026 to the EU institutions collects the breaches national authorities found after reviewing marketing communications between February 2023 and January 2025 (ESMA34-1921782652-2033). Its list of the most frequent ones includes "Presentation of past performance (IT, LU)", "Inconsistent information across documentation (e.g. across marketing material and fund documentation) (FR, IT, LU, NO, PL)" and communications "not identifiable as such (BG, HR, PT)".

The UCITS examples are specific enough to plant. Italy's authority cited a communication that "displayed cumulative past performance, without displaying the performance of the fund for each year of the considered period"; Luxembourg's, one that disclosed the current year's performance "at the end of the last month whereas it should have been disclosed at the end of the most recent quarter"; Poland's, a sub-fund fact sheet whose benchmark was "EONIA+2,5%, whereas the KIID stated that the applicable reference benchmark was €STR" (ESMA report, paragraph 72). Under Article 4 of the cross-border distribution regulation, Austria, Bulgaria and Luxembourg also reported UCITS communications that never said in which language the prospectus and key investor information could be had (paragraph 74).

The fund factsheet example: one class, one month

The example we use is fictional: Brandreth Global Dividend Fund, a Luxembourg UCITS, share class B EUR Acc, registered for sale in Luxembourg, Germany, Austria and Italy, factsheet as at 31 August 2026 in English, German and Italian. Its clean version is the one released last month, already reviewed and approved. The planted version carries twelve errors, each one reported by a supervisor or seen in practice as a reason to hold a factsheet, and each placed where it would hide in a real document: in a cell, a footnote or a translation, not in a headline.

Twelve errors to plant in a fund factsheet example

  1. Stale as-at date

    The performance table shows July month-end under an August heading.

  2. Wrong benchmark

    The factsheet names last year’s index; the current KID names a new one (ESMA paragraph 43).

  3. Cumulative only

    Five-year cumulative return shown without each calendar year (paragraph 46).

  4. Current year to month-end

    The year-to-date line uses August month-end where the quarter-end is required (paragraph 44).

  5. Management fee as the cost

    0.75% management fee shown; the KID shows ongoing charges of 0.92% (paragraph 21).

  6. Superseded risk class

    Risk indicator 5 from the old KID; the KID revised on the 12th says 4 (paragraph 36).

  7. Risks in small type

    Risk text set two sizes smaller than the rewards, at the foot of page 2 (paragraph 13).

  8. Missing label

    The German version lacks the marketing-communication label (paragraph 6).

  9. No language statement

    The Italian version omits where and in which language to get the prospectus and KID (Regulation 2019/1156, Art. 4(2)).

  10. Warning after the figures

    The past performance statement follows the table instead of preceding it (paragraph 47).

  11. Top ten before a split

    Weights from before a stock split; the table sums to 103.4%.

  12. A translation that changes the product

    The Italian objective calls the fund passively managed; the English, correctly, says actively managed (paragraph 28).

A fictional factsheet (Brandreth Global Dividend Fund, B EUR Acc) and our list. Paragraph numbers refer to ESMA's guidelines on marketing communications, ESMA34-45-1272.Source: ESMA guidelines; ESMA report, 6 January 2026

How to check a factsheet before release: where each error hides

Each planted error is found by comparing the factsheet with a source, never by reading the factsheet alone. That is the point of the test and the weakness of most reviews: a reviewer reading a well-formed factsheet sees a well-formed factsheet. The table sets out, for each error, the source that disproves it and the check that would catch it every time.

Each planted error, the source that disproves it, and the check that catches it

Planted errorDisproved byThe checkRule
Stale as-at dateSigned returns fileAs-of date of every figure equals the month-endESMA para. 23; Rule 482(g)
Wrong benchmarkCurrent KID and prospectusBenchmark name and ID match the legal documentsESMA para. 43
Cumulative onlyThe templateCalendar-year box present whenever a cumulative figure isESMA para. 46
Current year to month-endThe template and calendarYear-to-date line dated to the last quarter-end (EU)ESMA para. 44
Management fee as costCurrent KIDCost figure equals the KID’s, same basisESMA para. 21
Superseded risk classKID version historyCurrent approved KID only; revised KID holds the factsheetESMA para. 36
Risks in small typeThe rendered PDFRisk text size at least the body sizeESMA para. 13
Missing labelClause library, GermanLabel present in every languageESMA para. 6
No language statementRegistration list and clause libraryArticle 4 legends present for each marketReg. 2019/1156 Art. 4(2)
Warning after figuresThe rendered PDFStatement precedes the tableESMA para. 47
Top ten before a splitPortfolio accounting, corporate actionsWeights sum; holdings re-pulled after a posted actionESMA para. 23
Passive for active in ItalianEnglish master and prospectusMeaning of locked terms equal in each languageESMA paras. 20 and 28
Our mapping. A reviewer without the source beside the factsheet can find only the layout errors (risks in small type, warning placement), and only on the factsheets opened.

Seeding a US fund factsheet: five more errors

A US mutual fund or ETF factsheet needs a different set, because the errors that matter come from Rule 482 rather than ESMA's guidelines. Five plant well. Move the one-, five- and ten-year average annual returns back a quarter, so they no longer run to the last calendar quarter-end before the factsheet was submitted for publication (Rule 482(d)(3)). Push the past performance legend into a footnote, where Rule 482(b)(5) says it may not go in print. Show only the expense ratio net of a fee waiver, without "the total annual expenses without any fee waiver or expense reimbursement arrangement" (482(i)(1)). Drop the waiver's "expected termination date" (482(i)(2)). And leave out the statement that a sales load is not reflected in the returns, which 482(b)(3)(ii) requires where it is not.

These five share a property with the European twelve: each is invisible unless the reviewer has the rule and the source beside the page. A reviewer reading for sense will pass all of them on a well-designed factsheet.

How to run the test: blind, timed, on the real review path

The test is only worth running if it measures the review you actually do. Four conditions make it honest. First, blind: the reviewers do not know which factsheets are seeded, and ideally do not know a test is running that cycle; the head of marketing compliance and internal audit agree it in writing beforehand. Second, on the real path: the seeded factsheets enter the normal queue on the normal day, under the normal deadline, not in a quiet week. Third, spread: put the twelve errors in different factsheets, some in a class that sells in one market and one language, some across a whole fund, so the sample arithmetic plays out. Fourth, stopped at release: nothing seeded may reach a platform, so production holds the seeded copies back after review.

Record three things for each error: whether it was caught, by whom and at which step (production, four-eyes, compliance), and the time from submission to catch. Keep the evidence. A US adviser that shows performance must already keep the working papers that show how it was calculated (17 CFR 275.204-2(a)(16)); the test file shows that the review behind those figures works.

Counting the catch rate: by error type, not in total

A single number, "9 of 12 caught", hides what the test is for. Report the result by error type and by how many factsheets carried it, because each type points to a different fix. Errors that need a source to see (benchmark, cost, risk class, translation meaning) are caught only if the reviewer has the source open; if they are missed, the fix is to bring the comparison to the reviewer or to automate it. Errors that need the rendered page (font size, warning placement) are caught only on the factsheets opened; if they are missed, the fix is a layout check on every factsheet. Errors that depend on timing (stale date, current year, split) are caught only if someone knows the calendar; if they are missed, the fix is a freshness rule.

Repeat the test each quarter with different errors and keep the series. No regulator or independent study publishes a benchmark catch rate for factsheet review, so the series is your benchmark: a rate that rises after a process change is evidence the change worked, and a rate that falls is an early warning before a supervisor's sample finds the same thing.

The information was unbalanced in favour of highlighting possible returns, while under-communicating the significance of costs of the fund.

ESMA, report to the EU institutions, 6 January 2026, paragraph 72 (Norway)

How do regulators review fund marketing?

Mostly after the fact, on a sample or when something prompts them. ESMA's report records that the majority of national authorities "do not carry out systematic ex-ante verification of marketing material", while an increasing number supervise "on a case-by-case basis pursuant to a risk-based approach" (ESMA report, paragraphs 37 and 61). Where they do review, the requests for amendments add up. Between February 2023 and January 2025, authorities in 18 countries made requests for amendments to UCITS and AIF marketing communications after ex-post review; for UCITS alone the largest counts were these.

Requests to amend UCITS marketing communications after ex-post review, February 2023 to January 2025

Denmark
357
Italy
70
Bulgaria
32
Luxembourg
17
Poland
11
Czechia
10
Numbers reported by national authorities to ESMA; authorities differ in how they review and count, so the figures show where review happens, not where marketing is worst.Source: ESMA report to the EU institutions, 6 January 2026, paragraph 67

In the US, a FINRA member that distributes funds must file retail communications promoting a specific registered fund within 10 business days of first use, and the SEC's examiners review advertising as part of their examinations. The seeded test does not replace any of that. It tells you, before an examiner does, what your own review lets through.

Factsheet compliance review: making the result a control

A compliance review becomes a control when it is designed around the errors that matter and measured against them. The seeded test supplies both: a list of errors drawn from supervisors' findings and a catch rate for each. Three design changes usually follow from a first run. Move source comparisons out of the reviewer's head and onto the page, so that the benchmark, cost and risk class on each factsheet sit beside the KID values. Replace a purely random sample with a risk-based one that always includes every changed factsheet, every first-time class or market and every factsheet a check has flagged, plus a random share. And run the mechanical checks on every factsheet, not on the sample, because the sample arithmetic above cannot be beaten by reading more carefully.

The same logic applies to software. A tool that checks factsheets should be tested exactly as a reviewer is: seeded errors, blind, by type. ESMA's statement on AI in investment services asks firms that use it to put their AI tools through strict quality assurance, testing both the algorithms and the accuracy of what they produce (ESMA, 30 May 2024, paragraph 21). For a full trial on your own released factsheets, see the agentic AI proof of concept on last month's factsheets.

Running the same test on Factsheet Automation

Factsheet Automation is designed so that each planted error meets a check on every factsheet. Its seven agents run 41 field checks on each one, each tied to a source and an as-of date: the fund name against the prospectus, the NAV against the administrator, ongoing charges and the risk indicator against the current KID, every date against the month-end, the legends against the live registration list, every language against the English master. Box overflow is a failed check. A factsheet whose KID changed after the 15th of the month is held. A figure the agents cannot prove becomes a hold with an owner and a clock rather than a release.

The reviewer's sample is risk-based by design. In the solution's fictional sample cycle, a wave of 415 factsheets produced a release sample of 49: all 22 fixes and holds, all 6 first-time classes, and 21 at random (5%). Every fix one agent made had a second look from another agent before it could be released. Over the sample firm's last three cycles, people confirmed 241 of 250 flags the agents raised. Those are fictional figures; your own seeded test is the evidence that matters, and the reasons for holds in the sample cycle show what the checks are tuned to find.

Why Factsheet Automation held factsheets in one sample cycle

Holdings after a corporate action
19
Superseded or stale figure
10
Country legend missing or wrong
4
Name differs from legal document
4
KID figure changed
3
Translation meaning
2
Fictional sample data from the solution's September 2026 cycle report: 1,226 factsheets, 50,266 field checks. Each hold is a factsheet that would otherwise have gone out wrong or late.

A wave in Factsheet Automation: held factsheets first, the sample marked

The factsheet contact sheet for wave 2 of a fictional September 2026 cycle: 415 factsheets as thumbnails, grouped by fund, filterable by market, language and fund. Three held thumbnails sit first, the A CHF Hedged Acc class in Switzerland in German, French and Italian, with amber badges. Counters read 412 ready, 3 held, 49 in sample, 19 re-pulled, and 17,015 field checks. The agents' conversation runs down the right with the one decision that needs the reviewer.
Fictional sample data. Each thumbnail is the real rendered document; the S badge marks the factsheets in the release sample.

The release checklist and the risk-based sample

The release and sign-off screen for wave 2: a checklist confirming 17,015 field checks run, no factsheet held, sample reviewed 49 of 49, rules and clauses at approved versions, distribution targets reachable, maker and checker different people; the sample list with the reason each factsheet was sampled; and the signature dialog with a reason code and the attestation that the reviewer did not prepare the wave.
Fictional sample data. Each sampled factsheet shows why it was sampled: random 5%, or a first-time class or market.

Questions people ask

Where can I find a real fund factsheet example?

Every fund's own website and the fund platforms that list it publish current factsheets. For learning to check them, an annotated example is more useful than a pretty one: our guide on the ESMA factsheet review checklist maps each box to its check, the source and the paragraph behind it, and the fund factsheet template lists the rule behind each.

What errors do factsheets contain most often?

In supervisors' findings, the commonest are inconsistencies with the fund's own documents: the wrong benchmark, a management fee where the KID shows ongoing charges, an outdated risk indicator, past performance shown cumulatively or to the wrong date, and missing statements on where to get the prospectus and KID. ESMA's January 2026 report lists them by country.

How many factsheets should a compliance reviewer sample?

No rule sets a number. A random sample alone finds an error on one factsheet in proportion to its size: 10% opens it about one time in ten. A risk-based sample, which always includes changed, flagged and first-time factsheets plus a random share, finds more for the same effort, and mechanical checks should run on every factsheet.

Is it acceptable to plant errors in a live review?

Yes, if it is agreed in advance by the head of compliance and internal audit, documented, and seeded copies are held back from release. It is the same principle as testing any control: the test must be blind to the people being tested and invisible to clients.

How often should the seeded test be repeated?

Quarterly is a workable rhythm, with a new set of errors each time and a run after any change to the template, the team or the tools. The series of results by error type is the evidence that the review is a control.

Factsheet Automation.
See it on your own files.

An agentic solution from Botminds. Monthly fund factsheets for every class, market and language, checked before compliance signs. Book a demo, or explore how it works.