Evaluate · the replay test

Shareholder report example: replay last cycle before trusting agents

The shareholder report example that proves a drafting tool is the one you already filed. Rebuild last cycle's class reports from the same closed books, compare every figure with what went out, and sort each difference into a match, a tool error or an error in the filed report. Here is the method, what the SEC staff's common issues tell you to look for, and how Tailored Shareholder Reports supports a replay.

Updated · 12 min read

The best shareholder report example for judging a reporting tool is your own last cycle: the class reports you filed on Form N-CSR, rebuilt from the same closed books and compared figure by figure with what shareholders received. A shareholder report, for a US mutual fund or ETF, is the short annual or semi-annual report each share class must send under rule 30e-1, and every figure in it should tie to the fund's financial statements. A vendor's sample proves the tool can produce a nice report; a replay proves it produces yours, and it finds out how your filed reports compare with the SEC staff's list of common errors at the same time. Botminds' Tailored Shareholder Reports is built to be judged this way, on a past cycle's reports and financial statements, before anyone relies on it for the next one.

Where the SEC staff found recurring problems in the first cycle, by report part

Costs of a $10,000 investment
3
Performance discussion and index
3
Holdings graphic
2
Material fund changes and cover
2
Website address and links
2
Content and order
2
Fund statistics
1
Inline XBRL tags
1
Our count of the distinct issues and reminders the SEC Division of Investment Management describes in ADI 2024-14 (Nov. 6, 2024), grouped by report part; the staff's suggested practices are not counted. The ADI gives no frequencies, so the bars show where problems were found, not how often.Source: SEC staff, ADI 2024-14

What is a shareholder report? The document you are testing against

For a mutual fund or ETF registered on Form N-1A, a shareholder report is the annual or semi-annual report rule 30e-1 requires it to send, since July 24, 2024 in the tailored form: one short report per share class, in the order Item 27A of Form N-1A prescribes, tagged in Inline XBRL and filed as Item 1 of Form N-CSR. Each report rests on numbers that exist elsewhere first: the class's expense ratio and total return in the financial highlights, net assets and holdings in the statements, index returns from the index provider.

That dependency is what makes a replay possible. A filed report is a fixed answer to a fixed question: given these books, what should each line say? Rebuilding it from the same books and comparing line by line is a test with a known answer for every figure, which no demonstration on sample data can offer. The parts and their rules are set out in the annotated tailored shareholder reports example.

What errors did the SEC find in shareholder reports? The replay's first targets

The SEC staff's review of the first cycle under the new rule, published as ADI 2024-14, reads like a test plan. In the costs table, funds "annualized the expenses in dollars" in semi-annual reports, multiplied the percentage by $10,000, which the staff called "not correct", and showed cents where "the figure must be rounded to the nearest dollar". In the performance section, ETFs added market-price returns and funds used indexes that do not qualify as broad-based. In holdings, percentages lacked a stated basis. Funds described material changes without the cover statement, or the reverse. Links were broken or led nowhere specific, and some funds tagged every index in their data as broad-based.

Every one of those is mechanically checkable on a filed report, which makes the replay useful even before the tool's accuracy is the question. A replay that finds the staff's issues in your own last cycle has paid for itself in what it tells the treasurer; one that finds none is evidence the cycle's controls worked. Either result belongs in the conversation with the fund's chief compliance officer, since examiners are watching the same area: the SEC's fiscal 2027 examination priorities name fund fees and expenses, waivers and reimbursements included, as a particular focus for registered funds (SEC Division of Examinations).

How do you test a shareholder report tool? The replay, step by step

Run it as an audit, not a demonstration: freeze the inputs, let the tool work without help, and decide the scoring before anyone looks at the output. The steps below are a method we suggest; the controls they rest on, maker and checker separation and evidence for every figure, are the same ones the solution uses in a live cycle.

A replay of last cycle in six steps

  1. A person decides

    Freeze the inputs

    Collect what the team had at filing: the closed trial balance, class expenses and daily net assets, financial highlights, index returns as final at the period end, holdings, supplements and minutes. Nothing from after the filing date.

  2. Agent

    Draft every class report

    The agents rebuild each class report in Item 27A order from the frozen inputs, with every figure linked to its page.

    Never: sees the filed reports while drafting.

  3. Agent

    Recompute and tie out

    One agent recomputes each expense example and return from raw class data; another ties each figure to the statements without seeing the first working.

    Never: marks a figure tied without a page.

  4. Agent

    Compare with what was filed

    Every figure and text element of the rebuilt report is compared with the filed Form N-CSR Item 1 report and its Inline XBRL facts; each difference is listed with both values.

  5. A person decides

    Classify each difference

    Financial reporting and the fund administration lead sort each difference: a match after rounding, a tool error, or an error in the filed report.

  6. A person decides

    Score against the bar set in advance

    The treasurer reads the results against thresholds agreed before the run, and decides whether to go live and on which funds first.

Suggested method for a replay on a closed cycle. Person steps keep the judgments: what counts as an error, and whether the result is good enough.

Three kinds of difference: match, tool error, filed error

Every difference a replay finds falls into one of three bins, and the second and third matter in opposite ways. A match is a difference that disappears under the form's own rounding, such as a ratio carried to three decimals in one place and two in the other. A tool error is a figure the tool got wrong against the books: it counts against the tool and should be traced to a cause, because one cause can produce many errors. A filed error is a figure the tool got right and the filed report got wrong: it counts for the tool, and it is a finding the fund has to deal with.

The sample data shows what the bins look like in a live cycle. Across the fictional Quillfield family's annual cycle, two agents checked 8,649 figures independently and agreed on 8,635. Of the 14 where they differed, financial reporting settled 12 as rounding in the source workbook and 2 became breaks, each investigated with both workings. The same family's previous annual cycle, by its own account, ended with sign-off on day 58 and three amendments afterwards: the kind of filed errors a replay of that cycle would be expected to surface.

Sample cycle: where the 14 disagreements between two agents went

14 figures where the agents differed, of 8,649 checked

86%
14%
  • Rounding in the source workbook, 12 · 86%
  • Breaks, 2 · 14%

29 breaks caught across the cycle, as of day 45

41%
59%
  • Resolved, 12 · 41%
  • Open, 17 · 59%
Sample data from Tailored Shareholder Reports for the fictional Quillfield Funds, annual cycle for the year ended August 31, 2026. Breaks include figure disagreements and failed checks such as length, links and holdings graphics.

What should a shareholder report tool be measured on? Set the bar before you look

Measure it on agreement, on what it catches, on what it raises falsely and on the hours it leaves for people, and write the thresholds down before the replay runs. A threshold chosen after seeing the results tends to fit the results. No regulator or independent study publishes accuracy rates for shareholder report tools, so the numbers below are a starting point for a fund's own decision, not a benchmark.

Replay measures, and how to read each one

MeasureHow to compute itWhat a good result looks like
Figure agreementRebuilt figures equal to the filed or corrected value, after the form's rounding, ÷ all figuresVery close to all; every miss explained by a cause, not left as noise
Filed errors foundKnown amendments and later corrections the replay finds ÷ all of themAll known corrections found, plus any the team had not known about
Staff common issues checkedADI 2024-14 checks the tool ran on every report ÷ checks that applyEvery applicable check, on every class report
False alarmsBreaks a reviewer closed as not an error ÷ all breaksLow enough that reviewers read every break rather than skim
Root causes sweptErrors that share a cause found across all funds once one was foundOne cause, one sweep, every affected class listed
Reviewer hoursHours spent classifying differences and reviewing breaksWell below the hours the cycle took by hand
Suggested measures for a replay on a closed cycle; thresholds are the fund's own. 'Known amendments' means Form N-CSR/A filings and internal corrections from the replayed cycle.

SEC staff common issues, run as checks on every report

Settings screen listing tolerances (costs of a $10,000 investment within $1, costs paid as a percentage within 0.01 points, average annual total returns within 0.01 points and NAV-based, holdings graphic 100.0% within 0.1, key fund statistics exact, index values final only) and checks from the SEC staff's common issues in ADI 2024-14: expense example uses the average account value, semiannual dollar cost not annualized, dollar cost rounded, ETF returns NAV-based only, broad-based index qualifies and additional indexes tagged separately, past performance statement prominent, portfolio statistics under fund statistics, holdings basis and credit-quality method stated, material changes and cover statement both present, website link specific and working, no content Item 27A does not permit and rule order kept.
Sample settings in Tailored Shareholder Reports. In a replay the same tolerances and checks run against the filed reports, so the staff's issues are tested on last cycle as well as the tool's drafts.

Can you re-file a shareholder report? The N-CSR/A

Yes: a correction to a filed report is made by filing an amendment, Form N-CSR/A. The SEC staff addressed the common case in its tailored shareholder reports FAQ, where a filing contained many class reports and only one needs correcting. In the staff's view the amendment can reasonably carry the whole amended class report, every element of it tagged in Inline XBRL, not only the corrected parts; the XBRL tagging guide quotes the answer. That is a staff view, not a rule, and whether and how to correct is a judgment for the fund, its officers and counsel.

Two rules frame that judgment. The certifications the principal executive and principal financial officers signed say the report did not "omit to state a material fact necessary to make the statements made" not misleading, so a material error is not left standing. And for the website half of the obligation, rule 30e-1 has a safe harbor: materials that are temporarily not available as required still count as available if the fund "has reasonable procedures in place" and "takes prompt action" once it knows or should have known (17 CFR 270.30e-1(b)(2)(vi)). A replay that finds a broken link in last cycle's reports is exactly the kind of knowledge that starts that clock.

How many shareholder reports should a test replay? All of them, with people reading a sample

Replay every class report of the cycle if the tool can do it overnight, and spend people's time on the differences and on a risk-based sample of the matches. Rebuilding is cheap for software and checking is expensive for people, so the useful design is full coverage by machine and targeted reading by humans: every difference classified, plus a sample of agreed reports read end to end to confirm that "agreed" means right.

The solution uses the same logic for approval in a live cycle. In the sample, the treasurer approved 84 clean reports in one signed batch only after ticking each report in a 10% sample, eight reports chosen by risk: the largest funds, classes with a break in the last cycle and classes new this year. A replay's sample should lean the same way, toward the classes where errors are most likely and most costly.

Approving clean reports only after reading a risk-chosen sample

Approve 84 clean reports dialog: the Filing coordinator chose a 10% sample, 8 reports, by risk: the largest funds, classes with a break last cycle and classes new this year. Ticked: Health Sciences Class A, European Equity Class C, US Small Cap Growth Class I, Mortgage Securities Class R6, Equity Income Advisor class, with net assets and figures tied. An attestation that the sample was reviewed with no exceptions and all 84 reports are approved, reason code SM, sample reviewed, no exceptions, signed as the fund treasurer and principal financial officer.
Sample data in Tailored Shareholder Reports. The approval cannot be signed until every report in the sample is opened and ticked; the same risk-based sampling suits the human half of a replay.

What a replay tells you that a demonstration cannot

A demonstration shows that a tool can draft a well-formed report. A replay shows four things about your own operation: whether the tool reads your fund accounting and performance data as they really are, how often it disagrees with figures your team signed off, whether it finds the corrections you already know about, and how many of the staff's common issues were sitting in your filed reports. The last of those is useful on its own, whatever the decision about the tool.

It also shows what the people around the tool will actually do. In Tailored Shareholder Reports, every break carries both agents' workings and a proposed fix; a fund administration analyst proposes, a different person approves, and the decision goes on an append-only audit trail. A replay is the first time a team sees that rhythm on its own funds, and it is the honest way to decide whether the time saved is real. The limits the agents work under are the other half of that decision, and the cost of a tailored shareholder report is the baseline the time saved is measured against.

Setting up a replay of last cycle

  1. Pick the cycle and freeze its inputs

    Books, highlights, index returns and documents as they stood at filing.

  2. Gather the filed reports and their Inline XBRL

    Form N-CSR Item 1 for each registrant, and any N-CSR/A.

  3. List the known corrections

    Amendments and internal fixes from that cycle, so finding them can be scored.

  4. Write the thresholds before the run

    Agreement, false alarms, corrections found, reviewer hours.

  5. Keep the tool blind to the filed reports while it drafts

    Comparison happens after drafting, not during.

  6. Classify every difference with two people

    Match, tool error or filed error, with the reason written down.

  7. Trace each tool error to a cause

    One worksheet method can produce many wrong figures.

  8. Take filed errors to the treasurer and counsel

    They decide whether a correction or amendment is needed.

A suggested setup; the classification and the decision to correct stay with the fund's people.

To replay one of your own past cycles, start from Tailored Shareholder Reports.

Questions people ask

Where can I find a real shareholder report example?

Every Form N-1A fund files its class reports as Item 1 of Form N-CSR on the SEC's EDGAR system, and posts them with Items 7 to 11 at the website address on each report's cover. For testing a tool, the most useful example is your own fund's last filing, because the right answer for every figure is already known.

What is an N-CSR/A?

It is an amendment to a Form N-CSR filing, used to correct or complete a filed report. For tailored shareholder reports, the SEC staff said it would be reasonable for the amendment to include the complete corrected class report with all of its elements tagged in Inline XBRL, not only the corrected figure.

Should the tool see last cycle's filed reports while it drafts?

No. A tool that can copy last cycle's text and figures will match them, including their errors, and the replay will measure nothing. Give it the inputs the team had and compare only after it has drafted.

How long does a replay take?

The machine part runs overnight for a family of the sample's size. The human part depends on how many differences the run produces and how large a sample of agreed reports the team reads, so plan reviewer time from the count of differences, not from the number of reports.

What if the replay finds more errors in our filed reports than in the tool?

Then the replay has done its job twice. Each filed error is taken to the treasurer and counsel to decide on correction, and the pattern of errors shows which manual controls were weakest, which is also where the tool's checks should be watched most closely in the first live cycle.

Does a replay need the auditor involved?

Not to run it. It uses the fund's own books and filed reports, and the results are internal. The fund may choose to share the method and the evidence with its auditor or chief compliance officer, since both care about the controls behind the figures.

Tailored Shareholder Reports.
See it on your own files.

An agentic solution from Botminds. Drafts and ties out each share class's shareholder report and N-CSR items, for fund treasurers Book a demo, or explore how it works.