A NAV validation process is the set of checks a fund manager or administrator runs on each net asset value before it is released: comparing it with an independent NAV, testing price movements against tolerances, reconciling positions, cash and shares, and investigating whatever falls outside. Most teams can describe their checks; few can say what share of real errors those checks would catch. The way to find out is a seeded-break test: take past NAV files that are known to be right, plant breaks of known type and size, run the validation blind, and count, by break type, how many it caught and how many clean NAVs it flagged by mistake. Only then do the tolerance bands mean anything. Botminds' NAV Oversight is validated this way, on a fund's own past files, before it goes live and before any change to its agents.
Plant breaks in proportion to what actually goes wrong: one sample book's causes
The examples on this page come from NAV Oversight's fictional sample book, Quillfield Funds Trust and its fictional administrator, Northgate Fund Services. The rules and cases are the regulators' own.
What is a NAV validation process?
It is the control between the calculation and the publication. The FCA's handbook gives one of the clearest lists of what it must contain. Among the pricing controls an authorised fund manager should run, the controls should ensure that “reasonable tolerances are set for movements in the key elements of a valuation and movements outside these tolerances are investigated”, that asset prices are accurate and up to date, that investment transactions are promptly reflected, that the components of the valuation are reconciled to their source records, and that the sources of prices not obtained from the main pricing source are recorded and regularly reviewed.
The same guidance adds two lines that make testing unavoidable. The manager “needs to be able to demonstrate that it has effective controls over its calculations of unit prices”, and evidence of the controls being exercised should be retained. A list of checks demonstrates that controls exist; only a test demonstrates that they are effective.
Why a tolerance band alone misses what matters
Because the worst errors are the ones that do not move. A movement tolerance compares today's price with yesterday's and flags a jump. A stale price has no jump: it simply stays where it was while the market moves away. In the solution's sample evening, the administrator's dealer-quote feed carried a bond at 101.80 for five business days while the primary vendor's price fell, day by day, to 99.40; a day-on-day check on the administrator's own prices saw five quiet days.
The SEC's Heartland order describes the opposite trick with the same blind spot. Between March and May 2000, valuations of certain municipal bonds “were reduced daily in increments of 0.5 percentage points until the valuations reached 80 percent of par value”, without any market or credit event to explain them.
“The effect of these incremental price reductions was to spread out and thereby minimize the impact of the total valuation declines on the Funds' NAVs and on the performance reported by the Funds.”
SEC order, In the Matter of Heartland Advisors, Inc. et al., January 25, 2008
A half-point move a day can sit inside a movement tolerance. A validation process that is only tested on sudden jumps will report a high catch rate and still miss both patterns. That is why the planted breaks must include slow ones: a price frozen for several days, and a price walked down in small steps.
What breaks should a NAV check catch?
Every cause that has produced a wrong NAV in your book, and the ones that have not yet. The table lists the common types, how to plant each in a copy of a past NAV file, and which independent source should catch it.
The breaks to plant, and the evidence that should expose each
| Break type | How to plant it in a past NAV file | What should catch it |
|---|---|---|
| Stale price | Hold one holding's price flat for three to five days while the vendor moves | A second price source, and a days-unchanged test, not only a movement band |
| Walked-down price | Step a price down a fraction each day toward a lower level | Comparison with an independent price, which jumps where the walked price does not |
| Missed corporate action or distribution | Drop an ex-date dividend, a split or a rights issue | The corporate-action feed, read in the issuer's own language |
| Wrong FX rate or fixing time | Swap the valuation-time rate for another fixing | A second FX source at the policy's stated time |
| Late or missing trade | Remove a trade from the position file for its trade date plus one | Reconciliation to the manager's order records |
| Accrual error | Omit a fee change or misstate daily income | An independent accrual from the agreement or the board minute |
| Share count or capital activity | Alter subscriptions or redemptions for one class | The transfer agent's shares outstanding by class |
| Swing or dilution adjustment | Apply the wrong swing factor, or none | The fund's swing policy and the day's net flows |
| Several small errors | Plant three sub-threshold errors on successive days | An aggregation test across days, where the regime requires it |
Two types deserve more than their share of the test. Corporate actions are often announced in the issuer's own language, and a check that only reads English notices will miss them. And breaks in a single share class, such as a class-specific fee, disappear in fund-level checks.
The evidence a stale-price check needs: the price history and the issuer's notice
How do you know your NAV checks work?
By counting two numbers per break type: the share of planted breaks the process caught, and the share of clean NAVs it flagged. The first is recall, the measure that tells you what slips through; the second is the false-alarm rate, the measure that tells you how much of the team's evening is spent on nothing. A process that flags every NAV has perfect recall and is useless at 5:40 p.m.; one that flags nothing has no false alarms and is useless at all times.
The scorecard for a seeded-break test, one row per break type
| Measure | How to count it | What it tells you |
|---|---|---|
| Planted | Breaks of this type seeded into the test files, by size band | The denominator; plant enough of each type to matter |
| Caught | Planted breaks flagged at or above the right severity | With planted, gives recall: caught ÷ planted |
| Caught for the right reason | Flags whose explanation names the planted cause | Whether the process can explain, not only detect |
| Missed | Planted breaks not flagged, or flagged below the policy line | What would have been published wrong |
| False alarms | Clean NAVs flagged for investigation | With clean NAVs tested, gives the false-alarm rate |
| Time to flag | Minutes from files received to the flag | Whether the catch comes before the cut-off |
Size matters as much as type. Plant each type at three sizes: inside the band the team treats as noise, between that band and the policy line, and over the line. A good process catches the second band reliably, because those are the breaks that become errors on a heavier day. NAV Oversight's own design uses an inner band at half the policy edge, half a cent for a US fund on a one-cent policy, inside which the agents must still explain any gap.
How do you plant breaks without contaminating production?
Work on copies of past NAV files, keep the seed log away from the people and systems being tested, and mix in clean days. The design below separates the person who plants from the process that checks, which is the same maker-checker principle the process itself relies on.
A seeded-break test, from past files to a scorecard
A person decides
Test ownerPicks 20 to 60 past business days whose NAVs are known to be right, across funds, classes and regimes.
A person decides
Test ownerPlants breaks by type and size in copies of the administrator's files and the books, recording each in a sealed seed log.
Agent
Validation process under testRuns blind on the mixed set of seeded and clean days, exactly as it would on a live evening.
A person decides
Reviewer, independent of bothOpens the seed log only after the run, and scores each flag and each miss.
A person decides
Head of fund accountingReads the scorecard by type and size, and decides whether the bands, sources or staffing change.
Two practical rules keep the test honest. Never tune the process on the same seeded days you score it on: tune on one set, score on another. And re-run the test whenever something changes, a new administrator, a new price vendor, a new asset class or a new version of the software, because a catch rate belongs to the process that produced it.
What examiners and auditors expect to see
Evidence that the checks were designed to catch real errors and were tested. The SEC's Rule 2a-5 adopting release expects the valuation designee's records to include work papers “such as those documenting the valuation designee's monitoring and conducting of price challenges, stale price analysis, and testing such as calibration or back-testing.” The release concerns fair value methodologies, but the same discipline, a test with a result on file, is what a board or auditor will ask of NAV checks.
Testing some holdings and not others is itself a finding. In its 2016 order against Calvert Investment Management, the SEC noted that Calvert “failed to back-test fair value determinations for the Toll Road Bonds, despite engaging in such back-testing for other holdings.” The bonds left out were the illiquid ones whose prices were wrong. A seeded test that plants breaks only in liquid equities repeats the mistake in another form.
A seeded-break test for a NAV validation process
- Use your own past files
The administrator's real NAV files and your books, for days whose NAVs are known to be right.
- Weight by history, cover the rare
Plant in proportion to last year's causes, with at least a few of every type.
- Plant three sizes
Inside the noise band, between it and the policy line, and over the line.
- Include the slow breaks
Prices frozen for days and prices walked down in small steps.
- Include single-class and multi-day breaks
A class-specific fee, and small errors on successive days.
- Seal the seed log
Planted by one person, scored by another, unseen by the process under test.
- Score by type and size
Recall, false alarms and time to flag, not one blended rate.
- Re-run on every change
New administrator, vendor, asset class or software version.
Is NAV validation the same as NAV reconciliation?
No, though validation uses reconciliation. Reconciliation agrees the components, positions to the custodian, cash to the bank, shares to the transfer agent, and is mostly done by the administrator, which in Luxembourg must reconcile against reliable and independent sources. Validation asks whether the resulting NAV is right: it compares the whole figure with an independent NAV, explains the difference by cause and decides whether the NAV can be published. A NAV can reconcile perfectly and still be wrong, because a stale price reconciles to itself.
How NAV Oversight is tested on your own files
The solution applies the method to itself in two places. At first-time setup it runs a back-test over the last 20 business days of the administrator's NAVs, to find anything already published over policy; in the sample book that back-test found a five-day error, an inflation index ratio applied a month late, worth $203,357 to the fund. And any change to an agent's model or instructions is run on 1,200 past NAV files before it can become a new version for the Head of Fund Accounting to approve.
What the live history shows once the process runs: breaks caught, by cause and month
A seeded test tells you what the process can catch; the live history tells you what it did. Read together, they show whether the bands are set where the errors are. Why the agent that checks must use different sources from the one that builds explains the design choice the test is probing, and the policy lines the bands sit inside differ by regime. The NAV Oversight page shows the live evening.
Questions people ask
What does NAV validation mean?
Checking a calculated NAV before it is released: comparing it with an independent figure, testing the components against tolerances and sources, and investigating anything outside them. It is the control the FCA describes as reasonable tolerances set for movements in the key elements of a valuation, with movements outside them investigated.
What is the NAV validation cycle?
The daily sequence from the administrator's files arriving to the NAV's release: receive and check the files, build or refresh the independent NAV, compare, explain differences, decide, challenge where needed, re-verify any restrike and release before the cut-off. A seeded-break test measures how well each step catches what it should.
How often should a NAV validation process be tested?
Before it goes live, whenever a component changes, such as an administrator, price vendor, asset class or software version, and periodically in between; Rule 38a-1 requires an annual review of a US fund's compliance policies and their effectiveness, which a test result supports.
What is an acceptable false-alarm rate for NAV checks?
No regulator or independent study sets one. The practical test is time: the flags a team receives must fit before the cut-off with time to challenge the administrator. Measure it on your own seeded days and set the bands so the second size band, between noise and the policy line, is caught reliably.
Who should run a seeded-break test?
Someone independent of the people and the system being tested, such as internal audit or a model-risk team, with the seed log kept from the operators until scoring. The fund accounting team should see the results by break type, not only a single score.