Testing AI tools for regulatory writing means doing one thing the supplier cannot do for you: running the tool over documents whose right answers you already know. A closed submission is the only such corpus you have. Plant a set of realistic faults in it — a population name swapped, a superseded data cut cited, a result stated more strongly than the table supports — hand the dossier over, and count how many the tool's verifier catches by opening the page it cites. The reason this is the test rather than a nicer one is in the published evidence: when a single fabricated clinical detail is planted in a prompt, large language models elaborate on it between half and four-fifths of the time, and a mitigation prompt tested in the same study cut the average rate by about a third without removing it. Botminds' CTD Module 2 Authoring is built to be tested this way: every statement carries a citation, the verifier opens each cited page, and its verdict keeps the page it read.
The research below is quoted from peer-reviewed journals and from FDA and EMA, linked where it appears. The product figures come from a fictional sample dossier, the Veltrimab BLA, and are labelled as design targets where they are targets rather than measurements.
What happens when nothing opens the page: hallucination rate by model and prompt
How do you evaluate an AI tool for regulatory writing?
You evaluate it the way you would evaluate a new analyst: give it work whose answer you already know, and grade it against the source. Everything else — the demonstration, the reference customer, the accuracy claim in the brochure — is testimony about someone else's documents.
That makes the evaluation a design problem rather than a procurement one, and it has four parts. Choose a corpus where the truth is settled. Decide in advance what counts as an error and what counts as a catch. Decide who grades, and against what. And decide which numbers you will report before you see any of them, so the result is not chosen after the fact.
The rest of this page is one way to do that in two weeks. It is deliberately small: one closed submission, two sections, a short list of planted faults, four numbers. A larger pilot usually buys less than it costs, because the thing you are trying to learn — whether the tool checks its own output against a page — is visible on the first twenty statements.
Why a demonstration tells you nothing
A demonstration is a performance on documents the supplier has prepared. The corpus is clean, the questions are known, and the failure modes that matter in a dossier — a superseded version still in circulation, two sections built from different data cuts, a defined population named loosely — are not present, because nobody plants them in a sales demonstration.
There is a deeper reason, and it is the one the literature keeps returning to. A fabricated or mis-sourced sentence in a regulatory summary is not clumsy. It is fluent, correctly formatted, and plausible to a reader who has not opened the cited page. The EMA's reflection paper names precisely this: generative language models are prone to produce plausible but erroneous or incomplete output, which is why quality review has to be in place before anything is submitted rather than after it is read (EMA, September 2024). A human watching a demonstration is the weakest possible quality review, because everything on screen is designed to look right.
Read in that light, what a demonstration can honestly show you is narrow but real: whether a citation is a link that opens a page at a passage, whether the tool will tell you what it could not verify, and what it does when two sections disagree. Those are structural facts, visible in ten minutes. The accuracy claim is not.
The seeded-error test, step by step
A seeded-error test takes a dossier you have already filed, introduces a known set of faults into its Module 2 drafts, and measures how many of them the tool finds and how many true statements it wrongly flags. The method is borrowed from the adversarial study above, where the researchers "created 300 physician-validated simulated vignettes, each containing one fabricated detail (a laboratory test, a physical or radiological sign, or a medical condition)" and scored whether each model built on it.
The equivalent for a submission is a fault list drawn from your own quality-control history, not from imagination. Five types cover most of what actually goes wrong between Modules 3 to 5 and Module 2.
The seeded-error protocol, from choosing the dossier to reporting the four numbers
- Pick a closed submission, not a live one
An approved or withdrawn application whose source documents are final and whose numbers are no longer in dispute. Nothing in a live dossier should depend on a pilot.
- Choose two sections with different shapes
One numeric and dense, such as 2.7.4.1 exposure, and one argumentative, such as 2.5.4. A tool can be strong on one and useless on the other.
- Build the fault list from your own QC records
Wrong population name for a real figure; a figure taken from a superseded data cut; a result overstated relative to the table; a citation pointing at the wrong page of the right document; a number that appears nowhere in the sources.
- Plant between 20 and 40 faults, and write down where
Enough to separate a real catch rate from luck, few enough that one person can adjudicate every verdict in a day. Keep the key in a file the pilot team cannot see.
- Leave the true statements alone
The false-flag rate is half the result. A tool that flags everything catches every planted fault and is unusable.
- Run it blind, in one pass, with no hints
No one tells the tool which sections were altered. If the supplier wants to tune prompts, that is a second run and it is reported separately.
- Grade against the source page, not against memory
Every verdict is adjudicated by opening the page the statement cites. A verdict with no page behind it is scored as not verified.
- Report the four numbers, and the ones you did not plant
Faults the tool found that you did not plant are the most interesting line in the report: some will be real errors that survived your own QC.
That last step earns its place. On any real submission, a tool reading every page against every statement will surface things your own quality control missed. They are not noise in the pilot; they are the clearest evidence you will get about what the tool is worth, and they are also an uncomfortable finding to put in front of a steering group. Decide before the run how you will handle them.
How accurate is AI at medical writing? What the research actually shows
There is no published accuracy figure for AI drafting of CTD Module 2, and anyone quoting one is quoting a supplier. What exists is a body of work on clinical summarisation and on adversarial prompting, and it supports three conclusions that transfer.
The first is that unchecked models build on false premises at rates no regulatory process could absorb. In the adversarial study, every model tested elaborated on the planted detail in at least half of its outputs under default settings; the weakest produced hallucinations in 82.7% of short cases and the strongest in 50.0%. The spread between models is real but it is a spread between bad and worse.
The second is that prompt engineering helps and does not solve. A mitigation prompt lowered the mean rate from 66% to 44%, and for the best-performing model from 53% to 23%. Cutting an unacceptable rate by a third leaves an unacceptable rate. Adjusting temperature to zero did nothing measurable.
The third comes from the summarisation literature, and it is about measurement rather than models. Croxford and colleagues, reviewing the state of evaluation for medical summarisation, put the problem plainly: evaluating clinical summarisation by large language models "must address the intricacies of complex medical texts and tackle LLM-specific challenges such as relevancy, hallucinations, omissions, and ensuring factual accuracy" (npj Health Systems, 2025). Omissions sit in that list beside hallucinations, and they are the harder half: a summary that leaves out the subgroup where the effect was weakest reads as careful writing.
The four numbers a seeded-error test produces
1cited
share of statements of fact that carry a citation — easy to score, and a tool can win it without opening a page
2opened
share of those citations that survive someone opening the page and comparing the number, the population and the direction
3caught
share of planted faults found, reported by fault type rather than blended into one figure
4flagged
share of true statements wrongly flagged — what the writers will experience as the cost of using it
Citation coverage is not citation accuracy
The most common way an evaluation goes wrong is counting citations instead of checking them. Coverage asks what share of statements carry a reference. Accuracy asks what share of those references, opened, say what the statement says they say. A tool can reach complete coverage and fail accuracy badly, because attaching a plausible reference to a sentence is a generation task and verifying one is not.
Citation accuracy, and the trap in measuring coverage instead
Citation coverage
Statements with a citation ÷ statements of fact
Easy to automate and easy to score. A drafting rule can take this to 100% without a single page being opened.
Citation accuracy
Statements whose cited page confirms them ÷ statements with a citation
Requires someone, or something, to open the page and compare the number, the population and the direction of the claim. This is the number that moves when a tool is good.
Unverifiable rate
Statements whose citation could not be resolved ÷ statements with a citation
A missing page, a superseded version, a reference to a document rather than a page. Count these separately; folding them into either number above flatters the tool.
The worked example from the sample dossier shows what accuracy catches that coverage cannot. A statement in 2.7.3.2 read: 818 patients randomized 2:1 to veltrimab and placebo. It carried a citation, to the right document, to the right table. Opened, CSR VLT-302 page 112, Table 14.1.1.1 shows 824 randomized and 818 receiving at least one dose and forming the full analysis set. Coverage scores this statement as a pass. Accuracy scores it as a disagreement, and it is a disagreement: the number was real, the population name was wrong, and 2.5.4 already said 824.
That fault type — right number, wrong population — is the one worth over-weighting in a fault list, because it is the one a fluent summariser produces naturally and a human reviewer forgives. It is also the one that puts two sections of a dossier at odds with each other, which is how a single exposure figure travels and where it goes wrong.
What a verifiable draft looks like: a verdict per statement, with the page it was read against

Four numbers worth measuring, and three that are not
The four are in the figure above: coverage, accuracy, catch rate by fault type, and false-flag rate. Report them per section, because a tool that handles dense numeric text can be weak on the argumentative sections, and the two will be written by different people.
The three not to report are the ones that make the best slide.
Three measurements that look convincing and mean nothing
- Hours saved in the demonstration
Measured against a draft nobody had to defend, on a document the supplier chose. It tells you how fast the tool writes, not whether the writing is true.
- Words or sections produced
A tool that writes more produces more to check. Output volume is the input to the real cost, not a saving.
- Reviewer satisfaction after the pilot
Readers rate fluent text highly, and fluency is exactly what a fabricated sentence has. Ask the same reviewers to open four cited pages and the score changes.
Time saved is worth measuring eventually, but only after accuracy has passed, and only on a real section drafted end to end including the checking. The sample dossier's own time figure is stated as a design target rather than a result — a first Module 2 draft in five weeks, against about 22 weeks by hand on the sponsor's last submission — and a target is the right label for a number produced by the people who built the thing.
Who grades the output, and against what
Grade against the source page, and have it done by someone who did not run the pilot. Both halves matter. Grading against a reviewer's recollection of the study reimports the failure mode the test exists to measure; grading by the team who championed the tool invites an argument nobody can settle afterwards.
The summarisation literature is unambiguous about who should do it: "Human evaluations remain the gold standard for LLM outputs", Croxford and colleagues conclude, while noting that the established clinical rubrics were built for clinician-written notes and need adapting for model-generated text. For a regulatory pilot, adapting means scoring against the cited page rather than against a quality rubric, because in a dossier the question is not whether the writing is good but whether it is true.
In practice two graders and an adjudicator are enough for 40 planted faults. Give them one rule: a statement is verified only if the page opened confirms the value, the population and the direction. Anything else is a disagreement or an unverifiable, and the distinction between those two carries real information about the tool.
What a resolved disagreement should leave behind for a grader to read

What Part 11 validation adds, and what it does not
If the tool will touch records that are part of a submission, it falls under 21 CFR Part 11, whose first control is validating the system: showing that it is accurate, reliable and performs consistently as intended, and that it can tell an invalid or altered record from a sound one (21 CFR 11.10(a)). Your quality function will ask for that, and it should.
What validation establishes is that the system does what it is specified to do, repeatably, and that the record it keeps is sound. What it does not establish is that a generated sentence is true. A validated system can reliably produce a draft and reliably log who approved it while the draft contains a wrong population name, because the specification is about the system's behaviour, not about the content's agreement with a study report. The seeded-error test measures the second thing, and nothing in a validation package substitutes for it.
The two fit together neatly if you keep them separate. Validation covers the record: the audit trail, the signature manifestation, access control. The pilot covers the content: coverage, accuracy, catch rate, false flags. FDA's credibility framework is the bridge — it asks for evidence that a model performs in a stated, narrow context of use, which is exactly what a seeded-error report is (FDA draft guidance on AI, January 2025), and what each regulator allows and who may sign sets out the rest of the rule.
A two-week evaluation on a closed submission
Week one is preparation and runs mostly without the tool. Choose the dossier and the two sections. Pull the final source documents and the versions that were superseded during the original programme, because a tool that cannot tell a superseded data cut from a final one will fail on a real submission and should be given the chance to fail here. Build the fault list, plant the faults, and lock the key.
Week two is the run and the grading. Load the sources, let the tool index them, and have it draft or verify the two sections in one pass with no coaching. Export the statement-level verdicts. Two graders work through every flagged statement and a random sample of 60 unflagged ones — the unflagged sample is how you measure what the tool missed, and skipping it is the most common flaw in pilots of this kind.
What you should have at the end is a two-page report: four numbers per section, a breakdown of catches by fault type, a list of findings you did not plant, and a note on what the tool did with the superseded document. That report is the credibility evidence. It is also harder for a steering group to argue with than any demonstration, because it is about your own dossier and the committee can check it.
The end state the pilot is testing for: every statement verified, and the history of how

Every figure labelled as a target on the solution's own pages — statements cited to a source page, every citation checked — is a design goal, not a measured result, and a pilot is how it stops being one. To run the test on your own closed submission, start from CTD Module 2 Authoring.
Questions people ask
How do you test for AI hallucinations?
By planting a known falsehood and seeing whether the system builds on it. That is the design of the adversarial study the figures above come from — one fabricated detail per case, scored on whether the model elaborated — and it transfers directly to a dossier: a figure that appears in no source document, planted in a draft, should come back flagged as unverifiable rather than cited to a plausible page. Testing by reading output and looking for things that feel wrong does not work, because the failure is fluent.
How many documents do you need for a pilot?
Fewer than most pilots use. Two Module 2 sections from one closed submission, with their source documents, and 20 to 40 planted faults will separate a tool that opens pages from one that does not. The constraint is not statistical power, it is adjudication: every verdict has to be checked against a page by a person, and that is a day's work for 40 faults plus a sample of unflagged statements.
Can you use a closed submission to test an AI tool?
It is the best corpus available, and the only one where the right answers are settled. Use an approved or withdrawn application rather than one under review, keep the data inside whatever environment your agreements require, and treat the planted faults as test artefacts that never leave the pilot. Nothing from the exercise should be able to reach a live dossier.
Does 21 CFR Part 11 require validation of an AI tool?
Part 11 requires validation of systems used to create, modify, maintain or transmit electronic records subject to the predicate rules, to ensure accuracy, reliability, consistent intended performance and the ability to discern invalid or altered records. A drafting tool that holds submission content is such a system. What validation does not do is establish that a generated sentence agrees with its source, which is a separate measurement.
Should the writers who would use the tool be the ones who grade it?
They should be in the pilot and not be the only graders. The writers know which faults are realistic and which are contrived, which makes them the right people to build the fault list. Grading is a different job: it needs someone who did not choose the tool, working against the cited pages rather than against knowledge of the programme, with a third person adjudicating disputes.
Is a time saving a fair measure of an AI writing tool?
Only after accuracy has passed, and only when the measurement includes the checking. A draft produced in a day that takes three weeks to verify has saved nothing; a draft produced in a week whose statements are already matched to their pages has moved the quality-control work forward rather than removing it. Measure the whole path from source documents to a section its owner is willing to sign.