Validation document review, explained.

A validation document is a promise and its proof, and review decides whether either one can be trusted

A validation document's path from draft to approved record, with two reviews on the way

1. Write the protocolThe validation engineer writes tests and limits on the controlled template, against the requirements and procedures.
2. Review before executionA quality reviewer checks the protocol. Limits, part selection and citations can all still change at no cost.
3. Run the studyOperators follow the approved protocol and record results, which people call execution. Any significant change is written up as a deviation.
4. Review the reportThe reviewer checks the report against the protocol, the raw data and the requirements.
5. Approve and keepNamed people approve. The record stays available for inspection, and a later change can reopen it.

Steps 2 and 4 are the subject of this guide. The people, company and figures in the example are fictional.

What new validation reviewers rarely expect, from a date to look up to a score nobody publishes

What protocol, acceptance criterion, redline and ndc mean in a review

Fourteen terms are defined in the one-page summary.

IQ, OQ, PQ, test method and design verification documents: what a validation reviewer checks in each

In shortValidation is a family of documents, and each proves something different. The reviewer's questions stay fairly constant across them: is it on the right template, does it cite what was in force, does every requirement reach a test, were limits set first, can the raw data be trusted, and can a stranger read it.

A validation protocol review and a validation report review ask different things

A protocol review asks whether the plan can succeed: are the limits complete and stated, are the parts and samples the right ones, is the original record named, is the template current. A report review asks whether the plan was followed and what it showed: does each criterion have a result, do the numbers recompute from the raw data, are corrections explained, were instruments in calibration on those dates, does the conclusion follow.

The same person often does both, and many defects in a report began as a gap in its protocol.

Which rule asks for validation review now: Part 820 after the QMSR, Part 211, and EU GMP Annex 15 awaiting amendment

In shortNo rule says "use a validation document reviewer". The duty comes from rules that documents be approved by someone responsible, that criteria exist and that results be recorded. For US devices those rules, in 21 CFR Part 820, changed on February 2, 2026, when FDA's Quality Management System Regulation (QMSR) took effect. For US finished drugs they sit in Part 211. In Europe, Annex 15 of 2015, the EU GMP chapter on qualification and validation, applies, with a draft amendment due for consultation in December 2026 and publication planned for April 2028.8

The old device labels and the clauses that replaced them

FDA's final rule calls the requirements of ISO 13485 substantially similar to the old ones. It incorporates ISO 13485:2016 by reference and reserves the old sections, which means it empties them, including 820.30, 820.40 and Subparts C to O, where 820.72 and 820.75 sat.9 Many checklists, templates and training slides still carry the old labels. A reviewer should know both, because records made before February 2026 use the old ones.

The mapping is our reading of 89 FR 7496, the clause table in FDA's inspection program and FDA's letters.9,13,2,14 ISO sells its text, so we read summaries and FDA's wording.6

What each text says about review, limits and records

Why acceptance criteria must be fixed before the study: what EU GMP and FDA say about a limit set late

In shortA limit stated before measuring is a test. A limit stated after measuring can be drawn around the result, whatever anyone intended. That is why protocol review has the highest return in this work: everything found there is cheap, and several defects can only be fixed there.

Annex 15 speaks of "pre-defined acceptance criteria" and says results that miss them "should be recorded as a deviation and be fully investigated". It adds that "any significant changes to the approved protocol during execution, e.g. acceptance criteria" should be documented as a deviation and "scientifically justified".3 A later change to a limit is possible, then, and it carries a burden of proof. FDA's drug guidance says the protocol's departures must be "justified and approved" by the appropriate departments and the quality unit "before implementation".7

What the same defect costs, depending on when a reviewer finds it

This is our reasoning from the texts above, with no published outcomes behind it. We found no public figure for the cost of a repeated validation study. Section 18 shows how to find yours.

Which revision governs: checking every citation against its effective date and the first test date

In shortEach controlled document has revisions with effective dates, and a validation cites some of them. The reviewer asks which revision was in force when the work was done. ISO 13485 requires control of documents, and Part 211 requires that procedures and their changes be reviewed and approved.13,15 The specific test, "in force on the first test date", is a quality system's own rule, and a sensible one to write down.

Take the report in this guide's example, TMV-0193. It cites SOP-QA-0112 at revision C. Revision D took effect on June 15, 2026. The pull tests ran August 18 to 20. So the procedure the report names no longer governed the work when it was done, and the change was substantive: revision C allowed total gage error up to 30% of the tolerance and said distinct categories "may be recorded for information". Revision D set 20% and required at least five categories, reported.

Four date questions a good reviewer settles before judging a citation

1. Which date counts? Comparing the effective date with the first test date is common, and some companies use the protocol's approval date instead. Whichever you choose, write it into a procedure, since we found no regulator that sets one.
2. What changed between revisions? A renumbered annex and a tightened limit are different problems. The right response to the first is a correction, and to the second a look at whether the work meets the new limit.
3. Was there a transition period? Many procedures allow studies already under way to finish under the old revision. The citation may then be right, and it needs a sentence saying so.
4. Is the library itself current? External standards change on someone else's schedule. A company's standards register can lag, and a reviewer who trusts it inherits the lag. ISO 10555-1, for catheters, moved to a 2023 edition replacing the 2013 edition and its 2017 amendment.17

The four parts add up to 64, by FDA's cite IDs: 14722 and 447; 454, 3199 and 3838; 3203, 3204 and 3198; and 3201. An old citation inside a validation report would normally be cited under a broader row, so a low count on one line says little about how common it is.

Gage R&R and ndc worked through by hand, and why ten parts from one lot can fail a good camera

In shortA test method validation asks how much of what you measure is the part and how much is the measuring. Reviewers recompute the answer from the study's own numbers. Two statistics matter most: gage error as a share of the tolerance, the allowed range, and the number of distinct categories. The second depends on the parts chosen, so the protocol must choose them on purpose.

In a gage R&R study, several operators measure the same parts several times. The variation splits into repeatability (the same operator, the same part, a different reading), reproducibility (different operators) and part variation (the parts themselves differ). Repeatability and reproducibility together are the gage error, written GRR.

These terms come from the AIAG Measurement Systems Analysis manual, an automotive industry text many device makers adopt in their own procedures. It becomes binding on a company only through that company's procedure. Minitab, a statistics software maker, reports its guideline of under 10% of process variation as acceptable, and its recommendation of five or more categories.4,19 A limit such as 20% of the tolerance is a company's own, written into its procedure.

The arithmetic on the fictional pull-force report

The report printed all of that except the last line, and its procedure requires the last line. A reviewer who recomputes finds the method probably passes, and still sends it back, because a criterion with no reported result cannot be shown as met. She would also ask about the sample report's 14.1% of tolerance: the requirement has only a lower limit, at least 15.0 N, so the tolerance behind that percentage has to be stated somewhere.

Why the protocol must say how parts are chosen

The formula divides part variation by gage error. So ndc rises when parts differ more, with the measuring tool unchanged. Try it on the example's marker band, with a tolerance of ±0.3 mm, a span of 0.6 mm. Suppose the camera's gage error is 0.02 mm. Six times that is 0.12 mm, exactly 20% of the tolerance, so it sits at the limit in Table 2. Now pick the parts.

That is the point of the finding about part selection in the example. Taken at random from a single lot, the parts cluster, and a good camera looks unable to tell them apart. A reviewer who sees "at random from lot 26-0412" in a protocol can spare the engineer a failed study. The same logic works the other way: parts chosen too far apart can flatter a weak method. So the reviewer asks that the choice be stated, and match the range the method will meet in use.

Sample size: the reviewer's second piece of arithmetic

Design verification often proves a requirement by testing some units with no failures. The usual formula, the success run theorem, is n = ln(1 − C) ÷ ln(R), for confidence C (how sure you want to be) and reliability R (the share of units that must work).20 A reviewer checks that the stated sample size matches the stated confidence and that the reason for choosing that confidence appears, often through the requirement's risk. FDA has cited a firm whose validation plans lacked a "rationale for sample size" (section 9).14

Reading a requirements traceability matrix both ways, so an untested requirement and an orphan test both surface

In shortTraceability links each requirement to a test and a result. Read it forward and you find requirements nobody tested. Read it backward and you find tests that prove nothing anyone asked for. A rule that says a method covers a condition is also a requirement, and those are the ones that get lost.

The example's report shows how this slips. The product specification says tip-to-shaft pull force must reach at least 15.0 N after two hours soaking at 37 °C, because that is how the catheter is used. The test method's own scope lists hydrated catheters. A water bath is listed in the equipment table. And section 7.1 of the report covers three other requirements and stops. Nobody lied, and every page looks finished. A reader has to hold three documents at once to see that the hydrated condition was never tested.

What each direction of the trace catches

ALCOA in a validation report: corrections, hand-copied readings, original records and calibration on the test date

In shortA reviewer treats raw data as evidence, and FDA's ALCOA test says what good evidence looks like. On a validation document that comes down to four things to look for: who changed what, whether the reading in the report matches where it was first captured, whether that first capture still exists, and whether the instrument was in calibration that day.

FDA's 2018 guidance sets out the expectation: data "attributable, legible, contemporaneously recorded, original or a true copy, and accurate (ALCOA)".21 It compares audit trail review, the check of a system's own log of who changed what, to "assessing cross-outs on paper".21 MHRA, the UK medicines regulator, notes that the plus in ALCOA+ refers to complete, consistent, enduring and available, and says there is no difference in expectations.22

PIC/S describes the original record as "the first-capture of information".23 That phrase decides the example's finding on results copied by hand: the camera holds images and results with an audit trail, and the protocol tells operators to write readings onto paper. MHRA recognizes hybrid records of paper and electronic data, so this is a choice to make on purpose, and the protocol should name which record is the original.22

Four things on a data sheet that a reviewer stops at

What FDA cites in validation work: process validation rows on Form 483s, and letters under the new device rule

In shortProcess validation was cited 153 times in fiscal 2025 on device Form 483s, the lists of observations FDA investigators leave after an inspection, by our sum of FDA's rows. FDA's letters show what lies behind such rows: no test method validation, no raw data, no pass or fail criteria, no sample size rationale. FDA's newest device letters cite ISO 13485 clauses. The rows cannot say how many defects a document review would have caught.

Five FDA letters and what each says about validation paperwork

Each letter concerns one firm, usually with other findings. They show the kind of gap FDA writes down, and imply nothing about frequency. These firms were chosen for what the letters say.

Why two good quality reviewers return different findings, and how overlap estimates what both missed

In shortWe found no study of agreement between validation reviewers. Other fields measure it, and agreement is modest. Software engineering has studied better ways of reading for 30 years: readers assigned different perspectives cover more of a document than the same number using their usual approach. The overlap between readers can also estimate what everyone missed.

Those numbers come from elsewhere, and they do not transfer directly. A journal referee gives an opinion on whole papers, and a validation reviewer checks facts against a template. Intraclass correlation and kappa both reach 1 at perfect agreement, so .34 and .17 sit far from it. What carries over is the structure. A person reading for everything at once tends to catch what they are best at and what they saw first.

In a controlled experiment at NASA's Software Engineering Laboratory, teams reading from assigned points of view such as tester, developer and user achieved "significantly better coverage" than teams using their usual approach.30 A later replication against checklists pointed the same way, with one exception: on the one document where two perspectives found similar defects, perspective readers were no more effective. The authors offered complementary perspectives as a new hypothesis.31

From reading for everything to reading from one angle

What overlap tells you: the capture-recapture estimate

Biologists estimate a fish population by marking a catch and counting marks in a second catch. Software inspection borrowed the method in 1992 to estimate defects left after a review.32 The arithmetic is short. Reviewer A finds 12 defects, B finds 10, and 6 are the same. If B caught 6 of A's 12, that is half, so B probably catches about half of everything. Then the total is about 12 × 10 ÷ 6 = 20. Together they found 16, so about 4 remain.

How to score any validation reviewer: recall, false alarms and an answer key, with the arithmetic on 63 critical findings

In shortTake past documents, collect what your reviewers found and signed, and treat that as an answer key. Recall says how much of the key a new reviewer finds. The false-alarm rate says how much of what it raises is unsupported. Both have real limits: small numbers leave wide uncertainty, and the key itself contains human misses.

The four counts behind every accuracy claim

• Recall = hits ÷ everything in the key. Report it by severity, because finding every comma and missing a critical is a bad review.
• False-alarm rate, as used here, = raised and not upheld ÷ everything raised. Statisticians call this the false discovery rate, and it differs from a false-positive rate. Ask which one a vendor means.

How sure can you be of a high score on 63 criticals

Take a score of 61 critical findings out of 63, or 96.8%. It sounds precise. An exact 95% confidence interval, the range where the true score plausibly sits, runs from 89.0% to 99.6% for that count, by our calculation with the standard method for a proportion.34 Each single miss moves the score by 1.6 points.

Four ways an answer key can flatter or mislead

What regulators say about AI reading GxP documents, and where a large language model fits in validation review

In shortGxP means the good-practice rules for manufacturing, labs and trials. We found no rule that bans a tool from reading a validation document and flagging what it sees. Draft EU GMP Annex 22, the proposed chapter on AI, would keep large language models (LLMs) out of critical GMP uses, and that wording is under debate. For devices, FDA's Computer Software Assurance guidance reaches AI tools used in the quality management system. Where these texts speak to it, a qualified person stays responsible and the company assesses its own use. The practical risks are misses nobody sees and too much trust in the tool.

What each text means for a tool that reads validation documents

The risk nobody sees: a reviewer reads less because the tool read first

A person who decides each flagged item catches false alarms, and sees nothing of what the tool missed. If a reviewer stops reading the document and reads only the flags, the tool's misses become the review's misses.

A systematic review found automation bias, the tendency to over-rely on automation, is real and is shaped by trust, workload and time pressure, and that accountability, training and how advice is presented can reduce it.44 Those studies mostly concern clinical decision support, so treat them as a warning. Design the review so that a person still reads the parts that matter, sample some unflagged pages, and plant a known defect now and then to see whether people notice.

Who reviews validation documents today, where a sample reviewer's 3.5 hours go, and the public numbers that do not exist

In shortA validation document can pass through four or five people and return to its author more than once. The hours go to clerical checks more than to judgment, by one fictional team's estimate. Almost nothing in this work has a public benchmark, so your own stopwatch is the only credible baseline.

The author, the reviewer, the quality manager, document control and metrology

Four ways teams run validation review today

The few public numbers, and the many that do not exist

TMV-0195 on October 6: seven findings from eleven redlines, the missing ndc limit raised twice, and back to Sam before he measures

In shortSam Patel submits an 8-page protocol. It is read on arrival, then four AI reviewers read it separately, each from its own angle, and fill their own columns. Two of them flag the same missing limit. Dana Okafor, the quality reviewer, opens each finding with the procedure behind it, accepts it or dismisses it, and returns the document with a redlined copy. The software can never approve.

1

Before Sam's protocol, the report on Dana's desk

Dana's review panel, the screen where the four AI reviewers report, is open on TMV-0193, an 11-page test method validation report for tip-to-shaft pull force on the Dravenzo 7F catheter. Four columns hold each reviewer's own located redlines: severity, page, section, and a strike-and-insert edit. A strip across the top shows the panel's score against 36 past reviews from sample data, and each column shows its own.

2

An 8-page protocol, read on arrival

Sam's file arrives as a Word document. Before anyone reads it for defects, the software identifies the document and shows its confidence beside each line: a test method validation protocol, on template TPL-TMV-001 revision D, flagged because revision E is current. Product and requirement R-4.6, instruments VM-05 and GB-11, and the Leiden site are all read from the file.

3

Each reviewer reads all 8 pages on its own

The GxP reviewer checks references against the library. The data integrity reviewer reads data sheets and records. The test design reviewer matches requirements to limits. The writing reviewer reads for terms and numbering. Redlines land as they are found: a superseded template, a blank calibration due date, ten parts taken at random from lot 26-0412, an abbreviation nobody defined.

4

Two reviewers, separately, find the same missing limit

The test design reviewer writes that nothing in Table 2 shows VM-05 can tell parts apart. The GxP reviewer writes that Table 2 omits ndc of at least 5, required by SOP-QA-0112 revision D, section 6.2. They reach the same place from different directions, and the panel marks it. Merged by defect, eleven redlines become 7 findings: 2 critical, 2 major, 3 minor.

5

F-01, with the procedure passage behind it

In the marked-up document, finding F-01 sits on page 4, in Table 2. On the right: what was found, why it matters, and the evidence. SOP-QA-0112 revision D opens at page 6, section 6.2, with the sentence highlighted, and template revision E below it asks for each limit in the table before execution. Both are the sample company's own documents, and neither quotes a regulator.

6

Dana accepts F-01, and the template finding opens next

She accepts it, and it goes into the redlined copy. F-02 opens: written on template revision D, replaced on July 1, 2026 by revision E, which added the limits table. That is why F-01 exists. The suggested edit, the reviewers who raised it and how often your people upheld findings like it sit on the same screen.

7

Seven findings go back to Sam, before he measures

The rest are fixes Sam can make before execution: choose parts across 9.7 to 10.3 mm from at least three lots, name the VM-05 export as the original record, measure from the distal tip as R-4.6 does, add GB-11's calibration due date. Dana returns the protocol with a message: critical items first. The return is under her name on the audit trail.

The report findings, trace matrix, checklist, document library, accuracy scores and grading rules on real screens

In shortBehind the panel sit the reconciled list, the record of what was dropped and why, a located finding for each defect, the traceability matrix, the pass or fail checklist, every open review, the library of revisions, the accuracy screens and the settings that hold your own rules. All figures are sample data.

Four AI reviewers, a reconciler, a markup writer and an accuracy scorer: each one's job and the person who decides

In shortSeven agents each do one narrow job, and each has a line it never crosses. None can approve. A named quality reviewer for each document type decides every finding, with author and reviewer kept apart.

One job each: what the seven agents, from GxP reviewer to accuracy scorer, do and never do

Who writes, who decides and who owns the library: Dana, Sam, Omar, Lena and Maya

Advisory, located, recorded: the controls around a finding

From protocols, templates and past reviews to located redlines: what Validation Document Review reads, returns and aims for

In shortThe solution reads the documents your review already runs on, and returns markup in the formats your authors use. Its outcome figures are design targets, to be measured on your own documents. One proof of concept is on record, and it is stated here exactly as our site states it.

Document systems, validation tools and calibration records this works with, by category

It works with what you already run: a document or quality management system for controlled documents, a digital validation tool if you have one, a calibration or metrology system, and your standards register. Review stays in one place across plants, with a named decider for each document type.

14 minutes, 95% and under 10%: the design targets, and what each one counts

Targets are design goals, measured in the live solution. The 96.8% and 7.4% on the screens are sample data. Section 22 shows how to test any of these on your own files.

Review hours, repeat studies and rework after approval: what a slow or leaky validation review costs you

In shortThree costs add up: reviewer hours you can count, studies repeated because a gap surfaced late, and findings that reach an inspector. Nobody publishes any of the three for validation review, so the only trustworthy numbers are the ones you gather in a week from your own document system.

Five documents and a stopwatch: find your own review hours, rounds, repeats and late findings in a week

The launch date, the good reviewer's time, the inspector's second question: costs no calculator can price

Your return on AI-assisted validation review: reviewer hours, fewer repeat studies, and what stays unpriced

In shortEnter your own numbers. Hours are the part you can defend quickly. Repeat studies avoided are worth more, and you need your own history to claim any. The calculator keeps a person reading every document, so it starts from a saving far below what a vendor's headline would suggest.

Clerical hours, review rounds, repeat studies: the formulas behind a validation review saving

Put in your own document count: reviewer hours freed and repeat studies avoided

From a dated document library to an answer key you trust: eight questions that score your validation review

In shortAnswer honestly, 0 to 2 on each. Your score suggests whether to fix your library and rules first, run a pilot on past reviews, or plan a rollout.

Date your library first: what a score of 0–6, 7–11 or 12–16 asks you to do next

Measured accuracy, located findings, who approves, compliance claims: questions that test an AI validation review vendor

In shortAsk every vendor the same questions, us included. The answers show whether the tool can be scored on your own documents, whether each finding can be checked, whether it could ever approve, and how honest the claims are.

A blind pilot for validation review: score the tool on closed documents, then read one live protocol with it

In shortPull closed reviews and set some aside. Agree on the scores you need before you see any result, the same discipline you ask of a protocol. Score the tool blind against your people's findings, check a sample of its "false alarms" by hand, then run live documents with a person deciding. Decide on the numbers below.

1. Weeks 0 to 2. Write the pilot's own protocol: what you will test, your acceptance limits and who judges. Assess the tool as a supplier. Time five reviews for a baseline.
2. Weeks 2 to 4. Extract 30 or more closed reviews with findings and severity. Keep about a third aside, unseen. Load your library, templates and rules.
3. Weeks 4 to 6. Run the tool on the working set, and tune rules openly. Run it once on the held-out set, and stop tuning.
4. Weeks 6 to 7. Check by hand a sample of its findings your people never raised. Some will be false alarms, and some will be real defects your review missed.
5. Weeks 7 to 10. Read live documents with people deciding each finding, timed. Plant a known defect in a few. Show the record to an auditor who was not involved, and write down why you decided as you did.

Held-out recall, false alarms, new defects and reviewer hours: the numbers on this pilot's scoreboard

Protocols, IQ OQ PQ, ndc, acceptance criteria, the QMSR and AI reviewers: what people ask about validation review

In shortShort answers to what quality and validation teams search for, each one complete on its own. Rules and statuses are as of November 2026.