WorkfloweDiscovery & Litigation Support

TAR Validation

TAR validation that draws a blind, seeded sample from the documents you are not producing, measures recall with its interval, and drafts the disclosure your partner signs.

Watch it work Book a walkthrough

Follow the sample from the draw to the signature ↓

Watch with transcript →

The problem

The other side asks how many relevant documents the review left behind

Review stops at the model's cutoff, and the discard pile holds most of the collection, largely unread. Validation starts late. The sample size gets argued by email, reviewers code with the model's score in view, and a statistician on retainer works the interval out once in a spreadsheet and rounds it by hand.

Then a letter is drafted from a template, and the partner signs figures she would struggle to put to a judge. If the interval was never stated, or the coding was not blind, a motion to compel follows and the review can reopen, which is the most expensive outcome in the matter.

Built foreDiscovery managersReview statisticiansSupervising partnersSenior review attorneys

The Problem Today

When review stops before each document is read, opposing counsel will ask what was missed, and want proof.

66%: of 559 practitioners use generative AI for document review published
73%: of the cost of producing electronic documents goes to review published

Worked out again on each review

Sample size
Coding
Intervals
Disclosure

Coding that is not blindReviewers can see the score the model gave
A statistician on retainerHours each round, billed through to the client

The Solution

Sample sized and drawnFrom the discard pile, with a seed anyone can replay
Coded blind, twiceReviewers see only the document; an agent codes it again
Recall with its intervalA band drawn against the line the protocol sets
Explained, then signedEach number in plain words, citing its term; the partner signs

Built-in controls

The person who draws a sample may not code it
Only the supervising partner signs the disclosure
No client document is used to teach a model

What You Get

Teams stop review at a point they can defend, and serve a disclosure a judge can follow in one sentence.

2 hours: of statistician time per validation round, down from 12–20 our estimate

A band around each estimateExact intervals beside the elusion and recall figures
A sample anyone can replaySeed and population list kept for the other side
The cost of a tighter bandHow many more documents a narrower band would take

From draw to disclosure

Nine screens of TAR validation, each a step the other side can check

Sizing, drawing, blind coding, measuring and disclosing, in the order a validation round runs, each leaving a record opposing counsel can test.

01Earlier rounds
Validation rounds that show when the band's low end clears the line
How it works
- Each round draws 1,537 documents from the discard pile by the same method
- Round 1 put recall at 67.9%, with the whole band below 75%
- Round 2 read 76.4%, but a low end of 71.3% kept review going
- Validations on the firm's other matters sit below, each with its agreed line
Supervising partnerGets a measured reason to stop review, rather than stopping on the day the reviewers run out of documents.
02Sample-size calculator
A sample-size calculator that prices a narrower band before anyone codes
How it works
- Confidence, margin of error and the discard pile give the size: 1,537 at 95% and ±2.5%
- A setting that drops below the protocol's minimum is flagged
- At ±1.5% the sample grows to 4,269 documents, 2,732 more than drawn
- It shows the band that size would give: 79.2% to 84.6%
Review statisticianMeets a demand for a bigger sample with the documents and reviewer-hours it would cost, before the argument starts.
03Sample draw
A seeded draw that returns the same documents every time it runs
How it works
- A funnel from 2,400,000 collected to 312,000 produced and 2,088,000 in the discard pile
- 158 drawn from the reviewed pile and 1,379 from below the cutoff, in proportion
- The seed and a tamper-evident seal on the population list go on the draw record
- Re-drawing from the seed matched 1,537 of 1,537 documents
Review statisticianHands opposing counsel's expert a seed and a list that reproduce the sample, so nobody argues over the draw itself.
04Blind coding
Blind coding that shows the reviewer the document and nothing more
How it works
- Model score, stratum, earlier coding and custodian ranking stay locked
- Relevant, not relevant or ask counsel, measured against the plaintiffs' 14 requests
- The second reviewer agent's call stays hidden until the person codes
- Four reviewers coded the sample in 16.2 reviewer-hours
Contract reviewerCodes each document on its merits, with no score or earlier call to lean on, so the blind-coding term holds if challenged.
05Disagreements settled
A second reviewer agent that sends split calls to a senior attorney
How it works
- An agent codes each sample document again, with its reason and the passage it relied on
- Where the two calls differ, the document goes to the senior review attorney
- Of 23 disagreements, 9 were settled relevant and 14 not relevant
- The log of split calls stays out of the reviewers' view
Senior review attorneySees each split call with both reasons side by side, and records the final call with a reason of her own.
06Recall band
A recall band that narrows as each sample document is coded
How it works
- One tile per sample document, marked not relevant, relevant, settled or still to code
- Recall drawn with its 95% band against the protocol's dashed 75% line
- Once the last document is coded, it settles at 82.0%, between 77.1% and 86.3%
- Elusion of 2.67%, about 55,700 relevant left behind against 253,700 found
eDiscovery managerSees the moment review can stop, with even the low end of the band above the agreed line, and opens the draft straight away.
07Recall explained
Each recall figure explained in one sentence, beside its formula and term
How it works
- Five measures, each with its estimate, exact interval and the protocol term it answers
- A switch turns the plain words into the formulas behind them
- The 41 relevant documents broken out: 9 near-duplicates, 18 marginal, 14 routine, 0 key
- 38 go out in a supplemental volume, and 3 are privileged and logged
Supervising partnerCan put the recall figure to the judge in one sentence, which is her own test before she signs anything.
08Protocol check
A protocol check that matches each validation term to its evidence
How it works
- Each term of the parties' ESI order on its own row, with the record that answers it
- Of eight terms, six met, one met with a note, one waiting for the partner's signature
- The note: 38 documents to produce within seven days, scheduled inside the window
- Clicking a term opens the order's own wording
eDiscovery managerKnows before the draft reaches the partner that each term has evidence behind it, and which one still needs her signature.
09Validation disclosure
A validation disclosure drafted the same day the last document is coded
How it works
- A letter to co-lead counsel, each number a link to its source record
- Sample size, relevant found, elusion, recall, intervals and method, as the order asks
- A rail beside it shows how each term in the letter stands
- Only the supervising partner can sign; the eDiscovery manager routes it to her
Supervising partnerReads a cited draft on the day coding ends, where the letter used to take one to two weeks to agree inside the firm.

Recall of 82.0% from 41 relevant documents in a blind sample of 1,537

The final round on a review of 2,400,000 documents, with 2,088,000 left unproduced.

Measure	Estimate	95% interval
Relevant documents in the sample	41 of 1,537	Counted
Elusion rate	2.67%	1.92% to 3.60%
Relevant documents left behind	≈ 55,700	40,100 to 75,200
Relevant documents found by the review	253,700	Counted
Estimated recall	82.0%	77.1% to 86.3%
Sample for a band of ±3.1 points	About 3,400	Not asked for by the protocol

Around the measurement

More of the TAR validation record the partner and opposing counsel can open

What the team reads around the band, the letter and the draw.

Recall screen scrolled to three plain-words sentences on elusion, recall and the 75% line, each citing its term, beside the disclosure draft notice — **Three sentences under the band**About 3 in every 100 unproduced documents are relevant, the review found about 82 of every 100, and even the low end clears the line.

A single sentence for the judge summing up the elusion test and recall, drafted from the recall and disclosure terms, ready to add to the letter — **One sentence for the judge**Drawn at random, read blind, 41 relevant and none key: recall of 82%, and no less than 77% at 95% confidence. The partner may edit it.

The recall term of the ESI order opened over the protocol check, with its agreed wording and the evidence of 82.0% recall beneath it — **The order's own words**Clicking the recall term opens the wording the parties agreed: recall of at least seventy-five percent, reported with its 95% interval, with the evidence that meets it underneath.

Source record for the recall figure opened from the disclosure letter, giving the formula and how the interval comes from the elusion interval — **Where 82.0% comes from**Each number in the letter opens its record. Recall is 253,700 found over 253,700 plus about 55,700 left behind.

Settings for confidence, margin, interval method and blind coding beside a grid of who may size, draw, code, settle and sign — **Who may size, draw, code, settle and sign**The statistician who draws may not code, reviewers only code, the senior attorney settles and the partner alone signs. Records stay for the matter's life plus 7 years.

Audit trail of the validation round, from the review declared complete through the sizing, the seeded draw, the agent's calls and the settled disagreements — **The validation's audit trail**Who declared review complete, who sized, drew and witnessed the sample, each agent step and the last disagreement settled, each with its time.

Six agents size, draw, code, compute, check and draft; people settle and sign

What each one does, and the limits written into its role.

Sample calculator

Sizes the sample from confidence, margin and the discard pile, and projects the band it will give.

Never shrinks a sample below the protocol's minimum

Sample drawer

Draws a simple random sample with a recorded seed, in proportion to the strata, and seals the population list against later change.

Never redraws without a logged reason
Never sees the coding

Second reviewer

Codes each sample document on its own, with its reason and the passage it relied on.

Never overrides a person's call
Sends each disagreement to the senior review attorney

Recall calculator

Computes elusion, documents left behind and recall, with exact intervals.

Never rounds in the firm's favor
Never reports a number without its interval

Protocol checker

Matches each term of the protocol to the evidence that satisfies it.

Never marks a term met without evidence

Disclosure drafter

Explains each number in plain words and drafts the disclosure, citing the term each number answers.

Never sends or files anything
Never signs

Stopping review, settling split calls and signing stay with the firm's people

Agents run the numbers and write the draft. Four decisions belong to named roles.

Whether review can stopThe supervising partner decides, reading the band against the agreed line.

Which call stands on a split documentThe senior review attorney settles each disagreement and writes down why.

Who codes the sampleBlind reviewers code it, and the statistician who drew it codes none of it.

What reaches opposing counselOnly the supervising partner signs the disclosure, and the agents do not send or file it.

Elusion, recall bands and the TAR validation letter, answered

What is TAR validation?

TAR validation is the proof that a technology-assisted or generative AI review found enough of the relevant documents. This solution draws a blind random sample from the discard pile, measures elusion and recall with exact intervals, checks each protocol term against evidence and drafts a disclosure the supervising partner signs.

What is an elusion test, and how does it give a recall estimate?

An elusion test samples the documents you are not producing and counts the relevant ones. That elusion rate, spread over the discard pile, estimates how many relevant documents were left behind. Recall is what the review found, divided by what it found plus what it left behind.

Why show recall as a band around the estimate?

A sample gives an estimate with uncertainty, and protocols like the one in this matter ask for the recall estimate together with its confidence interval. The band shows where true recall sits at that confidence. When even its low end clears the agreed line, the partner can stop review on firm ground.

Can opposing counsel's expert reproduce the sample?

Yes. The seed and a tamper-evident seal on the population list are kept with the matter, and re-drawing from the seed returns the same documents in the same order. The draw record can be exported for their expert.

Does it work for generative AI review as well as continuous active learning?

Yes. The elusion test samples what the review left behind, whatever chose the documents, so the same method, intervals and protocol check apply to a model's cutoff or a generative AI review.

Who signs the validation disclosure?

The supervising partner. The eDiscovery manager runs the work and can route the draft to her but cannot sign it, and the agents only compute and draft.

eDiscovery & Litigation Support