MLR review automation means taking the mechanical half of medical, legal and regulatory review — opening each cited reference at its page, comparing the claim with the passage, checking safety text against the current label, measuring how prominent risk is beside benefit — and having it done before a reviewer opens the piece. It does not mean automating the decisions. Botminds' MLR Review Automation is built on that split: six agents do the comparing, named reviewers decide each finding and sign, and the filing package is assembled only after the last signature. The practical question for anyone with a budget is how to prove the split works on your own pieces before it touches a live one, and which number you will be asked about in six months.
The short answer to both: pilot on pieces you approved last year, where the right answers are already recorded, and measure four things. Everything below is the long answer.
What can actually be automated in MLR review?
Four kinds of work, and they are the ones that consume a reviewer's week without using their training. The first is finding and anchoring: every claim and statistic on the piece, pinned where it sits, each with the reference it points at. The second is the comparison itself — value, population, comparator, timepoint — done against the cited page rather than the abstract. The third is consistency work that is pure matching: safety text against the label version marked current, approved phrasing and trademarks against the brand guide, required statements and links against the channel's rules. The fourth is bookkeeping: routing a piece by its risk, assembling the Form FDA 2253 package after the signatures, and re-checking every live piece when a label changes.
What cannot be automated is every decision that weighs one thing against another: whether a comparative claim is fair, whether a finding can be dismissed, whether the piece may go out. Those stay with named reviewers who sign, and a programme that blurs the line will not survive a quality review.
Where a piece's review days go, as the composition shifts
Today · 29 days from submission to approval
- Opening every cited reference and checking each claim at its page · 21%
- Comment rounds with the agency · 28%
- Label, fair balance, brand and channel review · 20%
- Waiting for the weekly committee · 21%
- Signatures and the 2253 package · 10%
Target · 13.5 days
- Opening every cited reference and checking each claim at its page · 11%
- Comment rounds with the agency · 37%
- Label, fair balance, brand and channel review · 15%
- Waiting for the weekly committee · 30%
- Signatures and the 2253 package · 7%
That shape is the argument for automating the checking first. The reference work and the label work are the parts a system can finish overnight; the agency's revision cycle and the committee's calendar are not, and a pilot that promises to compress them is promising something it does not control.
What you bring: pieces, references, label versions and approved claims
Five inputs, and the quality of the fourth decides most of the result. The pieces themselves, in the formats you actually use. The references and data on file, annotated and indexed by page, so a claim can be anchored rather than searched for. The current label version and the earlier ones, so a comparison can say which version a piece's safety text came from. The claims you have already approved, each with the reference and page behind it. The brand guide and channel standards.
The page index is the one people underestimate. FDA's own guidance on electronic promotional submissions describes the same artefact from the other end: it asks firms to mark up the material with cross-references to the labelling and the references, and adds flatly that references make FDA's review more efficient (FDA guidance, April 2022, Revision 1). A team that already produces annotated submissions has most of the input; a team that assembles the annotations at submission time will have to move that work earlier. What goes into a Form FDA 2253 submission covers the package itself.
How to pilot MLR review automation: run it on pieces you already approved
A pilot on live pieces measures nothing, because there is no answer to compare against and every disagreement turns into a debate under deadline. A pilot on last year's approved pieces has the answers built in: the findings your reviewers recorded at the time, and the claims they let through.
The pilot in nine steps, none of which needs a live piece
- Choose twenty pieces you approved in the last year
Spread across formats — detail aid, HCP email, deck, broadcast script, congress panel — and across at least two brands. Twenty is enough to see a pattern and small enough to review by hand.
- Pull their references exactly as cited
Publications, study reports and data on file, in the versions that were cited then. Do not tidy the citations: the errors are part of what you are testing.
- Pull the label version in force at the time of each approval
Checking a 2025 piece against a 2026 label will generate findings that are correct and useless. Where you cannot establish the version, set the piece aside and say so.
- Pull the findings your reviewers recorded
These are the answers. Where the record is thin, ask the reviewer who signed to re-read the claims, and note which pieces were scored that way.
- Run the agents blind
No page numbers handed over, no tuning between pieces, nobody helping it find a reference. A piece that needed help is out of the test and recorded as such.
- Compare claim by claim, on the four fields
Value, population, comparator and timepoint — plus whether the analysis type was named right. Mark each as agreement, a finding you would have accepted, or a finding you would have rejected.
- Review every disagreement with the reviewer who signed
Both directions matter. A claim the agents flagged that the reviewer let through is the interesting case, and the one most likely to change a procedure rather than a setting.
- Set the thresholds from what you found
The fast-track and committee scores, the prominence threshold, the signers per brand and which checks run. These are your settings, and each change should be versioned from the first day.
- Write the method down before you report the result
Which pieces, chosen how, measured against what, with the disagreements listed. NIST asks for exactly this of any accuracy claim, and a quality reviewer will ask for it later anyway.
How do you prove MLR review automation worked? Four numbers
Four, and all four can be computed on past pieces before anything goes live. Two are about the checking, two about the process. Keep them separate: the first pair tells you whether the tool is any good, the second whether your process got better, and a programme can move one without the other.
Four measures, how to compute each on pieces you have already approved, and the caveat that goes with it
| Measure | What it is | How to compute it on past pieces | What a weak result looks like, and the caveat |
|---|---|---|---|
| Page hit rate | The share of claims whose cited reference the agents opened at the right page and found the supporting passage. | Count claims with an anchor; count how many resolved to a page where a reviewer agrees the passage sits. | A low rate, and reviewers will go back to opening references by hand. Caveat: it measures your reference index as much as the tool — a reference nobody has indexed cannot be hit. |
| Findings kept | The share of agent findings a reviewer accepted rather than dismissed. | Re-score the pilot findings with the reviewer who signed the piece, and count acceptances. | A low rate means a queue, not a saving. Caveat: a tool tuned to raise fewer findings will score well here and miss more — read it beside the claims it missed entirely. |
| One-round approvals | The share of pieces approved without a second round of comments. | For the pilot pieces, count how many of the recorded rounds the agents' findings would have collapsed into the first. | No movement means the rounds were caused by something else — usually the agency's draft or an unsettled claim. Caveat: this is a judgement on past pieces, not a measurement. |
| Median days to approval | Median elapsed days from submission to final signature, by lane. | Measure it on the last twelve months first. Without a baseline the number after go-live means nothing. | A mean instead of a median will be dragged by two stuck pieces. Caveat: do not compare your number with an industry figure — the figures in circulation rarely say where their clock starts and stops. |
The four numbers, and what the agents found behind them

How do you measure MLR review performance today?
Many teams discover during a pilot that they cannot, and that is a finding worth having early. Three definitions have to be settled before any number means anything.
When does the clock start? At the agency's upload, at the first reviewer's assignment, or at the moment the piece was complete enough to review? Pieces that arrive without references and wait are the ones that distort a median, and whichever rule you choose has to be applied to the baseline too. When does it stop? At the last signature, at approval in the system, or at the 2253 filing? And what is a round? A revision requested and returned is uncontroversial; a reviewer's question answered in a comment is not.
Two more counts are worth keeping from day one and cost almost nothing. Findings by origin — claim, reference anchor, label, prominence, brand, channel — because the mix tells you which checks are earning their place. And pieces routed to committee that generated no judgement call, because that is committee time with nothing to decide.
Findings by type across one month, which is where the mix tells you what to fix
The volume argument, in public numbers
The reason to do any of this is not promotional, and it is published. Over twenty years the amount of promotional material companies filed with FDA roughly tripled while the agency's violation letters fell by more than an order of magnitude.
The checking load grew while the external check shrank
97,252
promotional materials submitted for FDA review in 2016, against 34,182 in 1997
11
FDA violation letters for misleading drug marketing in 2016, against 156 in 1997
$20.3B
spent marketing to health care professionals in 2016, up from $15.6 billion in 1997
0
warning letters FDA sent to pharmaceutical companies in 2024, by its own count
Then the external check came back. On 9 September 2025 FDA announced thousands of letters warning companies to remove misleading ads, approximately 100 cease-and-desist letters and rulemaking to close the 1997 "adequate provision" allowance, and said it was already deploying automated tools to surveil and review drug advertisements. It is worth being clear about what has not changed: FDA does not review promotional material before use, except in rare instances such as a compliance action or an accelerated-approval product under the presubmission requirement (OPDP frequently asked questions). The piece goes out, and the specimen is filed at the time of initial dissemination under 21 CFR 314.81(b)(3)(i). Whoever opens the cited page, it is somebody inside the company. What FDA keeps citing in its letters reads the recent ones as a check list.
The quality of what has reached print is also published, and it is sobering rather than flattering: in a November 2008 sample of US physician-directed journal advertisements, "Few physician-directed print pharmaceutical advertisements adhere to all FDA guidelines; over half fail to quantify serious risks" (Korenstein et al., PLoS ONE, 2011). That is the baseline a programme is improving on.
What a round of review costs, and what a day of delay costs
Neither figure can be looked up, and a business case that quotes an industry average for either is quoting a number whose method it cannot see. Both can be computed from numbers you already hold, and the arithmetic is short enough to do in a meeting.
A round costs the reviewer hours it consumes plus the agency's revision cycle. Take your own three inputs: the hours each reviewer spends on a round of one piece, your loaded hourly cost, and the number of reviewers on the lane. Multiply, then add what the agency bills for a revision cycle and the internal time spent instructing it. Multiply by the rounds your pieces actually take — which is a number your system can give you for the last year, and usually the first surprise of the exercise.
A day of delay costs whatever the piece was going to earn that day, which only the brand can tell you, and it is zero for a piece with no date attached and large for one tied to a congress, a launch or a label change. The useful move is not to put a figure on an average day but to separate the pieces where a day matters from the ones where it does not, and route accordingly. That is what a risk tier is for.
Go live by lane, not by brand
The instinct is to start with one brand and do everything for it. The better order is to start with one lane and do it for everything, because a lane is defined by how much judgement a piece needs, and the low-judgement lane is where a mistake is cheapest and the evidence arrives fastest.
Twelve weeks, opening one lane at a time
- Back-test on approved pieceswk 1–3
- Thresholds and signerswk 4–5
- Livewk 6–12
- Back-test on approved pieceswk 1–3
- Thresholds and signerswk 4–5
- Watching the fast-track lanewk 6–8
- Livewk 9–12
- Back-test on approved pieceswk 1–3
- Thresholds and signerswk 4–5
- Watching the other laneswk 6–10
- Livewk 11–12
Two rules keep the sequence honest. Nothing goes live on a lane whose pieces were not in the back-test — a checker that has never seen a broadcast script should not be in front of one. And no lane opens without its own thresholds set and versioned, because a threshold changed informally rewrites the history of every decision made under it.
The settings a team owns: routing, signers and which checks run

What breaks after MLR automation goes live, in month two
Four things often break, and all four are cheaper to plan for than to discover.
The reference index goes stale first. New publications arrive, a study report is superseded, a market-research subscription renews with a new document number, and claims start failing to anchor. The tell is the page hit rate falling while the finding counts stay flat. Somebody has to own the index, and in month one nobody does.
The approved claims library turns out to be a wish list. Teams often discover that some of the claims in it were approved years ago against a reference nobody has re-read, and that some are tied to a label version that has moved. The first label change after go-live surfaces all of it at once — which is uncomfortable and is also the point.
The committee agenda empties and nobody reduces the meeting. A routing rule that sends only pieces with open judgement calls to committee will leave a weekly meeting with three items and an hour booked. The meeting then fills with the pieces it used to handle, and the measured gain disappears into the calendar.
And the agency's drafts get worse before they get better, because the feedback loop has changed: instead of a reviewer's comment, the agency receives a finding naming the page and the mismatch. That is more useful and less familiar, and it takes a cycle or two before revisions come back addressing the finding rather than rewording around it. The MLR review process, stage by stage, sets out the sequence the agency is working inside, and which of three products you need covers the choice that comes before the pilot. To see the checking on your own past pieces, start at MLR Review Automation.
Questions people ask
What data do you need to start MLR review automation?
Your pieces in the formats you use; the references and data on file, annotated and indexed by page; the current and earlier label versions; the claims you have already approved, each with its reference and page; and the brand guide and channel standards. For a pilot, add the findings your reviewers recorded on twenty past pieces — those are the answers you will measure against, and without them a pilot produces opinions rather than numbers.
How long does it take to implement MLR review automation?
Plan in lanes rather than in months. A realistic sequence is three weeks of back-testing on approved pieces, two weeks setting thresholds, signers and which checks run, then the fast-track lane live, the standard lane a few weeks later and the full-committee lane last. Twelve weeks is a reasonable plan for all three; the variable is usually how long it takes to assemble label versions and a reference index, not the software.
Does MLR automation reduce review cycles?
It can reduce the rounds caused by findings that arrive late — a subgroup presented as the study result, safety text from a superseded label, a claim the cited page does not make — because those findings now arrive in the first round with the page attached. It does not compress the parts that depend on other people: the agency's revision cycle and the committee's calendar. Measure the two separately or the result will be unreadable.
What are good MLR review KPIs?
Four: the share of claims whose cited page was found, the share of agent findings reviewers accepted, the share of pieces approved in one round, and median days from submission to approval by lane. Add two counts that cost nothing — findings by origin, and pieces routed to committee that produced no judgement call. Define when the clock starts and stops before you measure anything, and apply the same definition to the baseline.
How do you prove MLR review automation worked?
By having measured the same four numbers on the twelve months before it, with the same definitions, and by keeping the method written down: which pieces, chosen how, scored against what, with the disagreements listed. NIST's framework asks for exactly that of any accuracy claim, and a quality reviewer will ask for it whether or not NIST does. A result without a baseline is not evidence.
What breaks after MLR automation goes live?
The reference index goes stale and claims stop anchoring; the approved claims library turns out to contain claims nobody has re-read; the committee meeting keeps its hour after its agenda empties; and the agency's revisions take a cycle or two to start addressing findings rather than rewording around them. Each has an owner-shaped hole in it, which is the real month-two task.