WorkflowMedical Affairs

Buying · three shapes, one name

MLR review software: which of three products do you need?

Three quite different products answer to the name MLR review software, and they are layers rather than competitors: one stores the piece and routes it, one proofreads it against a style guide, one opens each cited reference at its page. This guide sets out what separates them, twelve questions that expose the difference in an hour on your own pieces, and where MLR Review Automation sits.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

MLR review software is one name over three quite different products, which is why two demos can sound identical and differ a great deal in what they cost and what they do. One stores the piece, versions it, routes it to reviewers and records their approvals. One proofreads it against a style guide, an approved-claims list and the brand's trademark rules. One opens each cited reference at its anchored page and compares the claim with the passage the claim rests on. They are layers rather than rivals, and the question is not which is best but which of your problems you are buying for. Botminds' MLR Review Automation is the third shape with the first built in: every claim checked at its page, against the current label and against a measured reading of prominence, then routed by risk to named reviewers who decide and sign.

This page names no product and ranks none. It describes the three shapes, the questions that separate them, and what each can honestly answer — so that a demo becomes a test rather than a presentation.

What does MLR review software do?

At minimum it gives a promotional piece a single place to live while medical, legal and regulatory reviewers look at it. That means a version history, a route, a set of comments or findings, a record of who approved what, and — for the United States — a package for the Form FDA 2253 submission at first use. Everything beyond that is the interesting part, and it is where products diverge.

The work the reviewers actually do is comparison. A medical reviewer compares each claim with the study behind it. A regulatory reviewer compares the safety information with the current Prescribing Information and judges whether risk is as prominent as benefit. A legal reviewer compares a comparative claim with what the data can carry. None of that is routing, and a product that only routes will make the queue faster while leaving every comparison exactly where it was.

Three products, one name: store and route, proofread, or open the source

The cleanest way to tell the shapes apart is to ask what each one has read. The first has read the piece's metadata. The second has read the piece. The third has read the piece and the documents it cites.

Ten questions, and which product shape can honestly answer each

The question a reviewer asksStore and routeProofreadOpen the source
Where is this piece, and who is holding it up?Yes — this is what it is for.No.Yes.
Who has signed, and what did the signature mean?Yes, if it records the meaning and not only the act.No.Yes.
Does the wording match our approved phrasing and trademark rules?No.Yes — its core strength.Yes.
Are the required statements and channel links present?Sometimes, as a manual check item.Yes.Yes.
Does the cited page actually say what the claim says?No.No — it has not opened the reference.Yes, with the page and the passage.
Was the result a primary, key secondary or exploratory endpoint?No.No.Yes — and it decides what the claim may be called.
Is the safety information the current label's, word for word?No.Only if the approved block is already in its list.Yes, against the version marked current.
How prominent is risk next to benefit, as a measurement?No.Partly — it can count words and check type size.Yes — words, type size, contrast, placement, and words a second for audio.
Which live pieces does this label change affect?Partly: it knows what is live, not what is now wrong.No.Yes — it re-checks each live piece against the new version.
Is the 2253 package built, and was it built after the signatures?Yes, usually on approval.No.Yes, and only after every required signature.
Our reading of the three shapes, written so that each row can be tested in a demo rather than taken on trust. No product is named: a given vendor may sit across two columns, and the point of the table is to find out which.

The shapes also fail differently, and that is the more useful thing to know before buying. A store-and-route product fails quietly: nothing is wrong, the queue moves, and the findings that were missed before are missed faster. A proofreading product fails confidently: it marks a claim consistent with the approved list when the approved list itself was built on a reference nobody re-read. A product that opens the source fails visibly — it points at a page, and a reviewer can see within seconds that the page does not say what it claims. Visible failure is the one to prefer.

How do you evaluate MLR review software?

On your own pieces, in an hour, with the references you actually cite. Every question below is answerable in a live demo, and none requires a pilot or a data migration. The two answers given for each are the ones to listen for: the one that would satisfy you, and the one that should not.

Twelve questions to put to any MLR review tool, on your own pieces

  1. Show me the page behind this claim

    Satisfying: the cited document opens at its page with the passage highlighted, beside a field-by-field comparison. Not: a reference list, or a confidence score with no page to look at.

  2. Show me a claim you marked supported that a reviewer would not

    Satisfying: it has one, and can say why it got it wrong. Not: a claim that it never gets anything wrong.

  3. Show me what you did when our label changed last month

    Satisfying: a list of live, approved pieces carrying the superseded text, each with the label section and a proposed revise or withdraw. Not: a notification that the label changed.

  4. Show me the measurement behind the words fair balance, not the verdict

    Satisfying: share of words, type size, contrast, placement, and words a second for audio, each as a number against your threshold. Not: a green tick, or a score with no components.

  5. Show me what you refuse to decide

    Satisfying: a stated list — comparative wording, whether a finding may be dismissed, approval itself. Not: a claim that it decides everything, or an answer that it depends on configuration.

  6. Show me the filing package, and when it was built

    Satisfying: built after the last required signature, containing the completed form, the final piece clean and annotated, the current Prescribing Information and the annotated references. Not: a package assembled at submission from whatever was current then.

  7. Show me a signature, and what it means

    Satisfying: the printed name of the signer, the date and time, and the meaning — review, approval, responsibility or authorship — as 21 CFR 11.50 requires. Not: a user id and a timestamp.

  8. Show me a signature moved onto a different version

    Satisfying: it cannot be done. 21 CFR 11.70 requires signatures to be linked to their records so they cannot be excised, copied or transferred. Not: a demonstration that it can, with an audit entry to show for it.

  9. Show me a dismissed high finding, and the reason

    Satisfying: the finding, who dismissed it, when, and a written reason that was required before the dismissal took effect. Not: a dismissal with a blank reason field.

  10. Show me a claim resting on an exploratory endpoint, and what you called it

    Satisfying: it names the analysis and refuses to record the result as a supported efficacy claim. Not: supported, because the number on the page matched.

  11. Show me the same piece after the agency sends a revision

    Satisfying: it re-checks the changed claims and keeps the decided findings decided. Not: a fresh start that throws away last week's reviewing.

  12. Show me what you need from us before any of this works

    Satisfying: a specific list — pieces, annotated references indexed by page, label versions, the approved claims, the brand guide. Not: nothing, we learn from your content.

The questions are ours. The two regulatory ones quote 21 CFR 11.50 and 11.70 at the current eCFR text; the filing question follows 21 CFR 314.81(b)(3)(i).Source: 21 CFR 11.50 (eCFR); 21 CFR 11.70 (eCFR)

Can it check a claim against its reference, at the page?

This is the question that separates the third shape from the other two, and it has an external benchmark. FDA's own guidance on electronic promotional submissions asks firms to make exactly this link when they submit annotated material: where the labelling or a reference supports a claim, the annotated piece should carry a hypertext link to the specific page that holds the support (FDA guidance, April 2022, Revision 1).

Read that in a demo and watch what happens. The agency is describing a claim-to-page link as a normal output of promotional review — not an advanced feature, but the thing the annotated copy is supposed to contain. A tool that cannot produce the link cannot produce the annotated submission either, which means somebody will build it by hand after approval.

Asking for the page also exposes the quiet failure mode of a checker that does not open anything. In a published audit of 1999 US journal advertisements, only 20% of data-on-file references could be obtained at all, through library and internet searches or by asking the manufacturer. A reference list is not evidence that anybody has read a reference. What data on file is, and what a claim can rest on, works through the audits.

What it has to refuse to decide

The most informative answer in a demo is to the question of what the product will not conclude. FDA's guidance on communications consistent with the labeling recommends that qualified medical, scientific, legal and regulatory personnel review a communication before it is disseminated, to make sure it is not false or misleading (FDA, June 2018). That recommendation is about people, and nothing in a tool displaces it. A product that will decide whether a comparative claim is fair is a product whose decisions a named reviewer still has to own — which means the reviewer has to re-do the work in order to own it.

One piece through the checks: what each step needs, and where the decisions stay

  1. Agent

    The piece and its references arrive

    The piece in its own format, with the references and data on file indexed by page, the label versions, the approved claims and the brand guide.

    Never: treats an unindexed reference as one it has checked

  2. Agent

    Every claim pinned

    Each claim, statistic and reference anchor found and pinned where it sits, with its value, comparator, endpoint and timepoint.

    Never: paraphrases a claim — it is quoted as written

  3. Agent

    Each cited page opened

    The reference opened at its anchored page, the passage found, and value, population, comparator and timepoint compared.

    Never: records an exploratory or subgroup result as supported efficacy

  4. Agent

    The current label compared

    Claims and safety information checked word for word against the label version marked current, naming the section and any missing warning.

    Never: uses a superseded label version

  5. Agent

    Prominence measured

    Share of words, type size, contrast, placement and, for audio, words a second, each against the team's threshold.

    Never: passes or fails a piece on a reading

  6. A person decides

    The comparative claim decided

    A named legal reviewer decides whether the wording is fair, with the source open at its page.

  7. A person decides

    Findings decided, then signed

    Each finding accepted, sent back or dismissed with a written reason; medical, regulatory and legal sign, each signature carrying its meaning.

    Never: a package built before every required signature

How MLR Review Automation divides the work. The green steps are the two that stay with people by design; the stated limits under each agent step are the product's own.

The label change test, and what it reveals

Ask what happened the last time a label changed. It is the question that best distinguishes a tool that thinks in pieces from one that thinks in claims, because a label change is retrospective: it does not affect the piece in review, it affects everything already approved and out in the field.

A store-and-route product knows which pieces are live. A proofreading product knows what the approved wording is today. Neither can tell you which live pieces carry a safety block that was correct in March and is wrong now, because neither holds the link from a piece to the label version its text came from. A tool that keeps that link can produce the list the same afternoon and propose, for each piece, a revision or a withdrawal for a person to confirm.

The cost of not having it is not theoretical. Promotional pieces outlive the labels they were approved against; a warning added to the Highlights of Prescribing Information stays listed as a recent major change for at least a year, while the pieces approved against the old text may stay in use far longer, and the sweep that follows a label change is retrospective work no routing system was built for.

Fair balance: ask for the measurement, not the verdict

Fair balance is where a demo most often substitutes a tick for an argument. The regulation does not give a number: 21 CFR 202.1(e)(7)(viii) asks for risk to be about as prominent and readable as the benefit, judged by the type, layout, contrast, headlines, paragraphing and white space the piece uses. Those named factors are measurable one by one, and the measurements are what a reviewer can argue with.

So ask for the components. On one piece: what share of the words are risk, what size is the risk type against the benefit headline, what contrast, where on the page, is the safety text the current one, and is there a direction to the full Prescribing Information? Six readings a reviewer can dispute are worth more than one verdict they cannot. A product that answers with a score and no components has made a judgement it is not entitled to make, and has hidden the evidence a reviewer needs in order to make it. How fair balance is measured, factor by factor, sets out the six readings.

The queue a store-and-route product solves, with the findings a checker adds to it

A review queue screen. At the top, two pieces that have arrived and not been checked, each with a button reading 'Pin and check claims'. Below, a table of pieces in review: each row gives the piece name and id, the brand, the format, the risk tier — full committee or standard — the stage shown as a progress bar, the number of open findings in red and amber, three signature avatars showing who has signed and who is waiting, and a due date. One row reads 'Remission at 12 weeks and at 1 year, full committee, reviewer check, 4 red and 6 amber findings, due Oct 8'.
MLR Review Automation on sample data. Every brand, person and piece named here is fictional. The queue columns are what the first product shape gives you; the two numbers in the open-findings column are what the third adds.

The filing, and whether the package is built after the signatures

In the United States the review ends in a submission, and the regulation is specific about its contents: each submission goes with a completed Form FDA 2253 and a copy of the product's current professional labelling (21 CFR 314.81(b)(3)(i)). The piece goes in at the time of initial dissemination, not later.

Two details in a demo are worth more than the feature list. First, when is the package built? A package assembled on approval, from the versions that were current at the moment the last signature landed, is a true record of what was approved. A package assembled at submission time, from whatever is current then, is a reconstruction. Second, can the package be built before every required signature is in? If it can, it will be, on the day someone is on holiday.

What you need to give an MLR review tool before it can work

Every shape needs something, and the third needs the most. The honest vendor answer is a list; the answer to distrust is that no preparation is needed. The list is short and most of it already exists somewhere:

The queue problem and the checking problem are not the same purchase

You have a queue problemYou have a checking problem
What it looks like

You have a queue problem: Pieces wait. Nobody can say where a piece is or who is holding it. The committee agenda is long and most of it is routine.

You have a checking problem: Pieces move, but findings keep arriving late: a subgroup presented as the study result, safety text from an old label, a claim the cited page does not make.

Where a reviewer's week goes

You have a queue problem: Chasing status, re-reading comment threads, attending a meeting for pieces that needed one signature.

You have a checking problem: Opening references by hand — the number, the population, the endpoint and the timepoint, fourteen times for one detail aid.

What to buy

You have a queue problem: A system that stores, versions, routes and records approvals, and builds the filing package.

You have a checking problem: A checker that opens each cited reference at its page, compares the claim with the passage, and measures prominence.

If you buy the other one

You have a queue problem: You get faster routing and the same findings, arriving at the same point in the process.

You have a checking problem: You get better findings with nowhere to put them, and reviewers reconciling two systems.

How you would know it worked

You have a queue problem: Median days from submission to approval, and the share of pieces that need a meeting at all.

You have a checking problem: The share of claims whose cited page was found, the share of findings reviewers kept, and the share of pieces approved in one round.

Many teams have both problems and buy for only one of them. The useful order is to name which one is costing more this year, because the two are measured by different numbers.

What a checking product needs from you, specifically: the pieces themselves in the formats you actually use; the references and data on file, annotated and indexed by page, so a claim can be anchored rather than searched for; the current label version and the earlier ones, so a comparison can say which version a piece's safety text came from; the claims you have already approved, with the reference and page behind each; and the brand guide and channel standards. Nothing on that list is exotic. All of it is work, and a vendor who says otherwise has moved the work to you without saying so.

How you would know whether it is accurate

We know of no independent, published accuracy figure for any product of this kind, and a vendor who offers one should be asked what it was measured against. What exists is a human baseline from the peer-reviewed audits of journal advertising, and it is a low bar that a checker ought to clear.

The baseline any claims checker has to beat

18.1%

of US physician-directed journal advertisements adhered to all FDA guidelines, in a November 2008 sample

48.2%

lacked verifiable references — no references at all, or only data on file or the package insert

57.8%

did not quantify serious risks

20%

of data-on-file references in a 1999 advertisement sample could be obtained at all when investigators tried

First three: Korenstein D, Keyhani S, Mendelson A, Ross JS, PLoS ONE 2011;6(8):e23336, on 83 unique full advertisements in high-impact US biomedical journals, November 2008 — 'Content important for safe prescribing was often incomplete.' Fourth: Cooper RJ and Schriger DL, CMAJ 2005;172(4):487-91. These measure what reached print after human review; they are the comparison, not a claim about any product.Source: Korenstein et al., PLoS ONE, 2011; Cooper and Schriger, CMAJ, 2005

The way to measure a candidate against that is a back-test rather than a demonstration: take twenty pieces you approved last year, with their references and the findings you recorded at the time, run the tool over them without telling it the answers, and compare claim by claim. Four numbers come out of it — the share of claims whose cited page the tool found, the share of its findings a reviewer accepted, the share of pieces that would have been approved in one round, and the median days the lane would have taken. How to pilot MLR review automation, and what to measure, sets the exercise out step by step.

What a claim-to-page comparison looks like when the reviewer opens it

A four-page detail aid with fourteen pinned claims. The open pin reads 'two out of three comes from a descriptive subgroup, overstated'. Beneath it the piece's sentence is quoted, then the cited maintenance study is shown open at page 9 with the relevant passage highlighted, including the line that in the subgroup of patients already in remission at week 12, 141 patients, 67 per cent remained in remission at week 52, and that this subgroup analysis was descriptive. A four-row table compares the piece with the source on value, population, analysis and timepoint, ticking value and timepoint and crossing population and analysis, and a note points to the label's maintenance section.
MLR Review Automation on sample data; every brand, study and person named is fictional. This is the view to ask for in any demo: the claim, the source at its page, and the four fields compared — not a score.

Whichever shape a team buys, the division of labour that should survive the purchase is the one in the middle of this page: the comparing is mechanical and can be done before a reviewer opens the piece; the judging is not, and stays with named people who sign. What an agent may decide, and what it must not, takes that line and tests it against the regulation. To see the checking on your own pieces, start at MLR Review Automation.

Questions people ask

What is the difference between a content management system and MLR review software?

A content system stores the piece, versions it, routes it and records approvals — it is the first of the three shapes and it solves the queue. MLR review software in the fuller sense adds the comparisons the reviewers make: the cited reference opened at its page, the safety information compared word for word with the current label, prominence measured against the factors the regulation names. Many teams own the first and call it the second.

Does MLR software replace the reviewers?

No, and the FDA guidance on communications consistent with the labeling is the reason to say so plainly: it recommends that qualified medical, scientific, legal and regulatory personnel carefully review a communication before it is disseminated. What software can take away is the comparing — fourteen references opened by hand for one detail aid — so that the reviewing is about the findings that need judgement. The signature, and what it means, stays with a person.

Does MLR software build the FDA 2253 package?

A product of the first or third shape usually does; a proofreading tool does not. The package has to contain the completed Form FDA 2253, the final piece clean and annotated, a copy of the product's current professional labeling and the annotated references. The question to ask is not whether it builds the package but when: a package built after the last required signature records what was approved, while one assembled at submission records what was current that day.

How do you know if MLR review software is accurate?

Back-test it on pieces you have already approved. Take twenty with their references and the findings recorded at the time, run the tool blind, and compare claim by claim with the reviewer who signed. Measure the share of claims whose cited page it found, the share of its findings a reviewer accepted, and the disagreements in both directions. We know of no independent, published accuracy figure for this class of product, so a number offered without a stated test set means very little.

What should an MLR review tool never do?

It should never approve a piece, never sign, never record an exploratory or subgroup result as a supported efficacy claim, never pass or fail a piece on a prominence reading alone, never invent a claim in place of one it has rejected, and never build a filing package before every required signature is in. A product that will not tell you its refusals in a demo has not thought about them.

What do you need to give an MLR review tool before it can work?

Your pieces in the formats you use; the references and data on file, annotated and indexed by page; the current and earlier label versions; the claims you have already approved, each with its reference and page; and the brand guide and channel standards. Routing thresholds, the signers for each brand and which checks run should be settings you own, versioned and on the audit trail, rather than a configuration someone else holds.

MLR Review Automation.
See it on your own files.

An agentic solution from Botminds. Every claim in a promotional piece pinned on the piece itself and checked against the cited reference page and the current label, so medical, legal and regulatory reviewers start from the evidence. Book a demo, or explore how it works.