WorkflowClinical Development

Choosing · the evidence

AI agents in clinical trials: what the published evidence shows

The clearest published head-to-head of a language model against expert reviewers on real protocol deviations does not say the model is better. It says the model disagreed with the experts 30% of the time — against a ground truth the two experts themselves disagreed on 20% of the time, in two minutes instead of a day and a half. This guide reads that result honestly, sets it against what FDA and EMA now ask of a model like this, and shows where Protocol Deviation Management stops and a person decides.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

AI agents in clinical trials are programs that carry out a defined piece of trial work end to end — read this document, classify this event against this specification, check it against earlier decisions, flag what it implies — and hand the result to a named person rather than acting on it. The honest case for them on protocol deviations is not accuracy. A peer-reviewed head-to-head on real deviations found a large language model disagreeing with an expert-adjudicated ground truth 30% of the time; the same paper found the two experts disagreeing with each other 20% of the time, and each expert disagreeing with themselves on 5–10% of repeats. What the model did was process the same 309 records in about two minutes, against roughly 1.5 cumulative working days for the experts. That is a different claim from being better, and it is the one both FDA and EMA now ask sponsors to evidence — as a human-AI team rather than as a model on its own. Botminds' Protocol Deviation Management is built to that shape: six agents that read, classify, compare, assess, watch and list, each with a stated limit, and a person who signs.

The worked screens below come from the solution's sample study — Northwind Bio's VLT-302, Veltrimab in plaque psoriasis, Phase 3, 64 sites and 612 participants, run partly through the contract research organisation Ashgrove Research. The sponsor, the drug, the sites and the people are fictional characters in that sample data. The published evidence is linked where it is used.

What are AI agents in clinical trials?

The word "agent" is doing real work in that phrase and is worth pinning down, because it is used for three different things. A model is a thing that maps an input to an output. An assistant is a model wrapped in a conversation. An agent, as the term is used in trial systems, is a model given one bounded job, a defined set of inputs it may read, a defined output it must produce, and limits on what it may not do — and then run as part of a process rather than a conversation.

The distinction matters at an inspection. A conversational assistant that a monitor consults leaves no reliable trace of what it was asked or what it said. An agent with a bounded job produces an auditable step: this input, this output, this confidence, this citation, at this time. The six agents on the sample study are an example of the shape — a deviation reader that writes one clean description from the trial systems and never edits a source system; a classifier that must cite a specification row and the protocol section breached and must answer both importance questions; a precedent finder that counts only signed decisions; an impact assessor that flags per-protocol consequences and possible serious breaches but removes nobody from an analysis set; a trend watcher whose alerts propose actions; and a listing writer that uses confirmed decisions only.

Every one of those limits is a design choice rather than a technical one, and they are the part of a system that an evaluator should read first. Where deviations fit in a risk based quality management stack covers the surrounding layers.

A published head-to-head on protocol deviations

Published evidence on language models applied to protocol deviations is thin, and one peer-reviewed study stands out because it ran the comparison properly and reported the uncomfortable number. Zou, Popko and Gaudio described using a large language model with a tailored prompt to label free-text protocol deviations from a sponsor's own deviation management system, published in Therapeutic Innovation and Regulatory Science in May 2025.

The set-up is worth stating precisely, because the task is narrower than general importance classification. The team selected Phase 2 and Phase 3 solid-tumour studies using progression-free survival or objective response rate as efficacy endpoints under RECIST 1.1 — sixteen studies in all — and built an evaluation set of 309 protocol deviations, about 210 drawn at random and about 110 containing keywords related to progression. The question put to both the experts and the model was whether each deviation was relevant to the assessment of disease progression. Two senior quality leads reviewed every record independently; their differences were reconciled, and the adjudicated output became the ground truth. Their labels split 44 as affecting disease progression, 60 as possibly affecting it and 205 as not.

One model against two experts, on 309 real protocol deviations

20%

disagreement between the two experts before reconciliation

5–10%

intra-rater inconsistency: the same expert, the same record, a different answer

30%

disagreement between the model and the reconciled ground truth

80%+

of deviations potentially affecting the progression assessment flagged for expert review

Zou M, Popko L, Gaudio M, 'Using Large Language Models for Advanced and Flexible Labelling of Protocol Deviations in Clinical Development', Therapeutic Innovation and Regulatory Science 2025;59(4):833–847, published 13 May 2025. The evaluation set was 309 protocol deviations from 16 Phase 2 and Phase 3 solid-tumour studies; the labelling task was relevance to the disease progression assessment, not general importance.Source: Zou, Popko and Gaudio, 2025 (PubMed Central)

The authors' own summary of the trade-off is the most useful passage in the literature for anyone evaluating this category: "In our experiment, establishing a baseline ground truth for 309 PDs required two experts approximately 1.5 cumulative working days. … In contrast, our automated approach processed a single prompt version with one parameter combination in approximately two minutes, yielding a 30% disagreement compared to the ground truth" (Zou et al., 2025, Discussion).

The 20% nobody mentions: expert against expert

The 30% is the number a sceptic quotes. The number that changes how it should be read is in the sentence beside it: "The reconciliation process revealed a 20% disagreement between experts and 5–10% intra-rater inconsistency" (Zou et al., 2025).

Two senior quality leads, reading the same free-text deviations against the same question, disagreed on one in five. Each of them, re-reading records they had already labelled, changed their own answer between one time in twenty and one in ten. That is not a criticism of the experts. It is what judgement on short free-text descriptions looks like when it is measured, and it is almost never measured — which is why "the medical monitor decides" tends to be treated as a fixed point rather than as a process with its own error rate.

Once the human baseline is visible, the model's 30% stops being a verdict and becomes a comparison. The ground truth it was scored against was built by reconciling a disagreement that was itself one in five. A model at 30% against a consensus that the two experts only reached after disagreeing on one record in five is not an accurate classifier, and it is not a useless one either. It is a candidate first pass.

The same 309 records, two ways

Two senior quality leadsOne prompt version, one parameter combination
Time taken

Two senior quality leads: About 1.5 cumulative working days to establish the ground truth, including reconciliation.

One prompt version, one parameter combination: About two minutes to process all 309 records.

Agreement with the final label

Two senior quality leads: The experts defined it — but disagreed with each other 20% of the time before reconciling.

One prompt version, one parameter combination: 30% disagreement with the reconciled ground truth.

Repeatability

Two senior quality leads: 5–10% intra-rater inconsistency: the same person, the same record, a different answer on a second pass.

One prompt version, one parameter combination: Not reported. The authors tuned the sampling settings toward catching more records, which trades some repeatability for recall.

What it costs to redo

Two senior quality leads: Another day and a half, and the experts’ attention is finite.

One prompt version, one parameter combination: Another two minutes, which makes prompt variants and parameter sweeps practical to test.

What it is good for

Two senior quality leads: Settling the hard cases, and defining what the right answer is.

One prompt version, one parameter combination: Reading everything, flagging what needs the experts, and never getting tired at record 280.

What it cannot do

Two senior quality leads: Scale, or stay consistent across sixteen studies and several years.

One prompt version, one parameter combination: Be the answer. The paper’s framing is a flag "enabling expert review", not a decision.

Both columns from Zou, Popko and Gaudio, 2025, on the same evaluation set of 309 protocol deviations. The reading in the last three rows is ours.Source: Zou et al., Therapeutic Innovation and Regulatory Science, 2025

How accurate is AI at classifying protocol deviations?

On the clearest published measurement, 30% disagreement with an expert-adjudicated ground truth on a narrow labelling question, with the model's recall tuned high enough that it "flagged over 80% of PDs potentially affecting disease progression assessment, enabling expert review". That phrasing is deliberate and worth copying: a system tuned to catch what matters will over-flag, and over-flagging is the correct failure direction when a person reviews everything that is flagged.

Four caveats travel with that number and should travel with any vendor's. It is one study, one sponsor's deviation text, one model family, one prompt. The task is relevance to a disease-progression assessment in oncology, not the general importance question ICH E3 poses — a narrower judgement, and arguably an easier one, since it keys on a specific endpoint. The evaluation set was deliberately enriched with progression-related records, so it is harder than a random sample in some respects and unrepresentative in others. And the comparison is against a two-expert consensus, not against truth, because on this kind of question no truth exists independent of expert judgement.

What the published record lacks, and what a buyer should therefore ask any vendor for directly, is a measurement on the general importance question against a sponsor's own signed decisions. What makes a protocol deviation important, and the two tests it turns on sets out the question that measurement would have to be made against.

Consistency and speed are the claim; accuracy is not

If accuracy is not the claim, what is? Three things, and all three are measurable.

A repeatable answer, where it is set up to be one. A fixed prompt run with low-randomness settings tends to return the same label for the same text, which is precisely what the 5–10% intra-rater figure says a human reviewer does not do. It is a property to test on your own deviations rather than assume: the paper tuned its settings toward recall, not repeatability. On a study classified by three different teams over three years, consistency is not a nicety; it is the property that makes a by-site comparison mean anything at all.

Everything read, not a sample. Two minutes for 309 records is 3,000 records in twenty minutes. The practical consequence is not cost saving but coverage: a first pass over every deviation rather than over the ones that reached a meeting agenda.

A citation attached. A human classifier's reasoning is usually unrecorded; a system can be required to produce the specification row and the protocol section with every proposal, which turns an opinion into something a second person can check in thirty seconds. On the sample study that requirement is a stated limit on the classifier rather than a feature of it.

One number from the sample data should not be read alongside the published 30%, and it is worth saying why explicitly. The solution carries a design target of 96.2% agreement with the medical monitor's final call. That is a target set on one study's sample data, not a measured production result, and the 30% is a published result on 309 real records under a different question. Putting the two in one sentence would compare a goal with a measurement, which is the error this whole guide is written against.

Each Protocol Deviation Management agent's step on one deviation, with its limits

The agents' run on one deviation: the deviation reader has read the monitoring visit report for participant 114-0087; the classifier has matched specification row EA-03, cited protocol section 8.1.2 and returned a confidence of 0.94; the precedent finder has found 3 signed decisions on similar deviations; the deviation has gone to the medical monitor because the contract research organisation proposed not important; and the impact assessor has flagged the participant for the Week 16 analyses. Each step shows its elapsed time and what it found.

The classifier must answer both importance questions, and below 0.90 confidence a person decides.

Protocol Deviation Management on the fictional sample study VLT-302. The classifier's stated limits are visible in what it produced: a specification row, a protocol section, both importance questions answered, and a confidence — below 0.90 a person decides instead.

What FDA's credibility framework asks of a model like this

FDA's January 2025 draft guidance on artificial intelligence to support regulatory decision-making sets out a seven-step framework, and two of its steps settle most of what a sponsor needs to think about for deviation classification. The third step is assessing model risk, and the guidance defines it as a pair rather than a single property.

FDA's model risk, applied to deviation classification

+

Model influence

the weight of the model’s output in the decision

FDA: "the contribution of the evidence derived from the AI model relative to other contributing evidence used to inform the question of interest." Where a medical monitor reads the specification row, the protocol section and three signed precedents before signing, the model’s influence is one input among four.

Decided by the process design, not by the model

+

Decision consequence

what an incorrect decision would cost

FDA: "the significance of an adverse outcome resulting from an incorrect decision concerning the question of interest." A missed important deviation on a primary endpoint is high; a documentation deviation classified one way or the other is not.

Decided by the question, and it varies by deviation

Quoted from the FDA draft guidance, Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products, §IV.A.3 (January 2025). The application to deviation classification is ours.Source: FDA, AI credibility draft guidance, January 2025

The practical consequence of that pairing is that model risk is not a property of the software. The same classifier is low risk when a person signs every important deviation with the evidence in front of them, and high risk when its output is written straight into a submitted dataset. A vendor cannot tell you your model risk; your process decides it.

The same classifier at four points on the risk matrix

How the output is usedModel influenceDecision consequenceWhat that combination owes in evidence
A proposal a medical monitor reviews with the specification row, the protocol section and the precedents in front of themLow — one input among severalHigh — importance drives the report and the analysisAgreement measured against signed decisions, and a record of how often the reviewer overrode it.
Automatic confirmation of agreed "not important" classifications above a confidence floorHigh — nobody else looks at most of themLower, but not low: a misclassified important deviation disappearsA sampling rate for re-checking confirmations, with the sample’s results reported.
A per-protocol flag sent to the statisticianLow — the statistician applies the analysis planHigh — analysis-set membershipEvidence that flags are complete rather than accurate: what matters is that nothing is missed.
An importance flag written into the submitted deviation tabulation without reviewHighHighThe most evidence of any row here, and the arrangement hardest to defend. Avoid it.
Our application of FDA's model influence and decision consequence pair (draft guidance, January 2025, §IV.A.3) to four ways the same classification output can be used. The second and third rows describe how Protocol Deviation Management is configured on its sample study.Source: FDA, AI credibility draft guidance

The guidance's fourth step, on planning what evidence establishes credibility, contains the sentence that should govern every claim made about a system like this.

"If the COU involves a 'human in the loop,' ensure that the evaluation methods consider the performance of the human-AI team, rather than just the performance of the model in isolation."

FDA, Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products, draft guidance, §IV.A.4 (January 2025)

COU is context of use — what the model is being used for, on what, by whom, with what else in the room. The instruction is to evaluate the pair. A model at 70% whose errors a reviewer reliably catches is a better system than a model at 85% whose errors are plausible enough to be waved through, and no model-only benchmark can tell the two apart. The measure that follows from this is not accuracy but the share of output a person actually reviewed, which is one of the measures on the sample study's dashboard for exactly this reason.

The human-AI team, measured in Protocol Deviation Management

The dashboard: deviations classified over time, agreement with the medical monitor, time from report to classification, the share of deviations reviewed by a person, and important deviations by study and by cause.
Protocol Deviation Management on the fictional sample study. 'Share reviewed by a person' is the measure FDA's human-AI team framing implies: a system whose output nobody reads has a different risk profile from the same system with a reviewer, whatever the model scores on its own.

What EMA expects, and where the responsibility sits

EMA's September 2024 reflection paper on artificial intelligence across the medicinal product lifecycle is shorter on method and blunter on accountability. Its general principle leaves no room for the idea that a vendor's validation discharges a sponsor's obligation.

The paper's key principle is that the sponsor, applicant or manufacturer — not the tool's maker — is responsible for making sure every algorithm, model, dataset and data pipeline it uses is fit for purpose and meets the legal, ethical, technical, scientific and regulatory standards of EU legislation, GxP and EMA guidance. It then adds a warning in plain terms: in some respects those requirements may be stricter than what data scientists treat as normal practice (EMA, EMA/CHMP/CVMP/83833/2023, §2.2).

The last sentence is the one to take into a procurement conversation. A vendor's benchmark, however carefully constructed, is a data science artefact; what a sponsor owes is fitness for purpose under GxP, and EMA says in terms that the bar may be higher than data science practice would set. That means a sponsor needs its own evidence on its own deviations, not a slide.

Good clinical practice applies to the system, not just the output

EMA's paper is equally direct about which rulebook applies, and about what may be asked for later. "The use of AI/ML within the context of clinical trials should meet applicable requirements in the ICH E6 guideline for good clinical practice (GCP) or VICH GL9 Good clinical practices (veterinary). Of note, if the use could be of high regulatory impact or high patient risk in a clinical trial, and the method has not been previously qualified by the EMA for the specific context of use, the full model architecture, logs from model development, validation and testing, training data and description of the data processing pipeline would likely be considered parts of the clinical trial data or trial protocol dossier and thus may be requested for comprehensive assessment at the time of market authorisation, clinical trial application or GCP inspection" (EMA reflection paper, §2.3.3.1).

Two things follow for a buyer. First, the question "is this high regulatory impact or high patient risk?" is worth answering in writing before deployment, because the answer decides how much of that list a sponsor might have to produce. A classifier whose every important output is reviewed and signed by a medical monitor is a far easier case to make than one that writes unreviewed values into a submission. Second, nobody should claim a regulator has qualified any of this. Qualification is a specific EMA procedure for a specific context of use, and a vendor that implies it without naming the procedure is describing something that did not happen.

ICH E6(R3) adds the validation scope, and it covers more than most sponsors plan for: validation reaches beyond a system's standard functions to the configurations and customisations made for a particular protocol, automated checks and calculations included, and to the interfaces between systems, which are to be defined and validated too (ICH E6(R3) §4.3.4, 6 January 2025). For a deviation classifier, the protocol-specific configuration is the classification specification — 46 rows on the sample study, versioned and approved — and it falls inside that scope. So do the connections to the trial systems the deviations are read from.

Where an agent has to stop: the decisions that stay with people

The clearest signal of whether a system was designed for a regulated trial is what it refuses to do. On the sample study four limits are load-bearing, and each has a regulatory reason behind it rather than a cautious one.

Importance is signed by a person. Every important deviation, and every case where a team and the classifier disagree, goes to the medical monitor, who signs with a reason and a stated meaning — "I confirm the classification of this protocol deviation" — and needs a reason to reopen a signed decision. Automatic confirmation applies only where a team and the classifier both say not important at 0.90 confidence or above, and 10% of those are re-checked by a person each week.

Analysis-set membership stays with the statistician. The impact assessor flags a participant for per-protocol review and notifies the statistician; it never removes anyone from an analysis set, because ICH E9 places those criteria in the analysis plan and they are not the monitor's to apply.

Serious breach decisions stay with clinical quality. A possible serious breach at an EU or UK site is routed with its seven-day due date to a named person, and the assessment is theirs.

Nothing is written back into the source systems. The deviation reader takes participant identifiers only, no names or contact details, and never edits the trial management, data capture or randomisation systems it reads from.

Two opinions on a balance, and a person deciding between them

The decision screen: the contract research organisation's proposal of not important on one side and the classifier's proposal of important at 0.94 confidence on the other, with the deviation entry, the monitoring visit report and the site's assessor training log beside the balance, each finding highlighted where it was found. The protocol passage requiring a trained assessor, specification row EA-03 and three agreeing signed precedents sit below.
Protocol Deviation Management on the fictional sample study. The screen is built so the reviewer can check the proposal rather than accept it: the evidence is open, the citation is explicit, and both importance questions are answered where the signature goes.

Nine questions for an AI vendor, with the weak answers

These are written for a sponsor's diligence rather than for a technical evaluation, and each one has an answer that should stop the conversation.

Nine questions, and the answer that should end the conversation

  1. What is the context of use, in one sentence?

    Weak answer: a list of capabilities. FDA’s whole framework hangs on a specific statement of what the model is used for, on what data, by whom, with what else in the room.

  2. Against what ground truth was it measured, and who built it?

    Weak answer: internal testing. Ask how many records, who adjudicated them, and whether the reviewers’ own disagreement rate was measured — because the published figure for that is 20%.

  3. What is the model’s influence on the final decision, and what is the consequence of being wrong?

    Weak answer: treating risk as a property of the product. Model risk is the pair, and it changes with how the output is used.

  4. How was the human-AI team evaluated, not just the model?

    Weak answer: a model-only accuracy figure. If a person is in the loop, FDA asks for the pair to be evaluated.

  5. What does it cite for every output?

    Weak answer: an explanation generated after the fact. A citation to a specification row and a protocol section is checkable; a paragraph of reasoning is not.

  6. What does it do below its confidence floor, and is the floor a setting?

    Weak answer: there is no floor, or it is fixed by the vendor. The threshold is a sponsor’s risk decision and belongs on the activity trail with a version.

  7. Which of the demonstrated numbers are measured and which are targets?

    Weak answer: no distinction. Every figure that cannot be traced to a dated dataset is a design goal and should be labelled as one.

  8. Has any regulator qualified this for this context of use?

    Weak answer: anything other than no, with a named procedure. Qualification is specific, and implying it without naming it is a serious misstatement.

  9. What would you have to hand over at a GCP inspection?

    Weak answer: "that would not be requested." EMA lists model architecture, development logs, validation and testing records, training data and the pipeline description as things that may be requested for a high-impact use.

Our diligence checklist, written against the FDA draft guidance on AI to support regulatory decision-making (January 2025), the EMA reflection paper on AI in the medicinal product lifecycle (September 2024) and ICH E6(R3) §4.3.4.

A vendor that answers all nine well is not thereby proven accurate; it is proven to have thought about the right things. The accuracy question only has an answer on your deviations, against your signed decisions, which is why a pilot on a sample of past deviation logs is worth more than any benchmark. To see the six agents, their limits and the signatures on one study, start from Protocol Deviation Management, or read what a protocol deviation management plan has to contain for the specification any classifier has to be measured against.

Questions people ask

Is AI allowed in clinical trials under good clinical practice?

Yes, with conditions. EMA states that the use of AI within clinical trials should meet the applicable requirements of the ICH E6 good clinical practice guideline, and that the sponsor is responsible for ensuring the algorithms, models, datasets and pipelines are fit for purpose. Nothing in GCP prohibits automated support; what it requires is validation, an audit trail, and a human owner for decisions that affect participants or the reliability of results.

Do AI agents decide anything in a clinical trial?

In a well-designed system, no — they propose, flag and draft, and named people decide. The decisions that should stay with people are the ones with regulatory or analytical consequence: whether a deviation is important, whether a participant leaves an analysis set, whether something is a serious breach, and what corrective action a site gets. A system that cannot show you where it stops has not answered the first question an inspector will ask.

How do you validate a system whose output is a judgement rather than a number?

By establishing what the right answer is for a sample, measuring agreement against it, and continuing to sample once the system is live. ICH E6(R3) requires validation to cover protocol-specific configurations as well as standard functionality, which for a classifier means the study's own specification is in scope. The ongoing part matters most: a fixed sampling rate for re-checking automatic outputs, with the results recorded, is what turns a one-off validation into evidence.

Is 30% disagreement good or bad?

It depends entirely on what it is compared with and what happens next. Against two experts who disagreed with each other 20% of the time and with themselves 5–10% of the time, on a judgement made from short free text, it is in the region of human variability rather than far outside it. As a first pass whose flags a person reviews, it is usable; as an unreviewed answer written into a submitted dataset, it is not.

Has any regulator qualified an AI model for protocol deviation classification?

Treat any claim that one has as unproven until it comes with a named procedure and a date. EMA's qualification procedure is for a specific context of use and its opinions are published, so a real qualification can be pointed to; and EMA's own reflection paper notes that where a method has not been qualified for the specific context of use and the use is high-impact, the full model architecture, development and validation logs, training data and pipeline description may be requested at inspection or authorisation.

What is a context of use, and why does FDA keep asking for it?

It is a precise statement of what the model is used for, on what inputs, informing what question, with what other evidence alongside it. FDA's framework makes almost everything else conditional on it: model risk is assessed within the context of use, and the credibility evidence a model owes is chosen to match. Two sponsors running the same software can owe very different evidence because their contexts of use differ.

Protocol Deviation Management.
See it on your own files.

An agentic solution from Botminds. Every protocol deviation classified against one spec, whichever team logged it, with causes and site signals on one map. Book a demo, or explore how it works.