WorkflowClinical Development

Choosing · evaluation guide

Risk based quality management software: where deviations fit

Risk based quality management tools are strong at the two layers the regulators describe explicitly — risk assessment up front and centralised signal detection during conduct — and quiet about the layer in between, which is deciding whether one deviation is important. This guide sets out the four layers, what each regulator has actually asked for, and the eight questions worth putting to any evaluation — with Protocol Deviation Management as the worked example of the middle layer.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

Risk based quality management software is the category of tools that help a sponsor identify what could go wrong on a clinical trial, set controls proportionate to those risks, and then watch the accumulating data for signs that something has. It covers four distinguishable layers — risk assessment, risk control, centralised detection, and action — and most products in the category are genuinely good at the first and the third, because those are the two the regulators describe in detail. The layer they mostly leave to a monthly meeting is the one in the middle: deciding, consistently and with a citation, whether each individual protocol deviation is important. That matters because every count a risk based quality management tool draws rests on that classification, so the first question in an evaluation is not what the software does with ten thousand deviations but what it does with one. Botminds' Protocol Deviation Management is built for that middle layer, and this guide is about the whole stack rather than about one product in it.

The worked examples come from the sample study inside Protocol Deviation Management: Northwind Bio's VLT-302, Veltrimab in plaque psoriasis, Phase 3, 64 sites and 612 participants, with the contract research organisation Ashgrove Research. The sponsor, the drug, the sites and the people are fictional characters in that sample data.

What is risk based quality management software?

It is software that operationalises the quality management approach ICH E6(R3) requires of a sponsor: identify the risks to the things that matter, evaluate them, control them in proportion, detect departures while the trial is running, and report what happened. ICH E6(R3) became the applicable good clinical practice guideline in 2025 and is the text most of the category now points at.

E6(R3)'s risk evaluation step is where the shape of the software comes from. The sponsor weighs each identified risk, and the controls already in place, on three questions: how likely the harm is, how detectable it would be, and how much it would affect participant protection and the reliability of the results (ICH E6(R3) §3.10.1.2). Likelihood, detectability, impact — the three axes of almost every risk assessment module on the market.

The control step is equally specific about proportionality: controls are to be sized to how much the risk matters to participants and to the reliability of the results, and the guideline's own examples of where mitigation can sit are protocol design and implementation, monitoring plans, agreements setting out each party's role, and training (ICH E6(R3) §3.10.1.3). Note what the examples of mitigation are: protocol design, monitoring plans, contracts and training. Not software. The software is where the decisions about those four things are recorded and watched, which is a useful thing to keep straight during a demonstration.

The four layers of a risk based quality stack

Treating the category as one product makes evaluations confusing, because two tools that both call themselves risk based quality management can occupy entirely different layers. Separating them makes the gaps visible.

Four layers, and which regulator text describes each

LayerWhat it producesWho owns itWhere the requirement is written
1 · Risk assessmentA list of critical-to-quality factors, the risks to each, and a score for likelihood, detectability and impact.The study team, before first participant in: clinical, data management, statistics and quality together.ICH E8(R1) §3.2 on critical-to-quality factors; ICH E6(R3) §3.10.1.1 and §3.10.1.2.
2 · Risk controlPre-specified acceptable ranges, a monitoring plan, training, and the roles written into agreements with each party.The sponsor, documented and proportionate to the risk.ICH E6(R3) §3.10.1.3.
3 · Centralised detectionSignals: key risk indicators, statistical outliers across sites, data checks, and the deviations themselves as they arrive.Central monitoring — "sponsor personnel or representatives (e.g., clinical monitors, data management personnel, or statisticians)".FDA, A Risk-Based Approach to Monitoring, Q&A (April 2023), Q4; ICH E6(R3) §3.11.
4 · Classification and actionA decision on each deviation, a corrective action with an owner, and a record of what the decision rested on.The medical monitor and the trial manager, with clinical quality on serious questions.ICH E6(R3) §3.9.3 and §3.10.1.6; FDA draft guidance on protocol deviations (December 2024).
Our layering of the category, with the regulator text that describes each layer. Layers 1 and 3 are where most tools in the category concentrate; layer 4 is where the counts underneath layer 3 are actually made.Source: ICH E6(R3); FDA, Risk-Based Monitoring Q&A

The order of the layers is not the order of dependence. Layer 3 produces counts, and those counts are only meaningful if layer 4 has already decided what is being counted. A stack built in the written order — assess, control, detect, then act — ends up with dashboards full of numbers nobody trusts, and the fix is always applied at the wrong layer: more charts instead of a classification specification.

Risk based monitoring and risk based quality management are not the same thing

The two phrases are used interchangeably in marketing and mean different things in practice. Risk based monitoring is about where to send monitors and what they look at when they arrive. Risk based quality management is the wider discipline that decides what matters on this trial and what controls it gets, of which monitoring is one.

Two phrases, two scopes

Risk based monitoringRisk based quality management
The question it answers

Risk based monitoring: Which sites need a visit, and what should be verified when we get there?

Risk based quality management: What could make this trial unreliable or unsafe, and what are we doing about each of those things?

When it starts

Risk based monitoring: Once sites are open and data are arriving.

Risk based quality management: During protocol design — ICH E8(R1) puts critical-to-quality factors before the protocol is finalised.

Its main instruments

Risk based monitoring: Centralised monitoring, targeted source data verification, triggered and reduced site visits.

Risk based quality management: Critical-to-quality factors, risk assessment, pre-specified acceptable ranges, monitoring strategy, training, agreements.

What centralised monitoring is

Risk based monitoring: "A systematic analytical evaluation of study conduct across multiple clinical sites, carried out by sponsor personnel or representatives."

Risk based quality management: One control among several. It detects; it does not decide what a finding means or who fixes it.

What it does with a deviation

Risk based monitoring: Treats a cluster of them as a trigger for a visit or a corrective action.

Risk based quality management: Treats each one as evidence about whether a control is working, and the aggregate as evidence about the risk assessment.

Where it is weakest

Risk based monitoring: It assumes the findings it counts mean the same thing at every site.

Risk based quality management: It can become a document rather than a practice: a risk assessment written once and never revisited against what the trial actually produced.

Quoted definition of centralised monitoring from FDA, A Risk-Based Approach to Monitoring of Clinical Investigations: Questions and Answers, Q4 (April 2023). The rest is our reading of the two disciplines.Source: FDA, Risk-Based Monitoring Q&A, April 2023

FDA's list of what centralised monitoring enables is worth having in full when a tool is being demonstrated, because it is the definition of the layer. It "may allow sponsors to (1) review study-wide data for inconsistencies or omissions; (2) perform activities such as data checks, for completeness and consistency; (3) verify source data; (4) ensure that institutional review board and informed consent documents are current; and (5) determine which clinical sites need on-site review" (FDA, Risk-Based Monitoring Q&A, Q4). Five capabilities, and every one of them is about detection or verification. None of them is about deciding what a protocol deviation means.

Quality tolerance limits: what they are, and what they are not

Quality tolerance limits are the one instrument in this category with an explicit obligation attached, and the obligation is reporting rather than alerting. ICH E6(R3) is direct about it.

"The sponsor should summarise and report important quality issues (including instances in which acceptable ranges are exceeded, as detailed in section 3.10.1.3) and the remedial actions taken and document them in the clinical trial report (see ICH E3)."

ICH E6(R3), Guideline for Good Clinical Practice, §3.10.1.6 Risk Reporting (Step 4, 6 January 2025)

That sentence does three things to an evaluation. It makes exceeding a limit an event that ends up in the clinical study report, which means the limits must be few and defensible. It ties the quality management layer directly to the deviation section of the report, because "important quality issues" and "remedial actions taken" are the same material the deviation section narrates. And it means a tool that cannot produce an auditable record of which limits were set, when, by whom, and what happened when one was exceeded, is leaving the sponsor to assemble that record by hand at the end.

What quality tolerance limits are not: a site league table, a per-site threshold, or a general alerting mechanism. ICH E8(R1) warns against the dilution that follows when they are treated that way — critical-to-quality factors "should be clear and should not be cluttered with minor issues (e.g., due to extensive secondary objectives or processes/data collection not linked to the proper protection of the study participants and/or primary study objectives)" (ICH E8(R1) §3.2, 6 October 2021). A study with thirty quality tolerance limits has not identified what matters; it has made a list.

Three instruments that are routinely confused

Quality tolerance limitKey risk indicatorSite deviation trending rule
LevelThe trial.Usually the site, sometimes the country or the organisation.The site, the cause or the team.
Set whenBefore first participant in, during risk assessment.At study start-up, with the monitoring strategy.Once enough data exist to see the study’s own distribution.
Built onA pre-identified critical-to-quality factor.A count or a ratio in the trial data: screen failures, queries, missed visits, deviations.Classified deviations, usually important ones only.
What happens when it firesAn evaluation of whether there is a systemic issue, and a line in the clinical trial report.A review, usually a monitoring decision: visit, call, retrain.One proposed action at one site, with an owner.
Who it is forThe sponsor, and ultimately the reader of the report.Central monitoring.The trial manager.
The hidden dependencyThat the acceptable range was set on something that actually matters.That the count means the same thing at every site.That importance was decided consistently and before the rule ran.
Compiled from ICH E6(R3) §3.10.1.3 and §3.10.1.6, ICH E8(R1) §3.2 and the configurable trending rules in Protocol Deviation Management. The last row is the point of the table: all three rest on something that is decided elsewhere.

Key risk indicators sit on counts, and counts sit on classification

A key risk indicator is a number computed from trial data and watched for movement. Deviation counts are among the most common, and they are the most fragile, because a deviation count is not an observation — it is the output of a judgement that somebody made, often at a contract organisation, often without a written rule.

The failure is easy to reproduce. Two organisations run twenty sites each on one protocol. One treats an assessment performed by an untrained assessor as important; the other treats it as a documentation issue because the visit itself was in window. The indicator for important deviations per site now ranks the first organisation's sites worse, and every downstream decision — which sites to visit, which to retrain, which to mention in the report — inherits that artefact. No amount of statistical sophistication at layer 3 repairs it, because the input is inconsistent rather than noisy.

The sample study measures this directly rather than assuming it away: over its last 30 days of sample data, agreement with the medical monitor's final call runs at 96% for the classifier, 91% for the in-house team, and 86% and 79% for the two contract organisations. A gap of seven points between the two outsourced teams, and twelve between the in-house team and the weaker of them, is the size of the measurement error sitting underneath any site ranking built on their counts. What a protocol deviation rate can and cannot tell you works through the rate itself.

What does RBQM software do with one deviation?

This is the question that separates the layers in a demonstration, and it is worth asking in exactly this form: show me what happens to this one deviation, from the moment it arrives to the moment somebody has decided about it. Most tools in the category will show a deviation arriving, appearing in a count, and contributing to a chart. The useful follow-ups are what happened in between.

On the sample study the answer is a sequence with times against it: the deviation reader takes the entry from the trial systems and writes one clean description; the classifier matches it to a row of the study's classification specification and cites the protocol section breached, with a confidence; the precedent finder retrieves the closest signed decisions on similar deviations from any organisation on the programme; and the impact assessor checks whether the per-protocol set or a serious breach clock is engaged. Where the team's proposal and the classifier's differ, or the deviation is important, it goes to the medical monitor to sign.

What the middle layer has to do on one deviation, in Protocol Deviation Management

41s

for four agents to read one deviation, classify it and check it against 3 signed precedents

Seen on the sample study

0.90

confidence floor below which a person decides, configurable from 0.80 to 0.99

A setting, versioned and on the activity trail

10%

of automatic confirmations re-checked by a person each week

Settable at 5%, 10% or 20%

96.2%

agreement with the medical monitor’s final call — the design target, not a measured production result

Called a target because it is one

Figures from Protocol Deviation Management on its fictional sample study. The 41 seconds is machine time on one deviation and excludes the human decision that follows it; the 96.2% is tagged in the product as a design goal rather than a measured outcome, and is quoted here as such.

Three of those four numbers are governance rather than performance, and that is the point. A confidence floor, a sampling rate for re-checking automatic confirmations, and an honest label on a target are the things that make an automated classification defensible. A tool that offers the speed without the three is offering the part that is easy.

One deviation's classification settings in Protocol Deviation Management

The settings screen: the classification specification by study and version — VLT-302 v3.0 with 46 rows, approved 12 September 2026 — then the routes, naming who decides when a contract organisation and the classifier disagree, who takes eligibility and dosing questions, who receives a possible serious breach with its seven-day clock, and who receives per-protocol impact; then automatic confirmation on or off, the minimum confidence of 0.90, and a weekly quality sample of 10% of automatic confirmations.
Protocol Deviation Management on the fictional sample study. Everything on this screen is a setting rather than a behaviour built into the software: one versioned specification per study, named owners for each route, and a confidence floor below which a person decides.

The layer most tools leave to a monthly meeting

Ask a category tool where classification happens and the honest answer is usually: in a meeting, into a free-text field, afterwards. The deviation is logged at the site, a proposal is attached by whoever wrote it up, and the decision is taken at a periodic review — on the sample study's own account of the manual process, about nine days after the deviation was reported, of which five are spent waiting for the meeting to come round.

That arrangement has three costs, and none of them is speed for its own sake. The first is consistency: a meeting that works through thirty deviations in ninety minutes is not applying a specification evenly, and nobody can later say which row it applied. The second is the clock on things that cannot wait — a possible serious breach at an EU site carries a seven-day notification deadline from the day the sponsor becomes aware, and a monthly cadence can consume it entirely; the serious breach test and its seven-day clock sets out what that deadline actually requires. The third is the per-protocol flag the statistician needs, which simply does not arrive.

FDA's draft guidance points the monitoring layers at exactly this material: "Finally, site monitors and centralized monitoring activities should focus on critical elements for early identification of important protocol deviations at specific sites or that suggest a pattern consistent with a systemic study issue" (FDA, Protocol Deviations for Clinical Investigations, draft guidance, December 2024). Early identification of important protocol deviations — which presupposes that importance has been established early.

The order the layers actually run in

  1. A person decides

    Risk assessment

    Names the critical-to-quality factors and evaluates each risk for likelihood, detectability and impact, before the protocol is final.

    Never: Never a document written once and filed

  2. A person decides

    Risk control

    Sets acceptable ranges, the monitoring strategy, the training and the roles in each agreement, in proportion to the risk.

    Never: Never thirty tolerance limits where three would do

  3. Agent

    Detection

    Watches the accumulating data across sites for inconsistencies, omissions and outliers, and takes in every deviation as it is reported.

    Never: Never decides what a signal means

  4. Agent

    Classification

    Matches each deviation to a specification row, cites the protocol section, answers both importance questions and gives a confidence.

    Never: Never classifies below the confidence floor without a person

  5. A person decides

    Decision

    The medical monitor signs importance with a reason; clinical quality takes serious breach questions; the statistician takes analysis-set questions.

    Never: Never delegated to the tool that proposed the classification

  6. A person decides

    Action and reporting

    One corrective action with an owner, and the quality issues and remedial actions summarised in the clinical trial report.

    Never: Never an alert nobody closes

Our reading of the layers, in the order they have to run rather than the order they are usually written. The classification step is the one most commonly missing from the category, and the decision step is the one that must stay with people.Source: ICH E6(R3) §3.10

What the regulators have actually asked for, and what they have not

It is worth being precise, because a good deal of the category's marketing implies a requirement that does not exist. The regulators require a risk based approach to quality management. They do not require software, do not name a tool, and do not prescribe an architecture.

What ICH E6(R3) does require is that risks are evaluated on the three stated axes, that controls are proportionate, that acceptable ranges are set where relevant and evaluated when exceeded, and that important quality issues and their remedial actions are reported in the clinical trial report. What FDA's monitoring guidance adds is that centralised monitoring is a legitimate and encouraged way to do part of that, carried out by sponsor personnel or their representatives. What FDA's protocol deviations guidance adds is that monitoring should be pointed at critical elements and early identification.

None of that is an argument against buying software; it is an argument for knowing what the software is for. A sponsor can satisfy every one of those requirements with a well-run team and good documents, slowly. Software earns its place where the volume makes the manual version unreliable — on a 64-site study with two contract organisations and 418 deviations, a monthly meeting is not a slow version of consistency, it is a different and worse thing.

Validation, and the audit trail an inspector will ask to see

Any system in this space is a computerised system under GCP, and ICH E6(R3) states the validation standard in one sentence: validation has to show that the system meets its established requirements for completeness, accuracy and reliability, and performs consistently with its intended purpose (ICH E6(R3) §4.3.4). Two phrases there do most of the work in an evaluation. "Established requirements" means the sponsor has to have written down what it expects the system to do, which is harder than it sounds for a tool whose output is a judgement. "Consistent with its intended purpose" means the validation has to cover the configuration the sponsor actually uses, not the vendor's default.

EMA's guideline on computerised systems adds the expectation that audit trail review is itself a planned, risk-based activity: "Procedures for risk-based trial specific audit trail reviews should be in place and performance of data review should be generally documented. Data review should focus on critical data" (EMA, Guideline on computerised systems and electronic data in clinical trials, 9 March 2023). For a classification tool, the critical data is the classification: who proposed it, what it cited, who signed it, whether it was ever reopened and why.

That gives a concrete test for a demonstration. Ask to see the activity trail for one deviation that was classified automatically, re-checked in the weekly sample, disagreed with, and reopened. If each of those four events is present with a who, a what, a when and a reason, the system is inspectable. If any of them is absent, the gap is the sponsor's to explain.

Eight questions for an evaluation, with the weak answer to each

These are ordered so that the first three can be asked in the first twenty minutes and will usually settle whether the rest are worth asking.

Eight questions, and the answer that should end the demonstration

  1. Show me what happens to one deviation, from arrival to a decision.

    Weak answer: a chart with the deviation counted in it. The layer being evaluated is what happened before the count.

  2. Against what does it decide that a deviation is important?

    Weak answer: "our model understands the protocol." The answer you want names a versioned, study-specific specification with rows, and shows the row and the protocol section cited on a real decision.

  3. What does it do when it is not sure?

    Weak answer: it returns its best guess. The answer you want is a configurable confidence floor below which a person decides, and a record of which decisions crossed it.

  4. Who signs, and what does the signature mean?

    Weak answer: a status field changed by whoever had the screen open. A signature should state its meaning and require a reason, and reopening one should require another.

  5. How do you know the classifications are right?

    Weak answer: an accuracy figure with no denominator. The answer you want is a sampling rate for re-checking automatic confirmations and an agreement measure broken down by team.

  6. Can it tell me whether two organisations classify the same way?

    Weak answer: a combined count. Without agreement by team, every cross-site comparison the tool draws is of unknown reliability.

  7. What does an alert propose, and who closes it?

    Weak answer: an alert addressed to a role inbox with no action attached. One proposed action, one named owner, and a recorded outcome.

  8. Which of these numbers is measured, and which is a design target?

    Weak answer: no distinction drawn. Any figure a vendor cannot locate in a dated dataset is a target, and should be said to be one.

Our evaluation checklist for the category, written against ICH E6(R3) §3.10 and §4.3.4, the EMA computerised systems guideline and the FDA monitoring and protocol deviation guidances.

What the detection layer looks like when the classification layer is underneath it

The site trends screen: three alerts each proposing one action — retraining for Site 114, a temperature-log check for Site 131, and a question to the contract research organisation whose sites miss pre-dose pregnancy tests 2.4 times as often per site as in-house sites. Every site's important-deviation rate per 100 participant-visits is plotted against the study's threshold, with panels for top causes and for agreement with the medical monitor by team.

Alerts propose actions; people decide.

Protocol Deviation Management on the fictional sample study VLT-302. The consistency panel is the part worth copying into any evaluation: a detection layer that cannot show how consistently its inputs were classified is reporting on itself.

Where a deviation tool stops, and what else a stack still needs

Being honest about scope is part of a buying guide. A deviation classification tool does not do risk assessment: it will not tell a study team what its critical-to-quality factors are, and nothing in it replaces the cross-functional conversation that produces them. It does not do statistical outlier detection across the whole trial dataset, which is a different discipline with different mathematics. It does not manage corrective and preventive actions through to closure, beyond proposing one and naming an owner. And it does not set quality tolerance limits, which are a trial-level judgement made before the data exist.

What it does is make the counts underneath the rest of the stack mean something, by deciding the same way at every site and in every organisation, within minutes rather than weeks, with the specification row and the protocol section attached to each decision and a signature on top. On a study with one in-house team and two contract organisations, that is the difference between a dashboard and a measurement.

If you are evaluating the category, the two guides to read next are what a trial management system will and will not decide about a deviation and what the published evidence on AI agents in clinical trials actually shows. To see the classification layer on one study, start from Protocol Deviation Management.

Questions people ask

Do I need risk based quality management software, or a better deviation process?

If deviations are being classified inconsistently or late, software at the detection layer will make that visible and will not fix it, because the counts it draws inherit the inconsistency. The sequence that works is to write a classification specification, decide who signs, and get decisions happening during conduct; a tool then multiplies a process that already works. The reverse order buys dashboards nobody acts on.

What should such a tool be able to show you about one site?

Its rate on a stated denominator with a stated window; the breakdown of that rate by cause, because one repeating cause and six scattered ones demand different responses; what the site's rate was before and after the last corrective action; how consistently the team covering it classifies against the final decision; and what is coming — how many participants reach the affected visit in the next month.

Does ICH E6(R3) require risk based quality management software?

No. It requires a risk based approach: risks evaluated for likelihood, detectability and impact, controls proportionate to them, pre-specified acceptable ranges where relevant, and important quality issues with their remedial actions reported in the clinical trial report. Software is one way to do that work; the guideline's own examples of risk mitigation are protocol design, monitoring plans, agreements and training.

Can a trial management system do the same job?

It holds the deviation entries, which is a necessary part and the part most teams already have. What it generally does not hold is a versioned classification specification, a citation to the protocol section breached, the earlier signed decisions a new proposal should be measured against, or a signature with a stated meaning. Those are the things that make a count comparable across sites and organisations.

Who owns risk based quality management inside a sponsor?

In practice it is shared, which is why it needs naming rather than assuming: clinical operations owns the monitoring strategy, data management owns the data checks, biostatistics owns the analysis consequences, and clinical quality owns the standard and the serious questions. The decisions that most often fall between them are importance classification and analysis-set impact, and both work better with a named owner and a named alternative.

How much of this can be done in a spreadsheet?

More than vendors suggest and less than sceptics hope. A single-site study with forty deviations can be managed in a spreadsheet with a written specification and a disciplined reviewer. The things that break at scale are consistency across organisations, the audit trail an inspector asks for, and any deadline measured in days — and all three break quietly, which is why they are usually discovered at the point where they are expensive.

Protocol Deviation Management.
See it on your own files.

An agentic solution from Botminds. Every protocol deviation classified against one spec, whichever team logged it, with causes and site signals on one map. Book a demo, or explore how it works.