WorkflowResearch & Discovery

Evaluate · choosing a tool

AI tools for biomarker discovery, and where they stop

Much of what is sold as AI for biomarkers is discovery — finding a candidate in omics or imaging data — and the published reviews of that field are blunt about its weaknesses. But discovery is rarely where a programme loses time. This guide sets out what the two regulators already expect of an AI tool, nine questions to ask a vendor, and what Biomarker Strategy Planning does and does not do.

Updated · 13 min read

See one case, screen by screen →
Watch with transcript →

AI tools for biomarker discovery apply machine learning to omics, imaging or clinical data to find candidate markers that separate one group of patients from another. They are a real and useful category, and they are not the only thing the phrase gets used to sell. A buyer searching for AI in biomarker work is usually trying to solve a different problem — assembling, citing and defending evidence about markers that are already known, and getting assays timed against studies — and the two kinds of tool are judged by different tests. Biomarker Strategy Planning is firmly the second kind: it does not discover markers, and this guide says so plainly before it says anything else.

What follows is written to be used in an evaluation. The regulatory expectations are quoted from FDA and EMA documents and linked where they appear; the questions at the end are ours, and they are the ones we would want asked of us.

What do AI tools for biomarker discovery actually do?

In the discovery sense, they take a high-dimensional dataset — transcriptomic, proteomic, metabolomic, radiomic or digital-pathology — and search it for features, or combinations of features, that predict a label: disease versus control, responder versus non-responder, progressor versus stable. Feature selection narrows tens of thousands of candidate measurements to a handful; a classifier is trained on them; performance is reported as an area under the curve. The output is a candidate signature, which is the beginning of a biomarker programme rather than the end of one.

Three families sit alongside that core. Multi-omics integration tools combine data types that are individually underpowered. Image-based tools derive features from scans or slides that a human reader would not quantify. And literature or knowledge-graph tools propose candidates by connecting published findings rather than by analysing a cohort. All three produce hypotheses. None of them produces evidence that survives a regulator's question about an independent cohort.

Why the published reviews are cautious: validation and shared code

The most useful thing a buyer can read before an evaluation is a systematic review of the field they are buying into. A 2026 comparative review of AI for transcriptomic biomarker discovery is direct about the two recurring weaknesses, and both are process failures rather than algorithmic ones.

The first is validation. "Studies that rely on internal validation without testing on an independent validation cohort reported results that may not be reflective of real-world performance" (Khoo and Dhillon, Briefings in Bioinformatics 27(3), 2026). A model tuned and tested on partitions of the same dataset learns that dataset, including its batch effects and its recruitment quirks. The number it reports is an upper bound on what you would see, not an estimate of it.

The second is reproducibility. "Most of the reviewed studies did not provide publicly accessible code repositories, and only certain studies provide their codes in GitHub or Zenodo." Without the code, a published performance figure cannot be reproduced, extended or audited. For a buyer this translates into a concrete demand: ask what the tool was validated on, who held the validation cohort, and whether the pipeline can be re-run.

Discovery is not usually the bottleneck

Here is the uncomfortable observation behind this whole guide. Many development programmes are not short of candidate markers. They are short of the assembled, cited, current evidence about the markers they already intend to use, and short of a clear-eyed view of whether the assays behind them will be ready. That work is not glamorous and it consumes weeks of senior time.

Where the working days go on one biomarker strategy, before any tool

Searching and reading papers for each candidate marker
10 days
Drafting the house template, every claim cited
8 days
Pulling values from our own validation and stability reports
6 days
Reading rival registry records and congress posters
5 days
Validation status, precedent and assay timing against each study
5 days
Scientist check, section co-signs and the board papers
4 days
The solution's own estimated split of about 38 working days for one programme's biomarker strategy written by hand, against a 6.5-day design target with agents. These are estimates and a target on fictional sample data, not a customer's measured result. Note what is not in the list: nobody spends any of these days discovering a new marker.

Nothing in that list is a discovery problem. The three longest items are reading, extracting and drafting — tasks where a language model is genuinely strong and where the failure mode is a wrong number rather than a wrong hypothesis. That is the distinction a buyer should hold on to: a discovery tool proposes something new that must then be proved, while a decision-support tool handles material that already exists and must be reported faithfully. The second is easier to check, and that is a feature.

Discovery tools and decision tools: different inputs, different proof, different failures

A discovery toolA decision-support tool
What goes in

A discovery tool: A cohort’s omics, imaging or clinical data.

A decision-support tool: Papers, internal reports, registry records, specifications and guidance.

What comes out

A discovery tool: A candidate marker or signature, with a performance estimate.

A decision-support tool: Extracted values, a drafted document, a gap list, a costed option.

How you check it

A discovery tool: An independent cohort, held by someone else, and a re-runnable pipeline.

A decision-support tool: The passage behind each number, read beside the number.

What a wrong answer costs

A discovery tool: A programme chases a signature that does not replicate.

A decision-support tool: A wrong figure enters a strategy and is quoted onward until someone checks.

Who has to sign it off

A discovery tool: Translational science, and eventually a statistician on an independent dataset.

A decision-support tool: A scientist who confirms the values, and the section owner who signs the claim.

Which framework applies

A discovery tool: The same risk-based credibility framework, applied to a model whose influence is high.

A decision-support tool: The same framework, applied to a model whose influence depends on whether a human reads the source.

Our framing, written to be used in an evaluation. Both columns sit inside the same regulatory frameworks; what differs is how cheaply a buyer can verify the output.

The FDA's test: model influence against decision consequence

FDA's January 2025 draft guidance on AI in regulatory decision-making does not grade tools by architecture. It grades them by what happens if they are wrong. What it offers is a risk-based framework for establishing and judging how far an AI model can be trusted for one particular context of use, where that trust has to be earned by gathering credibility evidence about the model's performance in that use (FDA, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making).

Model risk is then defined with unusual precision, as two factors combined. Model influence is how much the evidence from the AI model contributes to answering the question, set against everything else that informs it. Decision consequence is how serious the harm would be if a wrong decision were made on that question. The guidance also closes off a common misreading: model risk means the chance that the model's output leads to a wrong decision with an adverse outcome, not some risk built into the model itself.

The two axes, and four biomarker tasks placed on them

Low influence · Low consequence

Pulling a reported accuracy figure out of a paper

A human reads the passage beside it. A wrong extraction is caught in seconds.

Low influence · High consequence

Timing an assay against first participant in

The arithmetic is checkable and the dates are in the plan, but being wrong costs a study design.

High influence · Low consequence

Ranking candidate markers for a reading list

The model drives the output, but nothing downstream depends on the order.

High influence · High consequence

Setting a cut-off, or deciding who enters a study

Highest scrutiny, and in our view not a decision to delegate at all.

Our placement of four everyday biomarker tasks on the two factors FDA names. Model influence falls when a human reads the source passage beside the output, which is why the same task can sit in two different cells depending on how the tool is built.

The consequence for a buyer is the guidance's own conclusion: the stringency of credibility assessment, the performance acceptance criteria, the risk mitigation and the documentation should all be scaled to the model's risk and fitted to the specific context of use. A vendor who offers the same validation story for every use of their tool has not engaged with the framework, because the framework's whole point is that the answer differs by use. This is the identical move that governs biomarkers themselves — context of use, and how it sets the evidence bar — applied to a different object.

The EMA's test: high patient risk and high regulatory impact

Europe reaches the same place by a different route: it grades risk on the same two factors, and then adds a principle about who is accountable.

The degree of risk depends not only on the AI technology and data quality, but also on the context of use and the degree of influence the AI technology exerts.

European Medicines Agency, Reflection paper on the use of Artificial Intelligence in the medicinal product lifecycle, September 2024

That is the same two-factor test FDA applies, in European words. The paper's key principle settles accountability: it is the clinical trial sponsor, the marketing authorisation applicant or holder, or the manufacturer who must make sure that every algorithm, model, dataset and data pipeline it uses is fit for purpose and meets legal, ethical, technical, scientific and regulatory standards. The responsibility does not transfer with the licence fee. Whatever a vendor says about their model, the sponsor is the one who must be able to show it was fit for purpose. The paper also introduces two labels worth borrowing for an internal policy: high patient risk, for systems that affect patient safety, and high regulatory impact, for uses with a substantial effect on regulatory decision-making.

For pivotal trials, one further line deserves to be read out at any vendor meeting where the words "continuously learning" appear: "Incremental learning approaches are not accepted, and any modification of the model during the trial requires a regulatory interaction to amend the statistical analysis plan." A model that changes under a running pivotal study is not an improvement; it is an unplanned change to the analysis plan. For broader governance vocabulary, NIST's AI Risk Management Framework, released in January 2023 and meant to help organisations build trustworthiness into how AI products and systems are designed, developed, used and evaluated (NIST AI RMF), is a widely used reference, and it is voluntary by design.

What should an AI tool never be allowed to decide?

Our answer, stated as a rule we hold ourselves to: anything a person would have to defend by name. That is a shorter list than it sounds, and it is unambiguous. A model may read, extract, compare, draft, calculate a date and cost an option. It should not set a cut-off, close a gap, choose between options, change a value, or move a document past the point where a human being's judgement is recorded.

The boundary, drawn as the work actually runs

  1. Agent

    Read and extract

    Papers and preprints in full text, internal validation and stability reports, registry records and specifications. Each value comes back with its cohort, its reference standard and its page.

    Never: reports a number other than as written — no rounding, converting or combining across studies

  2. Agent

    Compare and time

    Rival designs against ours, assay activities against first participant in, each option costed with its estimate basis on screen.

    Never: books anything, or treats an option as chosen

  3. A person decides

    A scientist confirms the values

    Every extracted value below the configured confidence threshold is shown beside its source passage, and is confirmed or corrected before it can be used.

    Never: is bypassed — the drafting step uses only checked values

  4. Agent

    Draft and review

    Writes the house template with no uncited factual sentence, then reads the draft like a board member and lists every gap with severity, evidence, owner and date.

    Never: closes a gap, or chooses an option

  5. A person decides

    Section owners co-sign

    Clinical pharmacology for target engagement, diagnostics for patient selection; the diagnostics lead decides the laboratory and diagnostic route.

    Never: delegates the decision back to the tool

  6. A person decides

    The board signs with a reason

    Approve, approve with conditions, or return for rework, with an electronic signature recording the meaning, date, time and reason.

    Never: signs a document whose claims cannot be opened at their source

The authority boundary in Biomarker Strategy Planning, as the solution enforces it. The agents can move a strategy no further than scientist review; every person's step is recorded.

Each step visible while it runs

The agents at work on a question: a step list showing rival registry records being re-read, laboratories checked for the same assay platform, timelines compared against first participant in, and options being costed, each step appearing as it completes with what it found.
Biomarker Strategy Planning on the fictional sample programme. A step list is not decoration: it is what lets a reviewer ask which source produced a claim, and get an answer without rerunning anything.

Evidence you can check: the passage behind every number

The cheapest test of any tool in this space takes a few minutes. Pick three numbers in its output. Ask to see the sentence, table or figure each one came from, in the source document, at the right place. If the tool shows you the passage, its model influence just fell — a human is now in the evidence path, and the framework's scrutiny falls accordingly. If it shows you a reference, or a summary of the source, or a confidence score and nothing else, the number is unverifiable in practice and the model's influence over the eventual decision is total.

Two related habits matter as much. First, numbers must be reported as written: an assay result quoted to a different number of decimal places, a hazard ratio converted into a percentage, or two cohorts pooled, are all changes to the claim that keep the citation. Second, where a key claim rests on a single source, that should be visible — one study is often right, and it is never the same as two.

Nine questions to ask a vendor, and the answers that disqualify one

Take these to a demonstration rather than to a procurement questionnaire, because the useful part is watching the product answer them rather than reading what the vendor wrote.

Nine questions, and the answer that should end the conversation

  1. Show me the passage behind this number.

    Disqualifying: a citation with no locatable passage, or a summary of the source instead of the source.

  2. What is the context of use you are claiming for this model?

    Disqualifying: a single answer that covers every task the product performs.

  3. What was it validated on, and who held that data?

    Disqualifying: internal validation only, on partitions of the data it was built from.

  4. What does it refuse to do?

    Disqualifying: nothing. A tool with no stated limits has not thought about where its authority ends.

  5. Does the model change over time, and how would I know?

    Disqualifying: continuous learning with no versioning, in a regulated study. EMA does not accept incremental learning in a pivotal trial.

  6. What happens to a value the model is unsure about?

    Disqualifying: it is used anyway, with a confidence score the document never carries.

  7. Who is accountable if the output is wrong?

    Disqualifying: an answer that implies the vendor. The sponsor carries that responsibility under EU guidance.

  8. Can I see the activity trail for this document?

    Disqualifying: no record of which agent did what, when, and which human confirmed it.

  9. What does it do with licensed full text?

    Disqualifying: copying licensed content into your systems. Read in place, cite the passage, store nothing you are not licensed to store.

Our questions, written as the ones we would expect to be asked. Questions five and seven follow the EMA reflection paper directly; question two follows FDA's credibility framework.

Where Biomarker Strategy Planning fits, and where it does not

It does not discover biomarkers. It does not analyse an omics dataset, derive a signature, or propose a marker nobody has published. A team whose problem is finding a new candidate in their own cohort needs a discovery tool and a statistician, and should buy one.

What it does is the work in the bar chart above. Six agents read papers and preprints in full text, internal validation and stability reports, rival registry records and congress material, vendor specifications and regulatory records. They extract values with the cohort, reference standard and page; they set each marker's validation status for its stated use without inferring qualification where there is no record; they draft the eight-section house template with no uncited factual sentence; and they list every gap with a severity, an owner and a date. In the fictional sample programme the solution ships with, that produces 112 cited claims from 47 sources with no uncited claim left — demonstration figures on invented data, and the right shape of counter to ask any tool for.

The limits are the point. Values below the configured confidence threshold wait for a biomarker scientist. Agents can move a strategy no further than scientist review. The gap and risk reviewer lists and costs but never closes a gap or chooses an option, and an option stays a proposal until the diagnostics lead decides. Licensed full text is read in place and never copied out, and every step is on the activity trail. To see the boundary working, start from Biomarker Strategy Planning, or read how the assay evidence underneath it is judged.

Gaps listed, owned and dated — and never closed by a model

The gap analysis for the sample strategy: five gaps, each with a severity, the evidence behind it, a proposed action, a named owner and a due date, and controls to assign the gap or accept it as a risk. A note states that the reviewing agent never closes a gap.
Biomarker Strategy Planning on the fictional sample programme. The severity, the evidence and the proposal are drafted; the assignment and the acceptance of risk are a person's, and the screen is built so that it cannot be otherwise.

Questions people ask

Is AI biomarker discovery reliable?

It is useful for generating candidates and weak as evidence, and the published reviews say why. The recurring problems are internal validation without an independent cohort, which inflates reported performance, and unavailable code, which makes a result impossible to reproduce. Neither is an argument against the methods; both are arguments for asking what a reported figure was measured on before treating it as a programme decision.

What does the FDA expect when AI supports a regulatory decision?

A credibility assessment sized to the model's risk, where risk is model influence combined with decision consequence — not a property of the algorithm. The draft guidance asks that the stringency of the assessment, the acceptance criteria, the mitigations and the documentation all be commensurate with that risk and tailored to the specific context of use. In practice that means naming the use first and arguing about the evidence second.

What does the EMA say about AI in drug development?

Its reflection paper puts responsibility squarely on the sponsor, applicant or manufacturer for every algorithm, model, dataset and pipeline being fit for purpose and compliant. It grades risk by context of use and degree of influence, distinguishes high patient risk from high regulatory impact, and states that in pivotal trials incremental learning approaches are not accepted, with any mid-trial model change requiring a regulatory interaction to amend the statistical analysis plan.

Can an AI model choose a biomarker cut-off?

It can compute candidate cut-offs and show what each would do to sensitivity, specificity and the screening funnel. It should not choose one. A cut-off determines who enters a study and, eventually, who receives a drug; it sits at the high-influence, high-consequence corner of the regulators' own matrix, and it is a decision someone has to defend by name. The useful division is that the model does the arithmetic and the person does the choosing.

How do you check an AI tool's output against the source?

Ask for the passage, not the reference. A tool that opens the source document at the sentence, table or figure a number came from can be checked in seconds, and a human reading that passage lowers the model's influence over the decision. If a number can only be checked by finding the paper yourself, you are not verifying the tool; you are redoing its work.

Is discovery or decision-making the bottleneck in biomarker work?

For a programme in clinical development that already has its candidate markers, usually decision-making. In the solution's own estimate of a strategy written by hand, the days go on reading and extracting, drafting a cited document, reading competitors' designs and timing assays against studies — not on finding new markers. That is worth establishing before an evaluation, because it determines whether the right purchase is a discovery platform, a decision-support tool, or neither.

Biomarker Strategy Planning.
See it on your own files.

An agentic solution from Botminds. Each program's biomarkers laid out as lanes across the phases (target engagement, pharmacodynamics, patient selection and safety) with the assay, its readiness and the decision it supports, beside how rival trials use the same markers. Book a demo, or explore how it works.