WorkflowResearch & Discovery

Compare · where AI belongs

AI tools for toxicology: what they read and what they cannot decide

Reading, extracting, ranking and drafting are tasks European and US regulators now write guidance for, on condition that the sponsor stays answerable for the output. Deciding is not on that list. This guide sets out the line, the measured failure mode behind the citation rule, and how Target Safety Assessment keeps proposing and deciding apart.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

Ask about AI in toxicology and the first answer is usually prediction: a model that estimates whether a structure will be hepatotoxic, mutagenic or cardiotoxic. That is a real field with a long history, and it is not the work in front of a discovery toxicologist asked to assess a new target. That work is reading — knockout phenotypes, loss-of-function genetics, expression, the labels and reviews of drugs on the same pathway — and then scoring what was read. Machine help across that second kind of work is now explicitly contemplated by both the European Medicines Agency and the US Food and Drug Administration, with the same condition attached in both places: the output supports a decision, and the sponsor answers for it. Botminds' Target Safety Assessment draws its line in the same place, with a named toxicologist deciding.

What follows is about tools that read and draft for a target assessment, rather than tools that predict toxicity from structure; AI that drafts a GLP study report is a different job, covered in the guide to AI in toxicology report writing. Product examples are sample data on fictional targets; regulatory and research figures are linked where they appear.

What AI tools for toxicology are actually good at

The honest list is narrower than the marketing and more useful. They are good at breadth — covering a literature that no individual can read in the time a target decision allows. They are good at extraction: pulling the sentence that carries a finding, with its statistic, its species and its allele, into a structured field. They are good at consistency, applying the same reading rules to the fortieth target as to the first. And they are good at drafting from material that has already been structured.

They are poor at exactly the things that make a toxicologist valuable: judging whether a 14-millisecond difference in a rodent means anything in a person, knowing which of two contradictory lines of evidence to believe, and carrying the professional responsibility for the answer. The EMA's reflection paper makes the positive half of this explicit for non-clinical work — AI and machine-learning applications there may strive not only to achieve improved performance in data analysis and interpretation but could potentially also include AI/ML modelling approaches to replace, reduce, and refine the use of animals and improve human translatability (EMA reflection paper, 2024).

Three lanes of toxicology work, and what may be automated in each

Read and extract

Searching a literature index, pulling the sentence that carries a finding, extracting a rate with its confidence interval, quoting a label section, tabulating phenotypes by organ.

Automatable, with every output traceable to the passage it came from.

Propose and draft

Suggesting a probability and an impact, raising a challenge to a single-source claim, proposing the study that would settle a liability, writing a section of the report from the evidence.

Automatable as a proposal shown to a person, never as a decision applied in silence.

Decide and sign

Accepting, re-scoring or dismissing a liability; choosing what to fund; concluding that a target is safe to pursue; signing the assessment.

A named person, whose signature carries its meaning. Not automatable, and not delegable to a supplier.

Our reading of where the line falls, written to the two regulatory texts quoted on this page. The third lane is where responsibility sits, and it does not move.

Do regulators allow AI in nonclinical work?

Neither agency has published a rule that permits or forbids it in the way the question implies. Both have published expectations, and both put them in the same place: the use is acceptable, the documentation and the responsibility are not relaxed. The EMA's position on process is the one buyers tend to overlook: its reflection paper expects a company's existing standard operating procedures to cover every AI or machine-learning application it uses in non-clinical studies (EMA, 2024). A tool that cannot be described in a procedure, validated against it and audited to it is a tool that creates work rather than saving it.

On the US side, the FDA's draft guidance was announced in the Federal Register as being to provide recommendations to industry on the use of artificial intelligence (AI) to produce information or data intended to support regulatory decision-making regarding the safety, effectiveness, or quality for drug and biological products (90 FR 1157, 7 January 2025). The operative words are "to support". The model produces information; the decision is somebody else's, and the credibility of the information is what the agency expects a sponsor to be able to demonstrate.

The two regulatory texts, read side by side

EMA reflection paper, 2024FDA draft guidance, announced January 2025
What it addresses

EMA reflection paper, 2024: The use of AI and machine learning across the medicinal product lifecycle, with a section on non-clinical development.

FDA draft guidance, announced January 2025: The use of AI to produce information or data intended to support regulatory decision-making on safety, effectiveness or quality.

On non-clinical work

EMA reflection paper, 2024: Contemplates improved data analysis and interpretation, and modelling that could replace, reduce and refine animal use.

FDA draft guidance, announced January 2025: Framed by the regulatory decision the output will support, rather than by the stage of development.

On process

EMA reflection paper, 2024: Existing standard operating procedures are expected to extend to all such applications in non-clinical studies.

FDA draft guidance, announced January 2025: Recommends that sponsors establish and document the credibility of a model for its specific context of use.

On transparency

EMA reflection paper, 2024: Prefers transparent models, to strengthen procedural fairness, accountability and the prevention of bias.

FDA draft guidance, announced January 2025: Scales the level of oversight and the amount of documentation to the model’s risk and its context of use.

On responsibility

EMA reflection paper, 2024: Places it on the trial sponsor, applicant, holder or manufacturer to ensure every algorithm, model, dataset and pipeline is fit for purpose.

FDA draft guidance, announced January 2025: Addressed to sponsors, who plan, gather and document the evidence that a model’s output is credible.

Left column from the EMA reflection paper on AI in the medicinal product lifecycle (September 2024); right column from the Federal Register notice announcing the FDA draft guidance (January 2025) and its stated scope.Source: EMA reflection paper, 2024; 90 FR 1157, 2025

The citation problem, measured

The specific failure that disqualifies a general-purpose assistant from assessment work has been measured rather than merely asserted. A study in Scientific Reports examined the references produced by two widely used models, checking each against the literature: this study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works (Walters and Wilder, 2023). More than half of the earlier model's citations referred to works that did not exist, and even for the newer model nearly one in five did. Among the citations that were real, a substantial share carried errors in the author, title, date, journal or pagination.

Fabricated and erroneous references, by model and by kind of error

What was measuredGPT-3.5GPT-4
Share of citations that were fabricated — no such work exists55%18%
Real cited works examined for errors101340
Of those, share with one or more substantive citation errors43%24%
Share with an incorrect date22%16%
Share with an incorrect author name14%6%
From the Scientific Reports study of bibliographic citations generated by ChatGPT (2023). Substantive errors are incorrect authorship, title, journal, publisher, volume or pagination, as distinct from formatting differences.Source: Walters and Wilder, Scientific Reports 2023

Put that rate beside what an assessment is for. A liability register is a list of claims about what happens when a protein is removed, each of which a reviewer, a committee or a regulator may ask to see the source of. One fabricated reference in a signed document is not a quality problem; it is a finding against the process that produced it. The measurement is also why the fix is structural rather than a matter of choosing a better model: the newer model fabricated far less, and far less is not none.

How do you stop a model inventing a source?

By never letting it supply the source. The reliable design is the opposite of asking a model for references: the tool retrieves real documents first, extracts the passage, and then writes a claim that is attached to that passage, so the citation is a pointer to something already in hand rather than a string the model produced. A claim with no retrieved passage behind it does not get written.

What makes a claim in an assessment checkable

+

The claim

one statement about one organ

Scoped to an organ system and a species, so it can be argued with.

+

The passage

the words the source actually uses

Retrieved from the document, not reconstructed from memory.

opens where it was found

+

The source

a document the reader can reach

A paper, a label section, a regulatory review, a phenotype entry, a cohort scan.

+

The decider

a named person

Who accepted, re-scored or dismissed the claim, and why.

signs

−

An unsourced assertion

nothing to open

Plausible, unverifiable, and the one thing a target assessment must not contain.

The four parts of a claim an assessment can defend, and the one it must refuse. In the product the report drafter's stated limit is the same sentence: no statement without a citation.

Two further habits matter. Quote rather than paraphrase where a number or a rule is at stake, because a paraphrase cannot be checked against the original without reading the original. And keep the passage with the claim permanently, not just at the moment of writing: a source that can be opened six months later, when a committee asks, is the difference between an assessment and a document.

What human oversight means in practice

Oversight is often promised as a checkbox and delivered as a formality. The useful test is whether the human step changes anything. In the adjacent discipline of systematic review automation, where machine screening has been in use for years, the practical guide to those tools describes the arrangement plainly: in the available systems, human reviewers still screen a set of abstracts first and then review what the system recommends (Marshall and Wallace, Systematic Reviews, 2019). The machine changes the order and the volume of the work. It does not remove the reviewer from it.

Translated into target assessment, oversight means four concrete things. The person sees the evidence, not only the conclusion. The person can change the score, and the change is kept with its reason. Disagreements between evidence lines are surfaced rather than averaged away. And the challenge raised against a claim is shown to the person rather than applied on their behalf — because a challenge applied silently is a second model decision wearing the clothes of a review.

A challenge shown beside the liability, not folded into it

A liability page for a fictional target headed QT prolongation seen in mice only, scored medium risk, unlikely times serious, with agent confidence shown as 71% and a mouse versus human disagree badge. A summary of what the agents found is followed by an evidence ladder of six lines, each marked as supporting the risk, arguing against it or finding no evidence, with its finding in plain words. The right-hand panel shows the two cited sources with the exact passages highlighted, each with a link to open at that passage, and tabs for sources and critic. Buttons at the foot offer not relevant, re-score and accept as scored.
Sample data on a fictional target. The agents' confidence is printed as a number rather than implied, the challenge sits on its own tab for the toxicologist to weigh, and the three buttons at the foot are the only places a score actually changes.

Who is responsible for the output

The sponsor. The EMA says so in terms, and the FDA's draft guidance is written on the same footing. The EMA's reflection paper states as a key principle that it is the responsibility of the clinical trial sponsor, marketing authorisation applicant or holder, or manufacturer to ensure that all algorithms, models, datasets and data processing pipelines used are fit for purpose and in line with legal, ethical, technical, scientific and regulatory standards. The FDA's draft guidance points the same way from its own angle: it is written to help sponsors plan, gather and document the evidence that a model's output is credible for its context of use, which is work only the party relying on the output can do.

For a buyer this has a sharp implication. You are not purchasing a transfer of risk. You are purchasing a way of doing the work whose outputs you will have to defend, which means the things to evaluate are the ones that determine whether you can defend them: whether every claim opens its source, whether the human decisions are kept with their reasons, whether the tool can be described in your own procedures, and whether your own study data stays where your agreements say it must.

Where tradeoffs among the trustworthy characteristics arise, measurement provides a traceable basis to inform management decisions.

NIST AI Risk Management Framework 1.0, January 2023

The NIST framework names the properties a trustworthy system is expected to have — validity and reliability, safety, security and resilience, accountability and transparency, explainability and interpretability, privacy enhancement, and fairness with harmful bias managed. Most of those are not inspectable from a demonstration. What a buyer can inspect is whether each one is measured, and the sentence above is why that matters: measurement is what converts a claim about a tool into something a reviewer can weigh against the alternative.

Preprints, grey literature and which sources the agents may read

Which documents a tool is allowed to read is a safety decision, not a configuration detail. Preprints carry findings that are often months ahead of the peer-reviewed record and sometimes wrong in ways review would have caught. Conference abstracts omit the methods that decide whether a result means anything. Both have a place in target work — a target assessment that ignored a relevant preprint would be a poor one — but only if a reader can tell what they are looking at.

The workable rule is a switch with a label rather than a ban. In the sample settings of the product, preprints are off by default and, when switched on, every citation drawn from one is labelled as not peer reviewed wherever it appears. The same logic applies to internal material: the pathway and off-target agent's stated limit is that a company's own study data never leaves its own environment, which is a source rule and a confidentiality rule at once.

Which sources may be read, and what counts as high, are settings rather than assumptions

A settings screen with three panels. The report template panel lists eight required sections every assessment contains, with two optional sections below — species relevance and study design switched on, competitor safety comparison switched off. The scoring panel sets high risk as probability times impact of twelve or more, requires a second signature from the head of discovery safety, allows low risks to be accepted in one step, has the critic challenge every single-source liability, and shows re-scores and dismissals needing a reason as locked on. A third panel headed sources the agents may read is beginning, noting licences respected and nothing leaving the workspace.
Sample data in the product. Two of these are the ones a buyer should look hardest at: what counts as high, and which sources may be read — because both change what the assessment concludes.

How to test a tool before an assessment depends on it

Run it on a target you have already assessed by hand, and compare more than the conclusion. The conclusions will usually agree, because the obvious liabilities are obvious. The information is in the disagreements and the omissions: what did it find that you missed, what did you weigh that it did not, and where did it assert something you cannot trace. Then apply a short list of disqualifying behaviours, each of which can be observed in a single afternoon.

Five behaviours that should disqualify a tool in a trial

  1. A claim with no passage behind it

    Any statement in the draft that cannot be opened at the source it came from. One is enough: it means the writing step is not bound to the reading step.

  2. A challenge applied rather than shown

    If the tool revises its own score after an internal objection and presents only the result, the reviewer is reviewing a conclusion with its argument removed.

  3. A score with no named decider

    A risk rating that appears in the report without a person having accepted it is a model output in a document that will be read as a human judgement.

  4. A preprint or unreviewed source cited without a label

    The source class has to travel with the citation. A reader cannot discount what they cannot identify.

  5. Internal study data leaving your environment

    Your own toxicology archive is the most sensitive material in the assessment and the most valuable. Where it goes is a contractual and a technical question; ask for both answers.

Each line is observable in a trial on one target rather than inferable from a specification. Any single failure is worth more attention than a whole page of accuracy claims.

Where Target Safety Assessment draws the line between proposing and deciding

The product states the limit on each agent rather than in a policy page. The risk scorer proposes scores only; a toxicologist accepts, re-scores on the matrix or dismisses each liability. The report drafter writes no statement without a citation. The critic's challenges are visible to the toxicologist and never silently applied. The human genetics agent states whether evidence is homozygous or heterozygous and never infers a direction of effect without a source. The same-mechanism drug agent quotes the label section number and reports confidence intervals with every rate. The pathway agent keeps internal study data inside the company's environment.

Two settings enforce the same line from the other side: a re-score or a dismissal requires a written reason, locked on rather than optional, and a remaining high risk routes the assessment to a second signer before it goes to the committee. Together they mean that the trail of an assessment shows not just what was concluded but who concluded it and why they differed from what was proposed.

One review record: every agent step and every decision in Target Safety Assessment

A review record for a fictional target, listing every step in reverse order with a timestamp and an initial or an AI badge: the discovery toxicologist signed the assessment with a recommendation to go with a de-risking plan, the same person added a dog telemetry and hERG patch clamp study to the plan, the report drafter drafted eight sections with every statement cited, the critic challenged three liabilities and the agent agreed on two, the risk scorer scored seven liabilities by organ and found mouse and human disagree on three, and the assessment planner started six evidence agents in parallel. The liability list on the left shows all seven reviewed.
Sample data on a fictional target. The agent steps and the human decisions are on the same list, in the same form, which is what makes a later reviewer able to ask who changed what.

One figure in the product deserves a caveat, because it is the kind that gets quoted out of place: the share of proposed scores a toxicologist keeps is a design target for the live product, not a measured accuracy benchmark, and the rest are re-scored with a reason. A tool that proposed scores nobody ever changed would not be a better tool; it would be one whose reviewers had stopped reviewing. How the whole assessment is produced, and what it costs to run it three different ways, is compared in the guide to doing it by hand, outsourced or with agents; the evidence lines themselves are described in the guide to on-target and off-target risk. To see the line between proposing and deciding on a real screen, start from Target Safety Assessment.

Questions people ask

Can AI write a target safety assessment?

It can produce every part of one that is reading, extraction and drafting, and it cannot produce the part that is deciding. A tool can assemble the knockout phenotypes, the carrier scans, the expression pattern and the same-mechanism labels, each tied to a passage, and draft the report from them. The probability and impact scores, the conclusion and the signature belong to a toxicologist, and both the EMA and FDA texts are written on that assumption.

Do regulators allow AI in nonclinical work?

Neither has prohibited it, and both have set expectations. The EMA's reflection paper contemplates AI in non-clinical development, including approaches that could replace, reduce and refine animal use, while stating that existing standard operating procedures extend to all such applications. The FDA's draft guidance addresses the use of AI to produce information intended to support regulatory decision-making on safety, effectiveness or quality.

How do you stop an AI inventing a citation?

By designing so that it cannot supply one. Retrieve the document first, extract the passage, and attach the claim to that passage, so the citation points at something already held rather than at a string the model generated. The measured fabrication rates — 55% of one model's citations and 18% of a newer one's in the Scientific Reports study — are why this has to be structural rather than a matter of choosing a better model.

Who is responsible for an AI model's output in drug development?

The sponsor. The EMA's reflection paper names the trial sponsor, marketing authorisation applicant or holder, or manufacturer as responsible for ensuring that all algorithms, models, datasets and pipelines are fit for purpose and in line with legal, ethical, technical, scientific and regulatory standards. Buying a tool does not move that responsibility to the supplier.

Should preprints be included in a target assessment?

They can be, if they are labelled. A preprint may carry the only published evidence on a new target, and excluding it can leave a liability unseen; including it unlabelled lets an unreviewed finding carry the same weight as a reviewed one. The workable setting is off by default, switchable on, with every citation from one marked as not peer reviewed wherever it appears.

What does human oversight mean in practice?

That the person sees the evidence rather than the conclusion, can change the score, has their reason kept with the change, and is shown the objections rather than their result. The comparison worth making is with machine-assisted systematic review, where human reviewers still screen abstracts and then review the system's recommendations — the volume and the order change, the reviewer does not disappear.

Target Safety Assessment.
See it on your own files.

An agentic solution from Botminds. A body map of one target's on-target risks, organ by organ, from knockout mice, human carriers, expression and same-mechanism drugs, scored, cited and signed by a toxicologist. Book a demo, or explore how it works.