SystemSupply Chain

The complete guide · November 2026

No-code AI builders in regulated life sciences, explained.

A no-code AI builder lets the person who knows the data make an AI assistant by describing it in plain words. In a drug, biotech or device company the building is the easy part. The hard questions are who may build, what somebody must review before anyone relies on the answers, and what happens when something changes. This guide answers those three, with FDA's Computer Software Assurance guidance and the EU's good manufacturing practice (GMP) annexes on computer systems (Annex 11) and on AI (draft Annex 22) stated exactly as they stand.

This guide is for quality assurance (QA) leads, validation and IT quality staff, data stewards, and the business analysts and quality specialists who want to build. It also serves leaders deciding whether to allow it. No background needed.

See one case, screen by screen →
Watch with transcript →
A quick example · fictional

Imagine you are Lena Ortiz, a supply chain analyst at a medical device manufacturer. People keep asking you the same freight questions, and a new report request waits a median of 31 days in the engineering queue. On October 6, 2026, at 10:31 am, you describe an AI assistant in your own words: what we spend, with which carrier, per unit on each lane, how much goes on expedited freight.

The builder proposes six warehouse tables, you name the keys that join them, and it takes nine test questions from reports your finance data steward and logistics team have already signed off. Then it runs them. Seven pass. Two fail, and they fail on the number everyone would quote.

The assistant's first answer
September spend
$2,412,300
Expedited share
17.1%
The signed September report
September spend
$2,291,800
Expedited share
18.0%
$120,500 too high118 voided Kestrand Freight invoiceswere re-billed in September, so they counted twice. You add one sentence, leave out voided invoices, and all nine tests pass.

Good tests catch the confident wrong number before anyone acts on it. Freight is the easy case, because no quality record depends on it. Point the same tool at which version of a procedure is effective, and your QA lead will ask who built it, who checked it and who approved the change.

No-code AI builder guide
Part A · Understand it

Business-built AI assistants under GxP: who may build one, what gets reviewed, how changes are controlled, and what FDA and the EU have written

What a no-code AI builder does, where FDA's Computer Software Assurance guidance and EU GMP Annexes 11 and 22 stand in November 2026, why plain-language answers from a database go wrong, and how regulated companies handle the requests today, explained without selling anything. If you have never run a validation, start here.

01 · In plain words

A no-code AI builder turns one person's knowledge of the data into an assistant other people can question

In every drug, biotech and device company there are people who know where the answers live. A supply chain analyst knows which tables hold freight invoices and that a voided invoice gets re-billed. A document control specialist knows which procedure version is effective at which site. A quality engineer knows which deviation codes the annual product review counts. Each of them is asked the same questions every week, and for a new question the usual route is a ticket to a data or IT team that does not know the data as well as they do.

A no-code AI builder lets that person skip the ticket. They describe the questions in plain words. The software reads the data catalog, proposes tables, writes the query that answers each question, and runs it. The result is an AI assistant: anyone with access can type a question and get a figure, with the query that produced it. Nobody writes code, and the build can take minutes.

A regulated company works under GxP, the good practice rules for making medicines and devices (good manufacturing practice, or GMP), running clinical trials and testing in laboratories. There, a figure from such an assistant may end up in a quality record, an annual product review (the yearly review of each product's quality data) or a decision about a lot. So three questions decide whether the tool helps or harms. Who may build an assistant, and for which uses. What is reviewed before anyone relies on its answers, and by whom. And what happens when something changes: a table, a rule, the AI model underneath, the person who built it.

None of these questions is new. They are the questions behind computer system validation, change control and data integrity, asked of a new kind of software. What is new is that the builder may sit in finance, logistics or quality, outside IT, and that a large language model writes the first draft. Regulators have begun to say what they expect of both, and in Europe the text that will govern AI in medicine manufacturing is still a draft. This guide sets out what is written, and what each team can decide for itself.

Five steps from a plain-language question to an assistant people may rely on

  1. Triage the useBefore anything is built: is this answer going to feed a GxP decision or record, or is it business reporting? The answer decides everything after it.
  2. Describe and draftThe builder describes the questions; the software proposes tables, join keys, rules and a query for each question.
  3. Test against trusted answersEach question is run against figures someone has already signed off. A failure is explained, fixed and re-run. This step and the next are the subject of this guide.
  4. Review and signA named person confirms each question and its query; for regulated uses a second person, and quality where the procedure says so.
  5. Release and keep under controlThe assistant goes live under the change procedure, its tests re-run, and every change goes back through steps 3 and 4.

Steps 3 and 4 are where business-built tools usually fail an audit. The people, company and figures in the example are fictional.

Four things that catch out the first business-built assistant in a GxP company

The rules we read never say "no-code", and their duties still apply

We found no use of "no-code", "low-code" or "citizen developer" in EU GMP Annex 11, its draft revision, FDA's CSA guidance, FDA's data integrity questions and answers or ICH E6(R3), the international guideline on good clinical practice.1,2,3,4,5 Our reading: their duties attach to the use, the data and the change, whoever wrote the query.

FDA's CSA guidance is for device makers

Its scope is software "used as part of production or the quality management system for medical devices".3 It names "artificial intelligence/machine learning tools" among the software it can reach.3 A drug maker may borrow its method, and has Part 211 and Annex 11 to answer to.6,1

Europe's draft keeps large language models out of critical GMP uses

Draft EU GMP Annex 22 says generative AI and large language models "should not be used in critical GMP applications".7 It is a draft. A June 2026 workshop discussed widening it, and no revised text is published.8

A fluent answer can be wrong by six figures

One join that repeats a row, or a voided invoice counted beside its re-bill, gives a confident wrong total. On a 2024 benchmark of enterprise data work, a code-writing system built on one of that year's strongest models solved 21.3% of tasks.9 Testing against known answers is the protection.

Four words to know first: builder, test set, confirmation and change

Builder

The person who describes the assistant and answers for what it is meant to do. In a GxP company, a named role your procedure allows, with training for it.

Test set

Questions with known answers, taken from reports someone already signed off. The assistant must reproduce them before anyone relies on it.

Confirmation

A named person's recorded decision that a question, its query and its test results are right for the intended use. For regulated uses, a second person confirms.

Change

Anything that could alter an answer: a rule, a query, a table, the test set, the AI model version, the prompt. Each one goes through your change procedure.

Fourteen terms are defined in the one-page summary.

02 · What the builder does

Inside a no-code AI builder: catalog, draft query, test, confirmation and routing

In shortUnder the plain-language surface, a no-code AI builder for data does six things: it reads the catalog, proposes tables and joins, writes the rules down, drafts a query for each question, tests each query against known answers, and routes later questions to the right assistant. A person decides at two points, and each point leaves a record.

The phrase "no-code AI builder" covers several kinds of product. Some build chat helpers that answer from documents. Some automate workflows. The kind this guide covers answers questions from structured data, usually a warehouse or a reporting database, by writing a database query in SQL. The industry calls the core trick text-to-SQL: turn a question in words into a query in code. The builder wraps that trick in the steps a careful analyst would take anyway.

A careful build has two points where a person decides, and both leave a record
  1. 1Describe

    The builder says what people ask, in plain words, and which reports they already trust.

    BuilderMinutes
  2. 2Read the catalog

    Software proposes tables and join keys; the builder names the right ones.

    Software, then builderMinutes
  3. 3Draft queries and rules

    One query per question; rules such as which records to leave out, written in words.

    SoftwareSeconds
  4. 4Test

    Each query is run against answers taken from signed reports; failures are explained.

    SoftwareSeconds
  5. 5Confirm

    A named person reads each query and its tests and confirms or sets it aside.

    Builder, and a second personMinutes to days
  6. 6Release and route

    The assistant goes live under your change procedure; questions reach it by topic.

    Process ownerPer your procedure

Illustrative · replace with your own numbers

Show the numbers
StepWhat happensWhoTime
DescribeThe builder says what people ask, in plain words, and which reports they already trust.BuilderMinutes
Read the catalogSoftware proposes tables and join keys; the builder names the right ones.Software, then builderMinutes
Draft queries and rulesOne query per question; rules such as which records to leave out, written in words.SoftwareSeconds
TestEach query is run against answers taken from signed reports; failures are explained.SoftwareSeconds
ConfirmA named person reads each query and its tests and confirms or sets it aside.Builder, and a second personMinutes to days
Release and routeThe assistant goes live under your change procedure; questions reach it by topic.Process ownerPer your procedure

Why a database question needs a query someone can read

A chat answer that draws on documents can quote its source. An answer from a database is a calculation: sums, joins, filters, a currency rate. The only way to know whether it is right is to read the calculation or to compare its result with a figure you already trust. A builder that hides the query asks the reader to trust it blind.

A builder that shows the query, and runs the same query against signed-off figures before release, gives a reviewer something to check. This is the difference that matters for a regulated company, and Part C turns it into questions for any vendor.

What a no-code builder leaves to qualified people

Software can draft fast and test fast. It cannot decide what the answer is for, which records belong in it, whether a test set is fair, or whether a use is critical to product quality. Those are judgments, and every text in this guide gives them to a qualified person.

Draft Annex 22 says that where a large language model is used in a non-critical GMP application, "personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use".7 FDA, in a 2026 warning letter to a drug manufacturer that used AI to write its procedures, put it more bluntly: "If you use AI as an aid in document creation, you must review the AI generated documents".10

03 · The rules, dated

Where CSA, Annex 11, draft Annex 22, Part 11 and the AI Act stand on November 3, 2026

In shortFDA's Computer Software Assurance guidance is final, dated February 3, 2026, and written for device makers. EU GMP Annex 11 is still the 2011 text. The revised Annex 11 and the new Annex 22 on AI are drafts; inspectors aim to hand final text to the Commission in the fourth quarter of 2026, with no adoption date. The EU AI Act's high-risk dates moved to December 2027 and August 2028.

Most of what governs AI in GMP was written or moved in the last two years
  1. Jun 30, 2011Annex 11 in operation

    The current EU GMP text on computerized systems.

  2. Jan 2025FDA draft on AI

    Draft guidance on AI for regulatory decisions on drugs and biologics.

  3. Jul 7 to Oct 7, 2025EU consultation

    Revised Annex 11, Chapter 4 and new Annex 22 out for comment.

  4. Feb 2 to 3, 2026QMSR, then CSA final

    Device quality rule in effect; CSA guidance reissued the next day.

  5. Jun 30, 2026Annex 22 workshop

    EMA hears industry on large language models in GMP.

  6. Jul 27, 2026AI Omnibus in force

    AI Act high-risk dates moved.

  7. Q4 2026Inspectors' target

    Final Annex 11 and Annex 22 text to the Commission.

Source 3,11,1,12,8,13,14,15

Show the numbers
WhenWhat
Jun 30, 2011Annex 11 in operation
Jan 2025FDA draft on AI
Jul 7 to Oct 7, 2025EU consultation
Feb 2 to 3, 2026QMSR, then CSA final
Jun 30, 2026Annex 22 workshop
Jul 27, 2026AI Omnibus in force
Q4 2026Inspectors' target
Dec 2, 2027AI Act Annex III
TextStatus, November 3, 2026What it asks of a business-built assistant
FDA, Computer Software AssuranceFinal, February 3, 2026; supersedes the September 24, 2025 version; nonbinding3,16For a device maker's production or quality system use: intended use, a risk analysis on foreseeable failures, assurance sized to risk, and a record
21 CFR Part 820 (QMSR)In effect February 2, 2026; 820.70 removed, ISO 13485 incorporated11,3Validation of software used in production or the quality system, under ISO 13485 subclauses 4.1.6, 7.5.6 and 7.6
21 CFR 211.68 and 211.222025 CFR edition, the newest published6,17Changes to records only by authorized people; input and output checked; the quality unit approves procedures
21 CFR Part 11In force18Where the assistant creates or keeps electronic records that another FDA rule requires (a predicate rule): validation, audit trails, limited access, authority checks, trained people
EU GMP Annex 11January 2011 revision, in operation June 30, 20111,19Risk-based validation, an inventory, change and configuration management, access control, audit trails, periodic evaluation
Draft Annex 11 (revision)Draft; consultation closed October 7, 20252,12Configuration documented in a controlled specification; segregation of duties and least privilege; risk-based periodic reviews
Draft Annex 22 (AI)Draft; not listed in EudraLex Volume 47,19No large language models in critical GMP uses; qualified people answer for outputs in non-critical uses; change and configuration control
ICH Q9(R1), Q10Final20,21Formality and effort in proportion to risk; a change management system that evaluates each change
ICH E6(R3), section 4.3Final, January 6, 20255For trial systems: risk-based validation, and validated changes under change control
FDA draft on AI for regulatory decisionsDraft, January 202522A credibility assessment for AI output that supports a regulatory decision on a drug or biologic
EU AI Act and AI OmnibusLaw; Omnibus in force July 27, 2026; Annex III rules December 2, 2027; Annex I rules August 2, 202823,14,15Measures to support AI literacy of staff; in our reading most internal data assistants fall outside the high-risk lists, which is counsel's call

GAMP 5 (second edition, July 2022) has an appendix on AI and machine learning, and ISPE published a separate GAMP guide on AI in July 2025.24,25 Both are paid industry guides with no legal force; we read their contents pages. The European Medicines Agency's (EMA) reflection paper on AI in the medicinal product lifecycle, adopted in September 2024, places responsibility for algorithms, models and data on the sponsor, applicant, marketing authorization holder or manufacturer using them.26

04 · CSA, exactly

What FDA's February 2026 CSA guidance says about AI tools, analytics and spreadsheets, and whom it covers

In shortFDA's guidance reaches software a medical device maker uses in production or its quality management system, and it names AI tools, data analytics tools and automated workflows among them. It sorts each function by whether its failure could foreseeably compromise safety. Its own examples show that "it is only reporting" may still be high process risk when the report feeds quality trending.

The guidance's full title is "Computer Software Assurance for Production and Quality Management System Software". It was issued on February 3, 2026, replacing the version of September 24, 2025, one day after FDA's device quality rule took effect.3,11 Its scope sentence is short: "computers or automated data processing systems used as part of production or the quality management system for medical devices".3 It does not cover software that is itself a device, and by its terms it does not reach drug manufacturing or clinical systems.

CDER, FDA's drug center, lists in its 2026 guidance agenda a planned supplement on assurance for AI-based systems in drug manufacturing and clinical investigations; an agenda does not bind CDER, and nothing had been issued by November 3, 2026.27

The sentence that brings an AI builder into scope

FDA's framework "can be applied, but is not limited, to automation tools (e.g., BOTS or automatic workflows), data analytic tools, artificial intelligence/machine learning tools, and cloud computing when used as part of production or the quality management system".3 So a device maker cannot set a no-code AI assistant aside as an office tool if its answers feed quality data. The first question is the intended use, function by function.

Three of FDA's own examples, read for a business-built assistant

FDA's exampleWhat FDA concludedWhat it suggests for an assistant
Spreadsheet, basic inputDocumenting curing time and temperature: low process risk; vendor assessment and installation "may be sufficient"3An assistant that only lists records may need little more than supplier assessment and set-up checks
Spreadsheet, custom formulasA formula that calculates statistics to monitor the process: "additional validation by the manufacturer might be necessary"3A query that calculates a figure used in the quality system is the custom formula of this kind
Business intelligence applicationConnectivity and data integrity functions: high process risk, because a failure could mean missed quality trends that "in some cases" foreseeably compromise safety; detailed scripted testing3If quality trending rests on the assistant's numbers, test its data connections and its calculations hard

Reasonably foreseeable failures: FDA's test for the rare wrong answer

FDA asks manufacturers to consider "which failures are reasonably foreseeable (as opposed to likely)".3 A wrong join is not likely on any given day. It is foreseeable over the life of an assistant that hundreds of people question.

FDA calls a function high process risk when its failure "may result in a quality problem that foreseeably compromises safety", and gives examples such as functions that "measure, inspect, analyze and/or determine acceptability of product or process with limited or no additional human awareness or review".3 The phrase "with limited or no additional human awareness or review" is the design question for any assistant: is a person checking its answer before it counts?

FDA's CSA guidance reaches only the device maker's systems; drug and trial systems answer to other texts

Our reading of each text's scope. Darker means the text speaks directly to that use

Outside its scopeBy analogy onlyApplies in partApplies directly
FDA CSA21 CFR 211.68EU Annex 11Draft Annex 22ICH E6(R3) 4.3Device QMS reportingApplies directlyBy analogy onlyBy analogy onlyOutside its scopeOutside its scopeDevice production dataApplies directlyBy analogy onlyBy analogy onlyOutside its scopeOutside its scopeDrug plant quality dataBy analogy onlyApplies directlyApplies directlyApplies in partOutside its scopeDrug supply reporting, no GMP recordOutside its scopeBy analogy onlyBy analogy onlyOutside its scopeOutside its scopeClinical trial system dataBy analogy onlyOutside its scopeOutside its scopeOutside its scopeApplies directly

Our reading of the sources 3,6,1,7,5

Show the numbers
FDA CSA21 CFR 211.68EU Annex 11Draft Annex 22ICH E6(R3) 4.3
Device QMS reportingApplies directlyBy analogy onlyBy analogy onlyOutside its scopeOutside its scope
Device production dataApplies directlyBy analogy onlyBy analogy onlyOutside its scopeOutside its scope
Drug plant quality dataBy analogy onlyApplies directlyApplies directlyApplies in partOutside its scope
Drug supply reporting, no GMP recordOutside its scopeBy analogy onlyBy analogy onlyOutside its scopeOutside its scope
Clinical trial system dataBy analogy onlyOutside its scopeOutside its scopeOutside its scopeApplies directly

What FDA wants in the record

For each function FDA recommends a record of the intended use, the result of the risk analysis, the testing done, the issues found, a conclusion on acceptability, who performed the testing and when, and review and approval where appropriate.3 It prefers "digital records, such as system logs, audit trails, and other data generated and maintained by the software, as opposed to paper documentation, screenshots".3 A builder that keeps its own test runs, confirmations and versions can supply most of that record. Whether the record is enough is still your call.

05 · Annex 11 and draft Annex 22

EU GMP Annex 11 and draft Annex 22 on AI: what they ask of a builder that drafts with a large language model

In shortAnnex 11, in force since 2011, asks for risk-based validation, controlled changes and access, audit trails and periodic evaluation of any computerized system used in GMP work. Draft Annex 22 covers AI models, and keeps generative AI and large language models out of critical GMP uses, those with direct impact on patient safety, product quality or data integrity. In non-critical uses a qualified person answers for each output. Both drafts may change.

Annex 11 applies "to all forms of computerised systems used as part of a GMP regulated activities".1 Its principle is short: "The application should be validated; IT infrastructure should be qualified." Where a system replaces a manual operation, "there should be no resultant decrease in product quality, process control or quality assurance".1

Clause 1 ties the extent of validation to "a justified and documented risk assessment". Clause 10 says changes, "including system configurations should only be made in a controlled manner in accordance with a defined procedure". Clause 12 asks for access restricted to authorized people, and clause 11 for periodic evaluation.1

A business-built assistant used in GMP work falls under every one of these clauses, whoever built it.

The draft revision of Annex 11 adds two lines a builder will feel

The consultation draft, which closed on October 7, 2025, says it "should be clear what functionality, if any, is modified or added by configuration of a system", and that the chosen configuration "should be documented in a controlled configuration specification" (section 6.6).2,12 A plain-language rule such as "leave out voided invoices" is configuration in that sense. Section 11.10 sets two principles for access: segregation of duties, "users who are involved in GMP activities do not have administrative privileges", and least privilege.2 It is a draft and has no force yet.

Draft Annex 22, section 1, quoted

"Following the above, the document does not apply to Generative AI and Large Language Models (LLM), and such models should not be used in critical GMP applications. If used in non-critical GMP applications, which do not have direct impact on patient safety, product quality or data integrity, personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use, i.e. a human-in-the-loop (HITL) and the principles described in this document may be considered where applicable."7

Three consequences follow for a builder whose first drafts come from a large language model.

  1. The critical or non-critical call comes before anything else, and draft Annex 22 defines non-critical by impact: no "direct impact on patient safety, product quality or data integrity".7
  2. In a non-critical use the qualified person, the human in the loop, is a requirement, and where testing has been reduced because a person reviews the output, draft sections 3.3 and 10.5 ask that the operator's responsibility be written into the intended use and that records be kept of that review.7
  3. The draft puts a tested model and its system under change control and configuration control before use (section 10).7
Draft Annex 22 sorts AI by model type and by whether the GMP use is critical

Section 1 of the consultation draft, our reading. The annex covers AI in critical uses only

Should not be usedOutside the annex's scopeAllowed with a qualified person answering for each outputCovered by the annex's requirements
Critical GMP useNon-critical GMP useStatic model, same output for same inputCovered by the annex's requirementsOutside the annex's scopeModel that keeps learning in useShould not be usedAllowed with a qualified person answering for each outputModel whose output can vary for the same inputShould not be usedAllowed with a qualified person answering for each outputGenerative AI and large language modelsShould not be usedAllowed with a qualified person answering for each output

Our reading of the sources 7

Show the numbers
Critical GMP useNon-critical GMP use
Static model, same output for same inputCovered by the annex's requirementsOutside the annex's scope
Model that keeps learning in useShould not be usedAllowed with a qualified person answering for each output
Model whose output can vary for the same inputShould not be usedAllowed with a qualified person answering for each output
Generative AI and large language modelsShould not be usedAllowed with a qualified person answering for each output

The draft may move, and the record says how far

EMA's report on a workshop held June 30, 2026 says consultation comments "indicated support for allowing manufacturers to use Large Language Models and generative AI systems in GMP applications, whether critical or non-critical", that the drafting group "has therefore discussed" widening the scope under a documented, risk-based control strategy, and that "concerns remain".8 Its closing words are cautious: "there may be viable pathways".8 The same report records the view, from discussion, that "Human-in-the-loop is one possible control, not a default requirement for all AI uses".8

No revised text had been published by November 3, 2026, and EMA's inspectors' work plan sets the fourth quarter of 2026 as the target for giving the Commission final text of Annex 11 and Annex 22, with no adoption date.13 Plan for the draft as written, and design your evidence so it can change.

06 · Who may build

Who may build, who confirms and who administers a business-built assistant

In shortNone of the texts in this guide forbids a business user from building an assistant. In our reading, they govern access and duties: changes only by authorized people, administrators independent of the record's content, users without administrative rights, and training for anyone who develops or uses the system. A sound procedure names who may build for which uses, requires a second person to confirm regulated uses, and keeps administration with someone else.

The texts are specific about duties and silent about job titles. Part 211 says changes to master production and control records or other records must be "instituted only by authorized personnel".6 FDA's data integrity questions and answers add that "the system administrator role, including any rights to alter files and settings, should be assigned to personnel independent from those responsible for the record content".4 Guidance for inspectors from PIC/S, the scheme through which GMP inspectorates cooperate, says administrator rights "should not be given to normal users on the system (i.e. segregation of duties)".28

Part 11 asks for "authority checks to ensure that only authorized individuals can use the system, electronically sign a record" and for a determination that people "who develop, maintain, or use" such systems "have the education, training, and experience to perform their assigned tasks".18

Under the EU AI Act as amended, providers and deployers "shall take measures to support the development of AI literacy" of their staff.14

Builder, second confirmer, data owner, quality, administrator: who does what

A sound split keeps building, confirming and administering in different hands

An example split for one regulated use. Your procedure sets yours

No partConsultedDoes itApproves
Describe and draftApprove the test setConfirm a queryReleaseChange a shared ruleGrant accessBuilderDoes itConsultedDoes itNo partConsultedNo partData owner or stewardConsultedApprovesApprovesConsultedApprovesNo partProcess ownerConsultedApprovesConsultedApprovesApprovesConsultedQuality assuranceNo partConsultedConsultedApprovesApprovesNo partSystem administratorNo partNo partNo partDoes itNo partDoes it

Illustrative · replace with your own numbers

Show the numbers
Describe and draftApprove the test setConfirm a queryReleaseChange a shared ruleGrant access
BuilderDoes itConsultedDoes itNo partConsultedNo part
Data owner or stewardConsultedApprovesApprovesConsultedApprovesNo part
Process ownerConsultedApprovesConsultedApprovesApprovesConsulted
Quality assuranceNo partConsultedConsultedApprovesApprovesNo part
System administratorNo partNo partNo partDoes itNo partDoes it
RoleWhat the role doesWhere the duty comes from
BuilderKnows the data and the question; describes, drafts, fixes; trained in the tool and the procedurePart 11.10(i) training; AI Act Article 4 literacy18,14
Second confirmerReads each query and its tests; confirms or sends back; never the builder for a regulated useAnnex 11 clause 6 second check on critical data; Q9(R1) formality1,20
Data owner or stewardOwns the tables and the signed reports used as test answersAnnex 11 clause 2 process and system owners1
Quality assuranceApproves the procedure, the triage of uses and release of GMP-relevant assistants21 CFR 211.22(c); EU GMP quality oversight17
System administratorGrants access and keeps settings; holds no say over the contentFDA data integrity Q&A question 4; PI 041-14,28

One person building, testing and confirming is the gap auditors find first

The tools make it easy for one person to do everything: describe, draft, choose the test answers and confirm. For business reporting that may be fine. For a use that touches a GMP record it breaks the logic of every text above, because the same mind that made an error is asked to find it. Annex 11 already asks, for critical data entered by hand, for "an additional check on the accuracy of the data", which "may be done by a second operator or by validated electronic means".1 In our reading, a test set can serve as that kind of check only if someone other than the builder agreed it.

07 · Sorting uses by risk

Sorting assistant uses by GxP impact: freight cost, document status, review data, batch status

In shortThe same builder can answer a question about freight spend and a question about which procedure version is effective. The second feeds a GMP decision; the first does not. A short triage at intake, on two questions, sorts most uses: how much a quality decision rests on the answer, and how hard a wrong answer would be to notice.

Every text in section 3 sizes the work to the risk. Annex 11 bases the extent of validation on "a justified and documented risk assessment".1 ICH Q9(R1) says formality "is not a binary concept" and "can be considered a continuum".20 FDA's 2018 questions and answers say the extent of validation "should be commensurate with the risk posed by the automated system", and that when one system does both CGMP work (FDA's term, current good manufacturing practice) and other work, "the potential for non-CGMP functions to affect CGMP operations should be assessed".4 An assistant builder is exactly that kind of shared system.

Uses where a quality decision rests on a hard-to-spot answer need the most review
How hard a wrong answer would be to notice →
Test hard, keep it informational
Critical: full validation, and no large language model under the draft
Business reporting: light controls
Review every answer before use
Freight spend by carrierWarehouse capacityTraining due datesEffective procedure versionDeviation counts for annual reviewLots on quality hold
How much a GMP decision or record rests on the answer →

Illustrative · replace with your own numbers

Show the numbers
ItemHow much a GMP decision or record rests on the answer (0–1)How hard a wrong answer would be to notice (0–1)
Freight spend by carrier0.180.28
Warehouse capacity0.300.12
Training due dates0.420.25
Effective procedure version0.700.30
Deviation counts for annual review0.620.72
Lots on quality hold0.760.55

Five triage questions that place an assistant's use

  1. Will the answer be copied into a GMP record, a batch decision, a periodic review or a regulatory document? If yes, the use is GMP-relevant, and Annex 11, Part 211 or the device rule applies to it.
  2. Could a wrong answer have a direct impact on product quality or data integrity? If yes, draft Annex 22 would call it critical, and keeps large language models out of it.7
  3. Does a person check the answer against another source before it counts? FDA's high-risk examples turn on work done "with limited or no additional human awareness or review".3
  4. Does the answer read personal data? Then privacy rules and your data protection officer join the triage.
  5. Who owns the data, and who signs the reports used to test it? No owner, no build.

The answers go on the request, and the request goes in the inventory. Annex 11 clause 4.3 asks for "an up to date listing of all relevant systems and their GMP functionality (inventory)".1 An assistant answering GMP questions belongs on it, with its intended use and its owner.

08 · Why answers go wrong

Why plain-language data answers go wrong, worked through on one freight figure

In shortTurning a question into a correct query is hard on real company data. Published tests show accuracy falling from over 90% on simple teaching databases to about one in five tasks on enterprise data work. Most wrong answers come from a few causes: joins that repeat rows, records that should be left out, the wrong period or currency. A test set of questions with signed-off answers catches them before release.

The same system solved 91% of a teaching benchmark and 21% of enterprise data tasks

Share of tasks solved by one code-writing system built on a large language model (o1-preview), as published

Spider 1.0, teaching databases91.2%
BIRD, larger public databases73%
Spider 2.0, enterprise workflowsreal company schemas21.3%

Source 9

Show the numbers
ItemValue
Spider 1.0, teaching databases91.2%
BIRD, larger public databases73%
Spider 2.0, enterprise workflows21.3%

The Spider 2.0 authors built their test from real enterprise workflows, with large schemas and several steps per task.9 A 2023 study on an enterprise database found a large language model answered 16% of questions correctly when given the tables alone, and 54% when given a map of what the tables mean; its three authors worked at a company that sells such maps, which a reader should weigh.29 Both studies are older than the models on sale in November 2026, and both say the same thing: on company data, the meaning of the tables matters more than the cleverness of the model.

Describing what the tables mean more than tripled correct answers in one published test
Tables onlyTables with a map of their meaning
Questions answered correctly16% correct54% correct

Source 29

Show the numbers
Tables onlyTables with a map of their meaning
Questions answered correctly16% correct54% correct

The worked example: $2,412,300 against $2,291,800

In the fictional example at the top of this page, the September freight report, closed at month-end and signed by the finance data steward, gives total freight spend of $2,291,800: $918,400 to Kestrand Freight, $702,100 to Aerolis Express and $671,300 to Brevalin Logistics. The first draft query returns $2,412,300. The gap is $120,500. It comes from 118 Kestrand Freight invoices that were voided and re-billed in September; the query counted both the void and the re-bill, about $1,021 each on average.

The same error moves the expedited share. Air and courier spend is $412,600 in both cases, but divided by the inflated total it comes to 17.1% against the report's 18.0%.

FigureSigned reportFirst draftWhy
September freight spend$2,291,800$2,412,300118 voided invoices counted beside their re-bills: +$120,500
Expedited spend$412,600$412,600No voided invoice was air or courier
Expedited share18.0%17.1%$412,600 ÷ $2,412,300, the wrong total

A reader would not have caught it: $2.41 million looks as plausible as $2.29 million. A test caught it, because someone had a signed figure to compare against. That is the whole case for test sets.

Five causes behind most wrong figures, and the test that catches each

CauseWhat happensThe test question that exposes it
Fan-out from a joinA shipment has many lines; joining invoices to lines repeats each invoice once per lineA total you already know, for a month with multi-line shipments
Records that belong outVoided, cancelled, test or duplicate records countedA month where voids or cancellations are known to have happened
Currency and periodAmounts summed across currencies, or by invoice date where the report uses ship dateA figure in a second currency; a figure at a month boundary
The wrong definition"Expedited" read as air only, where the business means air or courierA share or ratio, which moves when the definition does
Empty periodsA month with no data returns nothing, and the answer silently drops itA period you know was quiet

Spreadsheets already show what untested business-built logic does

Auditors found errors in 94 of every 100 business spreadsheets they inspected
94 in 100

operational spreadsheets with at least one error, across 85 inspected in six audit studies from 1995 to 2001

Source 30

Show the numbers
MeasureValue
operational spreadsheets with at least one error, across 85 inspected in six audit studies from 1995 to 200194 of 100

A no-code assistant is the spreadsheet's successor in one way: logic built by the person who knows the business, outside the engineering process. Panko's review of audit studies found errors in 94% of 85 operational spreadsheets, and an average cell error rate of 3.9% across 14 laboratory studies.30 The studies are old and the tools are new. The lesson holds: careful people still make errors at a steady rate, and only testing against known answers finds them.

What a fair test set looks like

  • Answers someone else signed. Month-end reports, closed reviews, figures that went to management. The builder should not write the expected answers.
  • At least three questions per query, chosen to hit the causes above: a total, a share, a second period.
  • A tolerance stated before the run. A figure "matches" within a stated margin for rounding, decided in advance.
  • Kept apart from the build. Draft Annex 22 asks, for models in critical uses, that test data not be used in development and that its use be recorded (section 6).7 The draft does not cover large language models; the idea of a test set the builder cannot tune to is still sound.
  • Re-run on a schedule and after every change, because data moves.
09 · Change control

Change control for a business-built assistant: model versions, shared rules, tables and test sets

In shortAn assistant changes in more ways than ordinary software. Its query can change, and so can a rule shared with other assistants, a table upstream, the test set, the prompt and the AI model itself. Annex 11 clause 10, ICH Q10 and draft Annex 22 section 10 all ask that each change be evaluated and controlled. The practical question is which changes trigger which re-test, decided in advance.

Annex 11 clause 10: "Any changes to a computerised system including system configurations should only be made in a controlled manner in accordance with a defined procedure."1 ICH Q10 asks that quality risk management "be utilised to evaluate proposed changes", with "the level of effort and formality of the evaluation" commensurate with the risk, and that the system "provide a high degree of assurance there are no unintended consequences of the change".21

ICH E6(R3) says for trial systems: "Subsequent changes to the system should be validated based on risk".5 Draft Annex 22 asks that "any change to the model itself, the system, or the process in which it is used" be documented and evaluated "to determine if the model needs to be retested", and that "any decision not to conduct such retest should be fully justified".7

A change to a shared rule or the AI model touches every assistant, so it needs the widest re-test

An example re-test rule set. Your procedure decides

Not neededIf the use is GMP-relevantAlways
Re-run this query's testsRe-run all tests of the assistantRe-run every assistant's testsSecond person confirmsQuality approvesOne query editedAlwaysNot neededNot neededAlwaysIf the use is GMP-relevantNew question addedAlwaysNot neededNot neededAlwaysIf the use is GMP-relevantRule shared by all assistantsAlwaysAlwaysAlwaysAlwaysIf the use is GMP-relevantUpstream table changedAlwaysAlwaysIf the use is GMP-relevantIf the use is GMP-relevantIf the use is GMP-relevantTest set changedAlwaysIf the use is GMP-relevantNot neededAlwaysIf the use is GMP-relevantAI model version changedAlwaysAlwaysAlwaysIf the use is GMP-relevantIf the use is GMP-relevant

Illustrative · replace with your own numbers

Show the numbers
Re-run this query's testsRe-run all tests of the assistantRe-run every assistant's testsSecond person confirmsQuality approves
One query editedAlwaysNot neededNot neededAlwaysIf the use is GMP-relevant
New question addedAlwaysNot neededNot neededAlwaysIf the use is GMP-relevant
Rule shared by all assistantsAlwaysAlwaysAlwaysAlwaysIf the use is GMP-relevant
Upstream table changedAlwaysAlwaysIf the use is GMP-relevantIf the use is GMP-relevantIf the use is GMP-relevant
Test set changedAlwaysIf the use is GMP-relevantNot neededAlwaysIf the use is GMP-relevant
AI model version changedAlwaysAlwaysAlwaysIf the use is GMP-relevantIf the use is GMP-relevant

The AI model underneath is a supplier change

Most builders rely on a large language model run by someone else. When that supplier ships a new version, the next draft query for the same question may differ. Annex 11 says the regulated user remains responsible when relying on suppliers, and that formal agreements should state the third party's responsibilities (clause 3).1 Ask the vendor how you learn of a model change, whether you can hold the version, and whether existing confirmed queries change when it does. A builder that stores each confirmed query as text and re-runs that text is easier to control than one that regenerates the query at every question.

Going live straight away versus your release procedure

Some builders put an assistant live the moment it is confirmed, with no deployment step. For business reporting that is the attraction. For a GMP-relevant use, your change procedure decides when it is released and who approves, and the software's "live" button is one step inside that procedure. Configure the tool so that GMP-relevant assistants wait for the approval your procedure names.

Periodic review and drift

Annex 11 clause 11 asks that systems "be periodically evaluated to confirm that they remain in a valid state".1 The draft revision says the frequency should be "established and justified based on the risk".2 No text sets a fixed interval. For an assistant, a nightly re-run of every test, with a re-check whenever a result moves by more than a set margin, supplies evidence between formal reviews. The review itself still needs a person and a record.

10 · Records and audit trail

The records a regulated assistant must leave: who built, tested, confirmed and changed what, and when

In shortPart 11, Annex 11 and FDA's CSA guidance agree on the record's core: who did what and when, kept so it cannot be quietly altered, and readable by an inspector. For an assistant that means the build conversation, every query version, every test run, every confirmation with a name and time, and every change. In FDA's fiscal 2025 drug inspection data, by our count, controls over who may change records are the most cited computer finding.

Part 11 asks for "secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records", and says record changes "shall not obscure previously recorded information".18 Annex 11 clause 12.4 asks systems to "record the identity of operators entering, changing, confirming or deleting data including date and time", and clause 14 that electronic signatures be "permanently linked to their respective record" with the time and date applied.1

The data integrity guidance of MHRA, the UK medicines regulator, lists the ALCOA principles, attributable, legible, contemporaneous, original and accurate, and the "+" for complete, consistent, enduring and available.31

What the record holds for one assistant

RecordHoldsAnswers the inspector's question
Intended use and triageWhat the assistant is for, its GMP impact, its ownerWhy was this level of testing enough?
Build conversationWhat the builder said, what the software proposed, the rules written downWhere did this rule come from?
Query versionsEach query as text, with its version and dateWhich calculation produced the figure in this record?
Test runsQuestion, expected answer and source, actual answer, pass or fail, causeHow do you know it was right?
ConfirmationsWho confirmed which query, when, and what they sawWho decided it could be relied on?
Changes and re-runsEvery rule, table, test set or model change, and the re-test that followedIs it still in a valid state?

Who may change records: the computer finding FDA cited most often in drug plants in fiscal 2025, by our count

Citations under 21 CFR 211.68(b) on drug Form 483s reached 110 in fiscal 2025

Citations under the paragraph on computer controls, input and output checks and backups, by fiscal year

93FY18
102FY19
57FY20
41FY21
83FY22
81FY23
80FY24
110FY25

Source 32 · our sums of FDA's rows

Show the numbers
PeriodValue
FY1893
FY19102
FY2057
FY2141
FY2283
FY2381
FY2480
FY25110

FDA publishes the observations its investigators record on Form 483 each fiscal year. For drug inspections we summed the rows citing 21 CFR 211.68(b), the paragraph on computer and related systems.32 Fiscal 2020 and 2021 were the pandemic years; FDA's files give counts and no reasons. FDA notes that the files are not a complete listing of every observation.32 Our count covers only those rows: a computer problem cited under another section, such as laboratory records, is not in it. We re-read the fiscal 2021 and 2025 rows, and they match.

Eight in ten of those 2025 citations were about who may change records
Changes not limited to authorized people: 87Input and output not checked: 10Backup not exact and complete: 8Backup file not kept: 3No written record of program and validation: 2
110citations, FY25
  • Changes not limited to authorized people87
  • Input and output not checked10
  • Backup not exact and complete8
  • Backup file not kept3
  • No written record of program and validation2

Source 32 · our sums of FDA's rows

Show the numbers
PartValue
Changes not limited to authorized people87
Input and output not checked10
Backup not exact and complete8
Backup file not kept3
No written record of program and validation2

The largest row, 87 of the 110, is the finding that "appropriate controls are not exercised over computers or related systems" so that changes are made only by authorized people.32 A business-built assistant that lets its builder change a shared rule, a query or a test answer without a second person is a new way to earn that observation. In the UK, MHRA's newest published GMP deficiency data, for 2019, held 240 references to Annex 11 among 5,292, counted from its spreadsheet.33

11 · The trend

Business-built AI is spreading faster than the controls around it

In shortSurveys in 2025 and 2026 show most companies experimenting with AI, a growing share of validation teams trying AI for their own work, and unsanctioned AI use rising sharply in data breaches. No public figure counts no-code assistants in life sciences. The direction is clear: people will build, and the choice is whether they build inside a controlled tool or outside one.

39%

of 614 validation professionals, across more than 30 industries, said they were using or evaluating AI for validation, in a 2026 survey sponsored by a validation software vendor.34

Sponsored survey, July 2026
92%

of organizations with an AI-related incident lacked proper AI access controls, in IBM's 2026 breach study as a law firm summarized it.35

Breach study, 2026, read in summary
Q4 2026

is EMA inspectors' target for giving the Commission final text of Annex 11 and Annex 22. A target, with no adoption date.13

EMA work plan
Incidents involving employees' unsanctioned AI more than doubled in a year

Share of incidents involving shadow AI by employees, as a law firm's summary of IBM's 2025 and 2026 reports gives it; we did not read IBM's full report

20%2025 report
43%2026 report

Source 35

Show the numbers
PeriodValue
2025 report20%
2026 report43%

IBM's 2026 study of 602 breaches across 17 industries, between March 2025 and February 2026, found incidents involving unsanctioned employee AI use more than doubled to 43%. We read a law firm's summary and not IBM's full report.35 The figures are about all industries and about security. They point at the same risk a quality team worries about: AI built or used where nobody controls access or keeps a record.

The 2026 validation survey also found 78% of respondents confident AI will be standard practice in validation by 2030.34 No public figure gives the share of life sciences companies that allow business users to build AI assistants, the hours such builds take, or the error rate of their answers. Section 18 lists what you can measure in a week instead.

12 · How teams do it today

How a regulated company answers a new data question today: the request queue, the spreadsheet and the controlled builder

In shortA new data question usually travels from the person who knows the data to a team that knows the tools, and back, several times. In the fictional example the median wait is 31 days, most of it in a queue. Teams that cannot wait build spreadsheets outside any inventory. A third way, a controlled self-service builder, keeps the business person building and the quality controls in place.

Analyst, data engineer, data steward, quality: who handles a new data question

BA

Business or quality analyst

Knows the questions, the tables and the traps. Writes the request, explains the data, checks the result against the report they trust.

Cares about: an answer this month, and one they can defend.
DE

Data or IT engineer

Builds the query or pipeline. Usually does not know which records belong out or how finance counts.

Cares about: a clear specification and a queue that does not grow.
DS

Data steward or report owner

Owns the tables and the signed figures. Asked to agree definitions and to co-sign anything that touches cost, price or quality data.

Cares about: every number tying back to a report that was closed.
QA

Quality assurance and validation

Triage of GMP impact, the procedure, release of regulated tools, the inventory, periodic review.

Cares about: nobody relying on an untested figure in a quality record.
LD

Business lead

Requests new assistants and decides on the evidence they provide.

Cares about: a month for one question being too long.
A new data question crosses three teams before anyone can rely on the answer
  1. 1Request

    The analyst writes up the question and the report it should match.

    Business analystDay 0
  2. 2Queue

    The request waits for a data engineer.

    Data teamWeeks
  3. 3Explain

    The analyst explains tables, join keys and how to count.

    Analyst and engineerDays
  4. 4Build and reconcile

    The engineer builds; the analyst compares with the signed report; fixes go back and forth.

    Engineer and analystDays
  5. 5Release

    For GMP-relevant use, quality reviews and the change is recorded.

    QualityPer procedure

Illustrative · replace with your own numbers

Show the numbers
StepWhat happensWhoTime
RequestThe analyst writes up the question and the report it should match.Business analystDay 0
QueueThe request waits for a data engineer.Data teamWeeks
ExplainThe analyst explains tables, join keys and how to count.Analyst and engineerDays
Build and reconcileThe engineer builds; the analyst compares with the signed report; fixes go back and forth.Engineer and analystDays
ReleaseFor GMP-relevant use, quality reviews and the change is recorded.QualityPer procedure
Twenty of the 31 days go on waiting in the queue

Days for one new analytics question by request, in the fictional example

Waiting in the queueExplaining tables and how to countBuilding the queryReconciling to the signed reportFixes and release
By request20 days3 days4 days3 days

Fictional example from this guide

Show the numbers
Waiting in the queueExplaining tables and how to countBuilding the queryReconciling to the signed reportFixes and release
By request20 days3 days4 days3 days1 day

The 31-day median and its split are the fictional company's own estimate; no public figure measures this wait in life sciences.

Three ways companies handle new data questions

WayStrengthWeakness under GxP
Central data team and a queueEngineering practice, change control and testing in one placeWeeks of waiting; the engineer does not know the data; people decide on old figures meanwhile
Spreadsheets and desktop toolsFast; built by the person who knows the dataOutside the inventory; no audit trail; untested formulas; one copy per person
Controlled self-service builderBuilt by the person who knows the data, tested against signed answers, recordedNeeds a procedure, roles, triage and a supplier assessment before the first build

Many companies set up a small group, sometimes called a center of excellence, that trains builders, keeps the procedure and reviews regulated builds. Whatever the name, the group owns three decisions: who may build, what is reviewed, and how changes are controlled.

Part B · See it working

One freight cost agent in the Botminds No-Code Agent Builder: from first message to first answer, with the failed tests and the signatures

Part A described the work without naming a product. This part shows how Botminds does it, on the fictional case at the top of the page. Every person, carrier, site, report and figure on the screens is sample data. Freight cost is business reporting with no GMP record behind it, which is why it makes a clean first example; section 17 says what changes when the same tool answers a GMP question. The product carries controls that support your compliance. You assess and validate your own use of it, and your people sign.

13 · One build, start to finish

October 6, 10:31 to 10:45 am: a freight cost agent described, tested, fixed, confirmed and live in 14 minutes

In shortLena Ortiz, a supply chain analyst at a medical device manufacturer, describes what people ask her. The Agent Builder proposes six tables and writes down her rules. It takes nine test questions from three signed reports and drafts three skills. Seven tests pass and two fail on one cause, which she fixes in one sentence. She confirms each skill by name, and the agent joins the assistant's routing.

Read this case with Part A in mind. Freight spend sits outside the quality system, so the sample company treats this as business reporting, and one person, the analyst who builds, confirms each skill. Skills that read cost or price tables also need the finance data steward. For a use that feeds a GMP record, your procedure should name a second confirmer and a release approval, and both are settings in the product (section 14).
1

10:31 am: the conversation on the left, an empty agent on the right

The Build screen opens on a blank agent: purpose, data, rules, skills and a test board, each empty. Lena types what people keep asking her: what we spend, with which carrier, per unit on each lane, and how much goes on expedited freight. Below the canvas are agents built here before, with their build times, among them Warehouse capacity in 11 minutes and Supplier delivery in 17.

The first message. The canvas fills in as she answers; nothing is live yet.
2

10:33 am: eight tables found, six highlighted

The Agent Builder searches the supply chain warehouse catalog, read-only, and finds eight tables that look related. It highlights six: freight_invoices with 1.1 million rows, shipment_lines with 96.3 million, carriers, sites, customer_accounts and fx_month_end. Then it asks two questions a careful engineer would ask: how do they join, and which figures do you already trust?

Tables proposed. The agent now has a name and three questions it will answer, on the right.
3

10:36 am: join keys checked, four rules, nine test questions

Lena names the join keys, says expedited means air or courier, and points to the September freight report and the Q3 lane cost review. The builder checks all five join keys against the warehouse and writes down four rules in her words, including one it adds itself: shipment lines are summed per shipment first, so spend is never counted once per line. That is the fan-out error from Part A, section 8, caught before any test runs. It takes nine test questions from three signed reports: the September and August freight reports, signed by Omar Haddad at month-end close, and the Q3 lane review, signed by Sam Patel.

The test board before any test runs. The list of reports counts ten tests (five, two and three) while the board holds nine: an inconsistency in the sample data, shown as it is.
4

10:38 am: three skills drafted, nine tests run, two fail

One click drafts three skills, each a question with the query that answers it: freight spend by carrier, freight cost per unit by lane, and expedited freight spend. The Skill Tester runs all nine tests live and read-only. Seven pass. September spend comes out at $2,412,300 against $2,291,800 in the report, and the expedited share at 17.1% against 18.0%. The Skill Tester finds one cause for both: 118 voided Kestrand Freight invoices that were re-billed in September and counted twice. A failing test blocks Confirm on its skill.

Two fail, one cause. Expected figure, source, the agent's answer and the reason, on each row. Confirm is greyed out on both failing skills.
5

The query behind the failing skill, with no line for invoice status

"Show query" opens the query the first skill ran. A reviewer who reads SQL can see that it filters by invoice month and currency but has no line for invoice status. A reviewer who does not read SQL still has the test row above it, which is the point of testing against known answers.

The query before the fix. Every skill shows the query it will run, next to its tests.
6

10:42 am: one sentence, every test re-run, nine of nine

Lena types: "Yes, leave out voided invoices (status VOID). They were re-billed." The builder adds the rule to all three skills and re-runs every test, the passing ones too. Nine of nine pass: September is $2,291,800 and the expedited share 18.0%, matching the report. The Skill Tester never changes an expected figure and never marks a failing test as passed.

Fixed in plain words. The new rule appears under Rules, and the whole test board turns green.
7

10:42 to 10:43 am: each skill confirmed, with a name and a time

Lena reads each skill with its tests and confirms all three. Each card now reads "Confirmed by Lena Ortiz" with the time. Her confirmation is recorded in the audit trail: which skill, how many tests passed, who, and when.

Confirmed, one skill at a time. Three of three confirmed, each with its passing tests above the confirmation.
8

10:45 am: version 1.0 saved and routed

Go live saves version 1.0: instructions, five rules, three verified skills and nine tests. The agent joins the supply chain assistant's routing. Three probe questions route as expected: Kestrand Freight spend (match 0.94) and air freight share (0.91) go to Freight cost, and the least on-time carrier (0.89) stays with Shipments & carriers. The sample treats this as business reporting, so there is no separate release step; section 9 of Part A explains why a GMP use would wait for one.

Live, with its versions. v0.1 draft at 10:31 am and v1.0 live at 10:45 am, both kept.
9

10:46 am: the first live question

Lena asks the way her colleagues will: what did a unit cost to move from Leiden to Madrid in September? The assistant routes it to Freight cost at a match of 0.95. The answer: $0.84 per unit, $351,900 of freight across 418,900 units on 1,912 shipments, split by carrier, with a note that courier is about 10% of units and 24% of the lane's spend. Below the answer: the skill that produced it, who confirmed it, and "See the query".

The first answer. Kestrand road $231,600, Aerolis courier $84,700, Brevalin road $35,600; together $351,900.
10

"See the query": the void rule highlighted in the calculation that ran

One click opens the exact query that ran, read-only, with the lines that produced the cited number highlighted, including the new rule that leaves out voided invoices. Below it, the result for each lane and the skill's three test questions with their sources. Customer contact columns are hidden from every answer.

The query that ran. Verified by Lena Ortiz, read-only, with its tests underneath.
14 · Agents and sign-offs

Four agents draft, test, route and watch; the analyst and the data steward sign

In shortThe Agent Builder interviews and drafts. The Skill Tester runs every test and explains failures. The supply chain assistant routes questions. The Skill watcher re-runs tests every night. None of them can confirm a skill: a named person does, and skills on cost or price tables need the finance data steward too. Who confirms, how strict the tests are and what the agents may read are settings, recorded when they change.

AgentIts jobWhat it never does
Agent BuilderInterviews the analyst, reads the warehouse catalog, records tables, join keys and rules, drafts skills as question-and-query pairs, takes test questions from trusted reportsPublishes a skill nobody confirmed; writes to the warehouse
Skill TesterRuns every test of every drafted skill live and read-only; compares each answer with its expected figure and source; explains a mismatch in one sentenceChanges an expected figure; marks a failing test as passed
Supply chain assistantMatches each question to an agent by purpose and verified example questions; routes at 0.80 or above, and otherwise asks which area the person meansAnswers without showing the agent, the skill and the query
Skill watcherRe-runs every live skill's tests each night; asks the owner to re-check when a result moves by more than the set marginEdits a skill
Sign-off and tests. "Who confirms a new skill" offers the analyst who builds it, the analyst and a data steward, or a data steward only. For GMP-relevant uses choose two people.
Rules every agent follows, and roles. Each shared rule names who added it and when. A shared rule reaches every agent, so treat a new one as a change.

The sample's sign-off settings, and what a GMP procedure might choose instead

SettingThe sampleFor a GMP-relevant use
Who confirms a new skillThe analyst who builds itThe analyst and a data steward, so builder and confirmer differ
Skills on cost or price tables also needOmar Haddad, data stewardThe owner of the quality data the skill reads
Test questions per skill, at least35, with a total, a share and a second period among them
Confirm only when every test passesOnOn
A figure matches when within0.1%0%, or a tolerance written into the procedure
Ask for a re-check when a result moves more than10%5%, with a named owner for each re-check
Warehouse connectionRead-onlyRead-only, with access granted by an administrator who builds nothing

Every build step, test run, confirmation, rule change and go-live is written to an audit trail with who and when, and every version is kept with the conversation behind it. The audit trail sits in an administrator area this guide does not show.

15 · After go-live

After go-live: twelve agents, their tests every night, a result that moved and a paused agent waiting for its co-signer

In shortThe work goes on after the first build. The Agents screen shows every agent's owner, status, verified skills and accuracy. A skill whose nightly result moves is flagged for its owner to re-check. A new version waits until someone publishes it. An agent whose finance skills lack the steward's signature stays paused.

All twelve agents. Owner, status, verified skills, questions a week, accuracy and version. "Waiting on you" lists the new Freight cost agent, a re-check, a request and a ready version.
Skills and tests. 32 skills: 30 verified, one needing a re-check, one draft.
One skill opened. The query, its tests and sources, verified by Lena Ortiz; first nightly run tonight.
A result moved 14% overnight. 186 backorder lines were re-dated on October 5, a real change. The owner confirms the new result; it is recorded.
v1.4 waits to be published. Two new skills confirmed on October 5 reach the routing only when someone publishes.
Paused until co-signed. Purchase price variance stays off while one finance skill waits for the data steward.

How the sample's 30 live skills were verified

Seven in ten skills passed as drafted; a quarter needed the analyst's fix first
Confirmed as drafted: 71%Fixed by the analyst, then confirmed: 24%Set aside: 5%
30live skills
  • Confirmed as drafted71%
  • Fixed by the analyst, then confirmed24%
  • Set aside5%

Fictional example from this guide

Show the numbers
PartValue
Confirmed as drafted71%
Fixed by the analyst, then confirmed24%
Set aside5%

The dashboard in the sample shows how skills reached the agents over the last 90 days: 71% confirmed as drafted, 24% fixed by the analyst and then confirmed, 5% set aside. A quarter of skills needing a fix is the reason tests exist. The figures are the sample's.

The dashboard. Median 18 minutes to a new agent across 11 builds, 7,583 questions answered in 90 days, and 30 of 30 live skills passing their tests.
Every agent in the sample went from first message to live in under half an hour

Minutes from first message to live, by agent

Warehouse capacity11 min
Expiry and short-dated stock12 min
Freight costthe build in this guide14 min
Document status15 min
Supplier delivery17 min
Inventory18 min
Purchase price variance19 min
Demand forecast accuracy20 min
Backorders22 min
Quality holds24 min
Shipments and carriers26 min

Fictional example from this guide

Show the numbers
ItemValue
Warehouse capacity11 min
Expiry and short-dated stock12 min
Freight cost14 min
Document status15 min
Supplier delivery17 min
Inventory18 min
Purchase price variance19 min
Demand forecast accuracy20 min
Backorders22 min
Quality holds24 min
Shipments and carriers26 min
16 · In and out

A plain-language request and a signed report in; a tested, versioned agent and its record out

In shortThe analyst brings knowledge and a report she trusts. The warehouse stays read-only and stays where it is. What comes out is a live agent, verified skills with their queries and tests, versions and an audit trail. The design targets are labeled as targets, and the one customer result is quoted as the solution page states it.

From the analyst

  • What people ask, in plain words
  • Which tables, how they join, how to count
  • Which records belong out
  • The reports she already trusts

From the company

  • The warehouse catalog, read-only
  • Signed-off reports used as expected answers
  • Your sign-off rules, tolerances and roles
  • Rules every agent follows
Agents draft, test, route and watch Every skill tested against signed figures before anyone confirms it
A named person confirms every skill
Every answer opens its query

What comes out

  • A live data agent in the assistant's routing
  • Verified skills, each with its query and test results
  • Versions, each with the conversation behind it
  • An audit trail of every build step, test, confirmation and change

What changes for analysts and stewards

  • The analyst builds what she knows
  • The steward signs figures once, as test answers
  • Planners and buyers ask in plain words, with the query one click away

Warehouse, signed reports, item master, exchange rates: what the agents read

Data warehouseRead-only: inventory, backorders, purchase orders, shipments
Signed-off reportsMonth-end and quarterly reports, as expected answers
Freight invoices and carriersAmount, mode, currency, month, status
Shipments and sitesLines by site, item and account
Item masterItem code, product family, standard cost
Month-end exchange ratesSpend reported in US dollars
Identity and accessRoles: builder, data steward, lead

Design targets, the sample build, and one customer result

18 min

median from first message to a live agent.

Design target
14 min

from the analyst's first message to a live freight cost agent, in the sample build shown above.

Sample build
100%

of live skills pass their test questions before they go live.

Design target

Targets are design goals, measured in the live solution on your data. One customer result, in the solution page's words: "Delivered at a global medtech manufacturer: business users create and wire live data agents in minutes through plain-English conversation." The company, people and figures in the sample build are fictional and are not that customer's.

17 · Before a GMP use

What to set and decide before the same builder answers a GMP question

In shortThe freight build shows the mechanics. A GMP-relevant use, such as which version of a procedure is effective, or how many lots sit on quality hold, needs more around the same mechanics: a triage on record, two people confirming, a release step under your change procedure, a re-test rule for shared rules and model changes, and your own assessment of the supplier and of your use.

In the sampleBefore a GMP useWhy (Part A)
The analyst who builds confirmsSet confirmation to the analyst and a data stewardSection 6: builder and checker differ
Live the moment it is confirmedRelease under your change procedure, with quality approval where it names oneSection 9: Annex 11 clause 10, Q10
A shared rule applies to every agent from the next questionTreat a shared rule as a change; re-run every agent's tests before it appliesSection 9: a change reaching every assistant
The builder names the trusted reportsThe report owner agrees the test set before the buildSection 8: a test set the builder cannot tune to
Drafts written with a large language modelRecord the use as non-critical and a qualified person's review of each output, or keep it outSection 5: draft Annex 22, section 1
Supplier controlsAssess Botminds as a supplier; write the intended use; validate your useSections 3 and 4: Annex 11 clause 3, CSA intended use

Botminds holds a SOC 2 Type II audit and manages information security to ISO 27001. Those are statements about the supplier's controls. They do not validate your use, and no software can be certified for GxP, Part 11 or Annex 11. The product's version history, audit trail, roles and test records are controls that support your compliance; your people decide and sign.

Part C · Are you ready to automate?

Letting business and quality teams build AI agents at your company: the cost of the queue and of a wrong figure, your return, and how to start safely

You run quality, validation or a data team, or you are the analyst who keeps getting the same questions. This part is written for you: what the wait and an untested figure cost, what a governed builder could return, whether your company is ready to let people build, and what to ask any vendor of no-code agent builders, including us.

18 · What it costs you

A month in the queue and a confident wrong figure: the two costs to weigh before you let people build

In shortTwo costs pull against each other. The queue costs weeks on every new question, while people decide on old figures. The shortcut, an untested AI answer or a private spreadsheet, costs a wrong number nobody catches, and in a regulated company a finding when it reaches a record. Nobody publishes either cost for life sciences, so the numbers that matter are the ones you can find in a week.

21.3%

of enterprise data tasks solved by a leading code-writing AI system on the Spider 2.0 benchmark.9

Published benchmark, 2024, revised 2025
87

drug Form 483 citations in fiscal 2025 for computer records changed without controls limiting it to authorized people, by our sum of FDA's rows.32

FDA data
40%+

of agentic AI projects could be canceled by the end of 2027, Gartner forecast in June 2025, citing costs, unclear value and inadequate risk controls. We read a news report of the forecast, not Gartner's own release.36

Analyst forecast

The market is moving faster than the evidence. Gartner published an Emerging Market Quadrant for No-Code Agent Builders, for startup vendors, on June 11, 2026; we read its title and date in a listed vendor's announcement, not the paid report.37 In McKinsey's 2025 survey of 1,993 participants across industries, 62% said their organizations were at least experimenting with AI agents and 23% were scaling them somewhere in the enterprise; we read a news report of the survey, not McKinsey's own report.38 Neither figure is specific to life sciences.

In the example, the month goes on waiting and explaining, and the build itself takes minutes

Days per new question, by request and with the builder, in the fictional company

By requestWith the builder
Waiting in the queue20 days0 days
Explaining tables and how to count3 days0.02 days
Building the query4 days0.02 days
Reconciling to the signed report3 days0.04 days
Fixes, re-tests and release1 day0.02 days

Fictional example from this guide

Show the numbers
By requestWith the builder
Waiting in the queue20 days0 days
Explaining tables and how to count3 days0.02 days
Building the query4 days0.02 days
Reconciling to the signed report3 days0.04 days
Fixes, re-tests and release1 day0.02 days

The split is the fictional company's estimate. For a GMP-relevant use, add your review and release time to the right-hand column: confirmation by a second person and quality approval take what your procedure takes.

Where the cost of new data questions hides at your company, and how to find it in a week

CostWhere it hidesHow to find your number
The waitThe data team's ticket queueMedian days from request to first trusted answer, for the last 20 requests
Explaining the dataMeetings between analyst and engineerAsk three analysts how many hours their last request took them
ReconciliationBack and forth until the figure matches the signed reportCount the versions sent before sign-off
Private spreadsheetsShared drives, email attachmentsSearch a shared drive for workbooks with queries or links to the warehouse
Unsanctioned AIPersonal accounts on public chat toolsAsk IT security for traffic to public AI services; run an anonymous survey
Wrong figures caught lateCorrections, restated reports, deviationsCount corrections to management or quality reports in the last year, and their cause

An inspection question, an analyst who quits, a decision on old figures: costs a calculator leaves out

The question nobody can answerAn inspector asks where a figure in a quality record came from. A private spreadsheet, built by someone who has left, has no answer.
Decisions on last quarter's numbersWhile a question waits a month, carriers, lanes, stock and holds are decided on whatever figure was to hand.
The analyst's patienceThe person who knows the data and cannot use it is the person most likely to build something outside the rules.
The confident wrong total$2.41 million looks as plausible as $2.29 million. A figure that is wrong and believed costs more than one nobody trusts.
Engineering time on simple questionsEvery hour an engineer spends on a sum is an hour away from the pipelines only engineers can build.
19 · Your return

Your return from a governed agent builder: engineering hours back, review time paid, fewer wrong figures

In shortPut in your own numbers. The return comes from engineering and analyst hours on new questions, less the review time a governed build still needs, plus whatever wrong figures you stop. The calculator counts review time on purpose, and takes away what the software would cost you. Every default is a placeholder.

Engineering time, review time, wrong figures, the wait: where a governed builder pays

The engineering

Hours back on simple questions

Questions the analyst can now answer herself, tested and confirmed.

questions × (engineering and analyst hours today − build hours) × hourly cost
The review

Review time, subtracted

Second confirmer and quality release still take time, and the calculator subtracts it.

questions × review hours × hourly cost
The figures

Wrong figures caught before use

Errors a test set catches that a reader would not.

errors avoided a year × cost of one correction
The wait

Decisions on this month's numbers

Real, and left out of the calculator because its value depends on the decision.

days earlier × value of a better decision per day
The spreadsheets

Fewer private workbooks

Logic moved into a tool with versions and an audit trail.

workbooks retired × hours a year each one costs to keep
The inspection

A figure you can trace

The query, its tests and who confirmed it, one click from the answer.

questions answered from the record ÷ all questions asked about it

A calculator for new data questions, review hours and corrections at your company

Every default is a placeholder: replace it with your own. The review hours default assumes a second confirmer and a quality release for some builds.

New data questions
Wrong figures
Cost
Illustrative yearly value
Hours freed, after review time0
Hours freed, valued$0
Corrections avoided, valued$0
After the yearly cost$0

Formula: questions × (hours today − build hours − review hours) × hourly cost + corrections avoided × cost of one − yearly cost. If build and review together take as long as today's route, the first line is zero and the case rests on the wait and the corrections.

20 · Readiness check

Is your company ready to let business and quality teams build AI agents? Score eight questions

In shortEight questions, 0 to 2 each. Your score suggests whether to write the procedure first, pilot on one business-reporting use, or plan a rollout that includes GMP-relevant uses. Nothing you choose leaves this page.

1
Who may build, in writing

Does a procedure say who may build an assistant, for which kinds of use, with what training?

Why it mattersPart 11 and the AI Act both expect trained people; an auditor will ask who allowed the build.
A strong answer"A named builder role, a short course, and a list of uses each role may build."
2
Sorting uses by GMP impact

Can you sort a proposed use as business reporting, GMP-relevant or critical in ten minutes, and record it?

Why it mattersEverything after it, including whether a large language model may draft at all, depends on that call.
A strong answer"A five-question triage on the request form, signed by quality for anything GMP-relevant."
3
Builder and confirmer differ

For regulated uses, is a second person required to confirm, and is administration held by someone who builds nothing?

Why it mattersBy our count of fiscal 2025 drug inspections, the finding FDA cites most often under 211.68(b) is changes not limited to authorized people.
A strong answer"Steward confirms, quality releases, IT grants access."
4
Signed answers to test against

Do the reports people trust have an owner and a sign-off, so they can serve as expected answers?

Why it mattersWithout a signed figure there is nothing to test against, and a fluent wrong answer passes.
A strong answer"Month-end and quarterly reports, each with an owner and a closing signature."
5
Read-only data, with an owner

Is there a read-only connection to the warehouse, with a catalog someone keeps current?

Why it mattersA builder that cannot write to the data cannot damage it; a stale catalog gives wrong tables.
A strong answer"A read-only role, a catalog with table owners, and personal data masked."
6
Change control that covers AI

Does your change procedure treat a shared rule, a test set and an AI model version as changes with a re-test rule?

Why it mattersAnnex 11 clause 10 and draft Annex 22 section 10 both expect it, and a model update can change every answer.
A strong answer"A table of change types, each with the re-test it triggers and who approves."
7
Records and inventory

Would a GMP-relevant assistant go on your computerized system inventory, with its intended use, owner and record retention?

Why it mattersAnnex 11 asks for an up-to-date inventory; Part 11 asks for audit trails kept as long as the records.
A strong answer"Yes, as a row per assistant, linked to its versions and audit trail."
8
A baseline you measured

Do you know the median wait for a new data question and how many corrections reports needed last year?

Why it mattersNo public benchmark exists, so a pilot can only be judged against your own baseline.
A strong answer"Median 4 weeks over the last 20 requests; 6 corrections, 4 from wrong joins."
9
The questions you would ask first · for reflection

If the people who know your data could answer their own questions next month, which three would they build first, and which of those would you keep away from any GMP record until the final Annex 22 is out? Not scored.

From an unwritten builder role to a rollout with GMP uses: what each band means

0–6

Explore

Write the builder role and the triage, name the report owners, set up read-only access, and measure your wait. All of it pays off before any software arrives.

7–11

Ready to pilot

Start with one business-reporting use and one report owner. Then one GMP-relevant use with two confirmers and a release under change control.

12–16

Ready to implement

Plan roles across sites, connections to the warehouse and identity system, the re-test rules, your supplier assessment and the validation of your use.

21 · Questions for any vendor

Questions for any no-code AI agent builder vendor: tests, sign-off, model changes, records and claims

In shortAsk every vendor the same questions, us included. The answers show whether the tool tests against figures you already trust, whether a second person can be required, what happens when the AI model underneath changes, what record it keeps, and whether the vendor claims a certification no software can have.

AskA good answerA warning sign
Tests and answers
How do I know an answer is right before anyone relies on it?Each query runs against expected answers from signed reports; a failure blocks confirmation"The model is very accurate"
Can a reader see the query behind every answer?One click, the exact query that ran, with its testsA summary of the reasoning, no query
Is the confirmed query stored and re-run, or regenerated at each question?Stored as text and re-run; a new draft only on a changeRegenerated each time
Who signs
Can I require a second person to confirm, by data area?A setting, with named co-signers for chosen tablesOnly the builder confirms
Can a GMP-relevant agent wait for a release approval?Yes, and the approval is recordedLive on confirmation, always
Can the person who grants access be kept out of building?Separate rolesBuilders are administrators
Changes
How do I learn of a change to the AI model you use, and can I hold the version?Notice in advance, a named version, re-runs before switching"We always use the latest model"
What happens to live agents when a shared rule is added?Tests re-run before the rule appliesApplies at the next question with no re-test
What happens when a warehouse table changes?Nightly re-runs flag moved results for the ownerNobody knows until a reader notices
Records and claims
Which record does each build leave, and can I export it?Conversation, versions, tests, confirmations and changes, with who and when, exportableChat history only
Is your product validated or GxP compliant?"No software can be certified compliant, and validation is of your use. Here are our controls and our audit reports.""Yes, it is Part 11 and Annex 11 compliant"
How do you read draft Annex 22 for a builder that drafts with a large language model?Non-critical uses only, with a qualified person answering for each output, until the final text"Annex 22 does not apply to us"
22 · A pilot

A ten-week pilot of a governed agent builder: one reporting use, one GMP-relevant use, five numbers

In shortStart with your procedure and a baseline. Build one business-reporting agent first, where a wrong answer costs little. Then one GMP-relevant use with a second confirmer and release under change control. Have someone who was not involved read the record. Decide on the numbers below.

  1. Weeks 0 to 2. Write the builder role, the triage and the re-test rules. Measure the wait on your last 20 requests.
  2. Weeks 2 to 4. Assess the vendor as a supplier. Set up read-only access. Agree test sets with two report owners.
  3. Weeks 4 to 6. One business-reporting agent, built by an analyst. Count failed tests, causes and minutes.
  4. Weeks 6 to 9. One GMP-relevant use, such as effective document versions, with two confirmers and a release under your change procedure.
  5. Weeks 9 to 10. An auditor who was not involved reads the record. Decide, and write down why.
A ten-week pilot moves from a reporting use to a GMP-relevant one before deciding
Your teamWith the vendor
Procedure, triage, baseline
Supplier assessment; read-only access; test sets agreed
One business-reporting agent
One GMP-relevant use, two confirmers
Independent read of the record
Decide, and record why
Week 0246810

Illustrative · replace with your own numbers

Show the numbers
ActivityFromTo
Procedure, triage, baseline02
Supplier assessment; read-only access; test sets agreed24
One business-reporting agent46
One GMP-relevant use, two confirmers69
Independent read of the record8.59.5
Decide, and record why9.510

Failed tests caught, time to a confirmed answer, record questions answered: scoring the pilot

MeasureHowWhy
Errors caught by testsFailed tests on first run, with their causeShows what a reader would have missed
Errors that got past the testsWrong figures found later by peopleThe number that matters most; target zero
Time to a confirmed answerFirst message to confirmation, and to release for the GMP useAgainst your baseline wait
Review hoursConfirmer and quality time per agentThe cost a governed build keeps
Record questions answeredThe auditor's questions answered from the record aloneWhether an inspector could follow it
Deep questions to settle before you scale

Draft Annex 22. If the final text keeps large language models out of critical uses, which of your planned agents move to the other side of the line, and who decides?

Drafts and confirmed queries. If the model drafts and a person confirms a stored query, is the model part of the GMP use at all, or only the tool that wrote the query? Write your reasoning down and test it with quality.

Retirement. When a builder leaves, who owns their agents, and how fast is that re-assigned?

Shared rules. Who may add a rule every agent follows, and what re-test applies?

Reliance on the tests. How will you know, after six months, that confirmers still read the queries? Sample it.

Personal data. Which tables hold it, and how is it kept out of answers by default?

23 · Questions people ask

No-code AI agent builders in pharma and medtech: CSA, Annex 22, citizen developers and validation

In shortShort answers to what quality, validation and data teams search for, each complete on its own. Rules and statuses are as of November 2026.

What is a no-code AI agent builder?

A no-code AI agent builder lets a person who knows the data create an AI assistant by describing it in plain words, without writing code. For data questions it reads the catalog, proposes tables and joins, and drafts a query for each question. In a regulated company the useful ones also test each query against signed-off figures and record who confirmed it.

Can business users build AI agents in a GxP environment?

Yes, if your procedures allow it and the controls follow the use. None of the GMP texts this guide reads forbids a business user from building; in our reading, the duties attach to access, testing, change control and records. For a GMP-relevant use, a second person should confirm, administration should sit with someone who builds nothing, and release should follow your change procedure.

Does FDA's computer software assurance guidance apply to AI agents?

For a medical device maker, it can. FDA's final guidance of February 3, 2026 covers software used in device production or the quality management system, and names AI tools, data analytics tools and automated workflows among the software its framework can reach. It does not cover drug manufacturing or clinical systems by its terms; drug makers can borrow its method as an analogy.

Has the EU adopted GMP Annex 22 on artificial intelligence?

No. On November 3, 2026, Annex 22 on artificial intelligence was still a draft; EudraLex Volume 4 does not list it. The consultation ran from July 7 to October 7, 2025, and EMA's inspectors set the fourth quarter of 2026 as their target for giving the Commission final text, with no adoption date.

Does draft Annex 22 allow large language models in GMP?

The consultation draft says generative AI and large language models should not be used in critical GMP applications, those with direct impact on patient safety, product quality or data integrity. In non-critical applications, qualified people should always be responsible for ensuring each output suits the intended use. A June 2026 EMA workshop discussed widening the scope, and no revised text is published.

Is a no-code AI tool GxP compliant or validated?

No software can be certified as GxP compliant, Part 11 compliant or Annex 11 compliant. A tool can carry controls that support your compliance, such as audit trails, roles, version history and test records. You assess the supplier, write the intended use and validate your own use, and your people sign.

How do you validate an AI assistant that answers data questions?

Start with the intended use and a risk call on what a wrong answer could do. Test each query against expected answers from reports someone else signed, with a tolerance set in advance and at least three questions per query. Record the tests, who confirmed them and when, and re-run them after every change and on a schedule.

What is a golden test set for text-to-SQL?

A golden test set is a list of questions whose correct answers are already known, usually from signed reports, used to check that a generated query returns the right figure. Good test sets include a total, a share and a second period, because those expose duplicated rows, wrong definitions and missing records. The builder should not write the expected answers.

How accurate is text-to-SQL on enterprise data?

Text-to-SQL is far less accurate on enterprise data than on teaching databases. The Spider 2.0 benchmark reported one code-writing system solving 91.2% of Spider 1.0 tasks and 21.3% of enterprise workflow tasks. A 2023 study found correct answers rose from 16% to 54% when the model was given a map of what the tables mean.

What change control does an AI agent need?

An AI agent needs change control over every change that could alter an answer: a query, a rule shared by several agents, an upstream table, the test set and the AI model version. Annex 11 clause 10 asks that changes be made in a controlled manner under a defined procedure, and draft Annex 22 asks that a decision not to re-test be fully justified. Decide in advance which change triggers which re-test.

Who should confirm an AI-generated query before it is used?

For business reporting the builder may be enough. For a GMP-relevant use, someone other than the builder, usually the data owner or steward, should confirm, and quality should approve release where your procedure says so. In a 2026 warning letter, FDA told one drug manufacturer that, if it resumed using AI for CGMP activities, the output must be reviewed and cleared by an authorized person in its quality unit.

Does the EU AI Act apply to an internal data assistant?

Every provider and deployer must take measures to support the AI literacy of its staff, under Article 4 as amended by Regulation (EU) 2026/1744. In our reading, most internal data assistants fall outside the high-risk lists, whose rules now apply from December 2, 2027 and August 2, 2028. Whether a given use is high-risk is a question for counsel.

24 · One-page summary

Governed no-code AI agents at a glance: who builds, what is tested, how changes are controlled, and 14 terms

Print this page

In shortThe whole guide on one screen: the method, the rules as of November 2026, the readiness bands and the terms.

The method
  1. Sort the use: business reporting, GMP-relevant or critical. Record it.
  2. Name the builder, the confirmer and the administrator. Different people for regulated uses.
  3. Agree the test set with the report owner before the build.
  4. Test every query against signed answers, with a tolerance set in advance.
  5. Confirm each query by name. A failing test blocks confirmation.
  6. Release GMP-relevant agents under your change procedure.
  7. Treat shared rules, tables, test sets and model versions as changes, each with its re-test.
  8. Re-run tests on a schedule; review periodically; keep the record.
The readiness bands

0–6 Explore · 7–11 Ready to pilot · 12–16 Ready to implement.

The rules, November 2026

FDA CSA final February 3, 2026, device production and quality systems · QMSR in effect February 2, 2026 · Annex 11 at its 2011 text · revised Annex 11 and new Annex 22 drafts, final text targeted Q4 2026 · draft Annex 22: no large language models in critical GMP uses · FDA AI draft January 2025, still a draft · AI Omnibus in force July 27, 2026; high-risk rules December 2, 2027 and August 2, 2028.

14 terms
No-code AI builder
Software that builds an AI assistant from a plain-language description.
Text-to-SQL
Turning a question in words into a database query.
Skill
One question with the query that answers it, in the product shown in Part B.
Test set
Questions with known answers from signed reports.
Confirmation
A named person's recorded decision that a query is right for its use.
Fan-out
A join that repeats rows, so totals count some records several times.
GMP-relevant use
A use whose answers feed a GMP record or decision.
Critical GMP application
Draft Annex 22: direct impact on patient safety, product quality or data integrity.
Human in the loop
A qualified person who reviews each output before it counts.
CSA
Computer software assurance: FDA's risk-based approach for device production and quality systems.
Intended use
What the software or function is used for in your process.
Segregation of duties
Building, confirming and administering held by different people.
Audit trail
A time-stamped record of who created, changed or deleted what.
ALCOA+
Attributable, legible, contemporaneous, original, accurate, plus complete, consistent, enduring and available.
25 · Sources

Sources

Accessed November 2026. Regulatory status statements are as of November 2026. Vendor and sponsored surveys are labeled as such, and where we read a summary in place of the original the entry says so.

  1. European Commission, EudraLex Volume 4, Annex 11: Computerised Systems, revision 1, in operation from June 30, 2011. health.ec.europa.eu
  2. European Commission and PIC/S, draft Annex 11: Computerised Systems, consultation text, 2025. A draft, not in force. health.ec.europa.eu
  3. FDA, "Computer Software Assurance for Production and Quality Management System Software", final guidance, issued February 3, 2026, superseding the version of September 24, 2025; contains nonbinding recommendations; docket FDA-2022-D-0795. fda.gov
  4. FDA, "Data Integrity and Compliance With Drug CGMP: Questions and Answers", final guidance, December 2018, questions 3 and 4; nonbinding. fda.gov
  5. ICH, E6(R3) Good Clinical Practice, final, January 6, 2025, section 4.3.4 on validation of computerized systems. database.ich.org
  6. 21 CFR 211.68, Automatic, mechanical, and electronic equipment, 2025 edition (revised as of April 1, 2025), the newest on govinfo on November 3, 2026. govinfo.gov
  7. European Commission and PIC/S, draft Annex 22: Artificial Intelligence, consultation text, 2025. A draft, not in force. picscheme.org
  8. European Medicines Agency, report of the multistakeholder workshop of June 30, 2026 on expert contributions to Annex 22, EMA/156789/2026. Industry and regulator discussion, no revised text. ema.europa.eu
  9. Lei and colleagues, "Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows", arXiv 2411.07763, revised March 17, 2025. Results for one model framework at that date. arxiv.org
  10. FDA, warning letter 320-26-58 (MARCS-CMS 722591) to a drug manufacturer, Purolea Cosmetics Lab, April 2, 2026. One firm, with other CGMP findings, which had ceased drug production. fda.gov
  11. FDA, "Medical Devices; Quality System Regulation Amendments", final rule, 89 FR 7496, February 2, 2024, effective February 2, 2026. govinfo.gov
  12. European Commission, stakeholders' consultation on EudraLex Volume 4, Chapter 4, Annex 11 and new Annex 22: opened July 7, 2025, closed October 7, 2025. health.ec.europa.eu
  13. European Medicines Agency, GMP/GDP Inspectors Working Group three-year work plan 2026 to 2028: Annex 11, Annex 22 and Chapter 4, target date Q4 2026 to give the Commission final text. ema.europa.eu
  14. Regulation (EU) 2026/1744 (AI Omnibus), Official Journal, July 24, 2026, replacing Article 4 of the AI Act on AI literacy. eur-lex.europa.eu
  15. European Commission, "AI Act", read November 3, 2026: amendment in force July 27, 2026; Annex III rules from December 2, 2027; Annex I product rules from August 2, 2028. digital-strategy.ec.europa.eu
  16. FDA, guidance page for Computer Software Assurance, content current February 3, 2026, listing medical devices as the regulated product. fda.gov
  17. 21 CFR 211.22, Responsibilities of quality control unit, 2025 edition. govinfo.gov
  18. 21 CFR 11.10, Controls for closed systems, 2025 edition. govinfo.gov
  19. European Commission, EudraLex Volume 4 page, read November 3, 2026: lists Annex 11 at its January 2011 revision and no Annex 22. health.ec.europa.eu
  20. ICH, Q9(R1) Quality Risk Management, final, January 2023, section 5.1 on formality. database.ich.org
  21. ICH, Q10 Pharmaceutical Quality System, final, June 2008, section 3.2.3 on the change management system. database.ich.org
  22. FDA, "Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products", draft guidance, January 2025, docket FDA-2024-D-4689; page content current January 6, 2025. fda.gov
  23. Regulation (EU) 2024/1689 (AI Act), Official Journal, July 12, 2024, Article 3 definitions of provider, deployer and putting into service. eur-lex.europa.eu
  24. ISPE, GAMP 5 Guide, 2nd Edition, July 2022, table of contents listing Appendix D11, Artificial Intelligence and Machine Learning, and Appendix O8, Periodic Review. The guide is sold: we read its contents page. ispe.org
  25. ISPE, GAMP Guide: Artificial Intelligence, July 2025. The guide is sold: we read ISPE's page. ispe.org
  26. European Medicines Agency, reflection paper on the use of artificial intelligence in the medicinal product lifecycle, adopted September 2024. ema.europa.eu
  27. FDA, CDER Guidance Agenda, July 2026 posting, page current July 15, 2026: lists a planned supplement on computer software assurance for AI-based systems in drug manufacturing and clinical investigations. CDER is not bound to issue what the list names. fda.gov
  28. PIC/S, PI 041-1, Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments, July 1, 2021, on administrator rights and segregation of duties. picscheme.org
  29. Sequeda, Allemang and Jacob, "A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases", arXiv 2311.07509, November 13, 2023. The authors worked at a data catalog company. arxiv.org
  30. Panko, "What We Don't Know About Spreadsheet Errors Today", Proceedings of the EuSpRIG 2015 Conference, Table 2: 85 operational spreadsheets, audited studies from 1995 to 2001. arxiv.org
  31. MHRA, "GXP Data Integrity Guidance and Definitions", March 2018, section 3.10 on ALCOA and ALCOA+. assets.publishing.service.gov.uk
  32. FDA, Inspection Observations, fiscal years 2018 to 2025 spreadsheets, Drugs tab. Sums of the rows under 21 CFR 211.68(b) are ours. FDA says the files are not a complete listing of every observation. fda.gov
  33. MHRA, Good manufacturing practice inspection deficiencies, 2019 data, the newest published, page updated October 15, 2020. Count of Annex 11 references is ours. gov.uk
  34. "2026 State of Validation Report", sponsored content from Kneat, a validation software vendor, in ISPE's Pharmaceutical Engineering, July 1, 2026, 614 professionals. ispe.org
  35. Baker Donelson, "Ten takeaways from IBM's 2026 Cost of a Data Breach Report", July 30, 2026: 602 breaches, March 2025 to February 2026, 17 industries. We read this summary and not IBM's full report. bakerdonelson.com
  36. W.Media, June 27, 2025, reporting a Gartner press release: over 40% of agentic AI projects could be canceled by the end of 2027. A forecast. We read this news report, not Gartner's release. w.media
  37. Gartner, Emerging Market Quadrant for No-Code Agent Builders, Startup Vendors, June 11, 2026. A paid report: we read its title and date in a listed vendor's notice and did not read the report. pipefy.com
  38. IDM, November 6, 2025, reporting McKinsey, "The State of AI in 2025": 1,993 participants, 105 nations, June 25 to July 29, 2025; all industries. We read this news report, not McKinsey's own report. idm.net.au
Next step

See it on a report you already trust.

Explore the solution, or bring one signed-off report and the questions people keep asking about it, and we will build a first agent with your analysts, tested against that report, with your own people signing.

botminds.ai The complete guide to no-code AI agent builders in regulated life sciences · November 2026. Lena Ortiz, Sam Patel, Omar Haddad, Dana Okafor, Kestrand Freight, Aerolis Express, Brevalin Logistics, every site, account, report and figure in the example and the screens are fictional. Regulatory status is as of November 2026; check the sources before relying on it. This guide is general information and is not legal advice. Software cannot be certified for GxP, 21 CFR Part 11 or EU GMP Annex 11: it can carry controls that support your compliance. You assess and validate your own use, and your people sign.