An AI vendor risk assessment questionnaire should ask what AI the product contains, what data trains it, how it is tested, what changes it may receive, and what rights the insurer has to audit it and to the vendor's help with regulators. For an insurer the questionnaire is the weaker half of the assessment. The vendor answers it once; the contract, its amendments and each release note decide what the product actually does next quarter, and regulators hold the insurer, not the vendor, responsible for the result. The stronger practice is to answer the questionnaire from those documents and read every new one as it arrives, which is the job the AI finder does in Botminds' AI Governance & Model Inventory.
The rules leave no doubt about who carries the risk. New York's Circular Letter No. 7 puts it in one sentence, and the NAIC bulletin and Colorado's regulation say the same in their own words.
“Insurers retain responsibility for understanding any tools, EDCIS, or AIS used in underwriting and pricing … that were developed or deployed by third-party vendors …”
NY DFS Insurance Circular Letter No. 7 (11 July 2024), ¶35
What should an AI vendor risk assessment questionnaire ask?
It should ask the questions a department will later ask the insurer. The NAIC bulletin lists them: due diligence on the vendor and its data, models or AI systems; the contracts, including terms on representations, warranties, data security and privacy, data sourcing, intellectual property, confidentiality and cooperation with regulators; audits of the vendor's compliance; and validation, testing and drift documentation (NAIC Model Bulletin, Section 4, items 2.1 to 2.4). New York adds written standards for vendor oversight and contract terms giving audit rights and cooperation with regulatory inquiries (CL7 ¶36–37). Colorado says that for life, private passenger auto and health insurers using external consumer data, “the insurer remains responsible for ensuring all requirements in Section 5.A. are met,” and must document a process for selecting and overseeing vendors (3 CCR 702-10, 5.B).
Turned into questions, that is a short list: what AI the product contains and may contain; what data it was trained on, including any of the insurer's own; how it was validated and how it is monitored for drift; whether it has been tested for unfair discrimination and how; what notice the insurer gets before a model changes; what rights the insurer has to audit the model, not just the vendor's data security; and whether the vendor will cooperate with the insurer's regulators. The checklist later on this page sets out where each answer is found.
Why a questionnaire alone misses vendor AI
Because the AI often arrives after the questionnaire. A claims or policy platform licensed years ago gains a recommendation feature in a release; a call-centre product starts summarising calls; an intake tool adds a generative summary. None of these is a new purchase, so none triggers a new assessment, and the annual survey of business units asks people who may not know the feature is on. In the NAIC's homeowners survey, models used to evaluate images and for speech analysis “tend to be developed by third parties” (NAIC Home AI/ML Survey memo, 2023), and those are the features most often shipped inside platforms.
The contract usually says it first. Software agreements increasingly reserve the right to add AI features, to train them on the customer's data pooled with other customers' and to change them by release. A questionnaire answered in January says nothing about an amendment signed in August. The documents that matter arrive through four doors: the contract repository, vendor release emails and portals, accounts payable and the vendor's own questionnaires and assurance reports. Reading all four as they arrive is the only way to know what changed.
The same vendor question, answered by the questionnaire and by the documents
| Question | What a questionnaire typically says | What the documents show |
|---|---|---|
| Does the product use AI? | Describes the AI the vendor was marketing on the day it answered | The contract’s right to enable features; each release note that switches one on |
| What data trains it? | Customer data is protected and never sold | The data-use clause: whether de-identified data is pooled with other customers’ to train models |
| Can we audit the model? | We undergo annual independent audits | The audit clause: often data security only, with models expressly excluded |
| Will you help with our regulators? | Yes | Whether a regulatory-cooperation clause exists, and what it covers |
| How will we hear of changes? | Through our release notes | Whether features may be enabled without notice, and the release that did it |
Who is responsible for a vendor's AI? The insurer, in every text
The insurer is, and the documents say so without exception. The NAIC bulletin brings vendor systems inside the insurer's program and asks for diligence aimed at holding vendor-supported decisions to the same legal standards as the insurer's own (guidelines 1.8 and 4.1). New York's sentence above covers underwriting and pricing. Colorado's regulation says the insurer remains responsible and may let the vendor send documents straight to the Division on its behalf, which moves the paperwork but not the duty.
The NAIC's evaluation tool turns this into exam questions. Its governance checklist asks how the company “Provides standards and guidance for procuring and engaging AI System vendors,” and its narrative option asks the company to discuss “the validation and testing procedures performed on third-party vendor-supplied AI Systems” (NAIC tool, Exhibit B). A vendor's assurance answers neither; the insurer's own file of terms, evidence and tests does.
Inside the AI finder: one amendment, read clause by clause
The AI finder is the agent in AI Governance & Model Inventory that reads vendor contracts, amendments, release notes, invoices above a threshold and vendor questionnaires, and asks of each one whether it adds, changes or reveals an AI system. Its work is easiest to see on one document. Take a sample insurer as it runs through the solution: Calder Ridge Insurance Group, a fictional carrier writing personal auto, homeowners, small commercial and term life in 14 states. At 02:14 on Monday 12 October 2026 a 38-page document arrived from its contract repository: Amendment 4 to the master subscription agreement for its claims platform, from a fictional vendor, Vantor Systems.
The finder wrote a margin note against each clause that matters. Section 7.4 let the vendor enable “Intelligent Features” that use machine learning to generate recommendations without further notice. Section 7.6 let it train those features on the insurer's de-identified claims data, pooled with other licensees'. Section 12.2 gave audit rights over data security and expressly excluded the vendor's models. Section 14.1 did require the vendor to cooperate with the insurer's regulators, and Section 18.3 set renewal for 1 March 2027.
AI Governance & Model Inventory: the AI finder's margin notes on a vendor amendment
A right to switch on AI is not yet an AI system, so the finder looked for the release that used it. It matched Section 7.4 to release notes 24.3, which described an “Auto-Reserve Assist” feature suggesting an initial reserve on new bodily-injury and personal injury protection claims, on by default since 3 September. It checked the register: no system of that kind among 147 rows. It then counted use in the claim system: 2,316 claims with a suggestion, 212 adjusters, 71% of suggestions accepted unchanged. Because reserves set settlement authority and offers on injury claims, the insurer's policy puts the system in the High tier, and the finder proposed row 148 with that tier and the clause quoted under it.
AI Governance & Model Inventory: from a contract clause to a named owner
Agent
AI finderReads Amendment 4 overnight and notes §7.4 (AI may be enabled without notice), §7.6 (pooled training data), §12.2 (no model audit) and §14.1 (regulator cooperation).
Never: contacts the vendor
Agent
AI finderMatches §7.4 to release notes 24.3, finds no matching row among 147, counts 2,316 claims and 71% accepted unchanged, and proposes AIS-148 as High risk.
Never: registers a system
Agent
Model card writerDrafts the card from three documents, every line cited; notes that the vendor supplied no discrimination test.
Never: invents a metric
A person decides
AI governance leadReads the clause and the count, registers AIS-148 and names the claims analytics lead as model owner, who accepts in writing.
A person decides
AI governance lead and third-party riskSend the vendor an evidence request under §14.1, due in ten business days, and flag §12.2 for the March renewal.
A person decides
VP Claims, as business ownerApproves an interim control until validation: adjusters set their own reserve first, and gaps over 25% go to a supervisor.
What the AI finder is never allowed to do
Its limits are what make its output usable. The finder proposes; only the AI governance lead registers or dismisses a system. Every finding cites the clause and page it rests on, so a reviewer, or an examiner, can read the passage itself. And it never contacts a vendor: the evidence request in the sample was sent by people, under the contract's cooperation clause, with a copy to the third-party risk manager. Agents across the solution have read-only access to contracts and model documentation.
The finder also does not judge the vendor's model. It does not decide that a 71% acceptance rate is too high, or that the absence of a discrimination test is disqualifying; it reports both, and the model owner, the validator and compliance decide. Its value is that the question reaches those people the morning the amendment arrives, rather than 39 days after the feature went live, which is how long the sample system ran before anyone knew.
An AI vendor due diligence questionnaire template, answered from the documents
Use the questionnaire to ask, and the documents to verify. The twelve questions below follow the NAIC bulletin's third-party items and New York's vendor paragraphs; beside each is the document that answers it more reliably than the vendor's form.
Twelve vendor AI questions, and the document that answers each
- What AI features does the product contain today?
Current release notes and the order form, not the marketing sheet.
- What AI may it contain tomorrow?
The contract’s rights to add, enable or change features, and whether notice is required.
- Which features are on by default?
Release notes since the last review; confirm in your own system’s configuration and usage logs.
- What data trains the models?
The data-use clause: your data, pooled data, de-identified or not, and for what purposes.
- What does the model take in and put out?
Product documentation and the vendor’s model card, if one exists; inputs and outputs per feature.
- How was it validated, and against what?
Validation reports by line and state, not a summary statistic.
- How is drift monitored?
A monitoring report with dates and thresholds; how often you will receive it.
- Has it been tested for unfair discrimination?
Method, groups, results and dates; if race is estimated, the estimation method.
- How will you notify us of model changes?
A notice clause with a lead time, and a change log for every release since the last review.
- Can we audit the model, not only data security?
The audit clause’s scope; models are often excluded expressly.
- Will you cooperate with our regulators?
A regulatory-cooperation clause covering inquiries and examinations.
- When can these terms change?
The renewal date: the point at which a missing term can be negotiated.
What should a third-party AI risk assessment do when evidence is missing?
Ask for it under the contract, and control the system until it arrives. The NAIC bulletin asks insurers not only to have audit and cooperation terms but to use them (guideline 4.3). NIST's framework asks for contingency processes for failures or incidents in third-party AI systems deemed high risk (GOVERN 6.2) and for third-party risks to be monitored with controls applied and documented (MANAGE 3.1) (NIST AI 100-1).
In the sample, the AI governance lead sent the vendor a request under the cooperation clause the same morning, due in ten business days: the model card, accuracy against incurred losses at 90 days by state, a monthly drift report, adverse impact tests by imputed race and ethnicity, sex and age band on injury claims, and the change log for every release since 24.0. Until the tests arrive and the insurer's own validator signs, an interim control approved by the VP Claims changes how adjusters use the tool: they save their own reserve first, see the suggestion after, and any gap over 25% goes to a supervisor.
An evidence request sent under the contract's cooperation clause
An interim control while the vendor's evidence is outstanding
Third-party AI vendor risk management: before, at renewal, and on every release
A framework for vendor AI has three moments, and most programs only have the first. Before contract: the questionnaire, diligence on data and testing, and the terms the bulletin and New York point to. At renewal: the missing terms, negotiated with the evidence of what the vendor actually shipped. On every release: a read of what changed, and a reopened sign-off when the model changed. NIST's framework asks that pre-trained models used in development be monitored as part of regular maintenance (MANAGE 3.2), which in practice means release by release.
The sample's vendor screen shows what that produces: 31 vendors and 87 AI tools, each with its audit rights over the model, model card, test evidence, regulator-cooperation clause and renewal date. Fifteen tools from five vendors had no audit rights over the model; each renewal date is a dated negotiation. The register, not the vendor file, is where each tool's validation and sign-offs live, and our inventory template lists the vendor fields a row should carry.
Does an insurer need a written vendor AI policy?
In New York, effectively yes: Circular Letter No. 7 says insurers should develop written standards, policies, procedures and protocols for acquiring, using or relying on external data and AI systems from vendors. Colorado requires a documented process for selecting and overseeing vendors as part of its framework. Elsewhere the NAIC bulletin lists vendor standards among the things the program may address.
The policy can be short if it is specific. It should say which documents are read for AI and by whom, what a vendor must supply before a High-risk use, which contract terms are required and which are sought at renewal, and what reopens a sign-off. A policy that says only that vendors will be assessed leaves the examiner's question, how did you know, unanswered. Our guide to the NAIC bulletin sets out the third-party section in full.
Questions people ask
Is there an AI vendor due diligence questionnaire template for insurers?
The twelve questions on this page are a working template, each paired with the document that verifies the answer. They follow the NAIC bulletin's third-party items and New York's ¶35–37; add your own model risk policy's evidence requirements for each tier.
Who is liable when a vendor's AI produces an unfair outcome?
For regulatory purposes, the insurer: the NAIC bulletin holds decisions supported by vendor AI to the insurer's own legal standards, and New York says insurers retain responsibility for vendor tools in underwriting and pricing. Contract terms can allocate costs between the parties, but not the regulatory duty.
How often should vendor AI be reassessed?
At least at renewal and whenever a release changes a model the insurer relies on. New York asks for discrimination testing whenever material updates or changes are made to external data or AI systems, which a vendor's retrain usually is.
Can a vendor send model documentation straight to the regulator?
In Colorado, yes: the amended regulation lets vendors provide requested documents directly to the Division on the insurer's behalf. The insurer still answers for the content, so it should hold its own copy and know what was sent.
What contract terms should insurers seek for vendor AI?
Audit rights over the model or audit reports from qualified auditors, a duty to cooperate with regulatory inquiries, notice before AI features are enabled or models changed, limits on use of the insurer's data for training, and delivery of validation, drift and discrimination test evidence on a schedule.