NAIC AI Systems Evaluation Tool: the four exhibits, before the vote

The NAIC AI Systems Evaluation Tool is a draft questionnaire, in four exhibits, that state insurance regulators can use in market conduct exams, financial analysis and financial exams to learn how an insurer uses AI and whether its governance works. Exhibit A counts the insurer's AI systems by business area; Exhibit B asks for its governance framework, as a narrative or a checklist; Exhibit C asks for details of each system the insurer itself rates high risk; Exhibit D asks which data elements its AI uses and where they come from (NAIC tool, draft 4.0). Twelve states piloted it from March to September 2026, and the NAIC's plan is to consider an updated version for adoption at its Fall National Meeting in November. Until a vote, it is a draft, but it is the clearest preview of the questions insurers will be asked, and the answers come from the kind of register Botminds' AI Governance & Model Inventory keeps.

What is the NAIC AI Systems Evaluation Tool?

It is a set of “Optional Supplemental Exhibits for State Regulators,” written by the NAIC's Big Data and Artificial Intelligence (H) Working Group on a charge to help regulators identify and assess AI risks on an ongoing basis. The tool says it “is designed to supplement existing market conduct, financial analysis, and financial examination review procedures,” and that regulators should keep treating existing NAIC handbooks as authoritative. It is a questionnaire for regulators to send, not a rule for insurers to follow.

Its premise is stated in its background: insurers “must demonstrate to regulators that appropriate risk-based oversight mechanisms are in place and are functioning effectively.” That is a shift in burden. Under the NAIC's model bulletin, adopted in 29 jurisdictions as of 8 October 2026 (NAIC map), the insurer is expected to have a written AI program. The tool asks it to prove the program works, system by system, with counts, dates and names.

Who piloted the tool, and what happens in November?

Twelve states: California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia and Wisconsin (NAIC pilot summary). The summary says they would use the tool from March to September 2026 “in different types of work, including market conduct exams and reviews, financial analysis, and financial exams,” focusing on domestic insurers across property and casualty, life and health, and spending more time on high-risk systems than on low-risk back-office ones. Each state could adapt the questions.

The plan then runs to a vote. The working timeline has the tool updated on pilot feedback in September and October, issued again for review, and in November: “Consider the updated Tool for adoption at the Fall National Meeting.” The working group's page lists a Fall National Meeting session on 16 November 2026, and was separately exposing an AI Risk Evaluation Supplement, version 6.0, for comments until 30 October 2026 (NAIC Big Data and AI (H) Working Group). Until the NAIC adopts it, the tool is a draft; even then it remains optional for each state.

The four exhibits, and the evidence each asks for

The exhibits are built to be used in order. The tool suggests regulators “may wish to first use Exhibit A” and decide from it whether to go further; a company whose AI use is “so limited or low in inherent risk” may not need the later exhibits. Each exhibit asks for an “as of” date and covers all companies and lines of business, with separate submissions where governance differs by entity, line or state. The table sets out what each asks and the evidence an insurer needs to answer it.

Exhibit A: how many AI systems, in which business areas?

Exhibit A is a grid. Rows are operational areas, from marketing, premium quotes and underwriting through ratemaking, claims and adjudication (salvage and subrogation included), customer service, utilization management, fraud, investment and capital management, legal and compliance, producer services, reserves and valuations, catastrophe triage and reinsurance. Columns are four counts: AI system models currently in use; those with direct consumer impact; those with material financial impact; and those implemented in the past 12 months, plus a description of each use case.

The definitions decide the counts. The tool treats models that “augment or automate decision making related to consumers” as having direct consumer impact, and defines augmentation as a system that suggests an answer or advises a human, automation as one with no human intervention and support as one that informs without suggesting a decision. A vendor's reserve suggestion shown to an adjuster is augmentation with direct consumer impact. The hardest column is the last: a system a vendor switched on in a release this year counts as implemented in the past 12 months, whether or not the insurer noticed.

Exhibit A from a register: AI systems counted by business area in AI Governance & Model Inventory

Exhibit B: governance, as a narrative or a checklist

Exhibit B asks for the governance framework in one of two forms. The narrative option asks five sets of questions: the framework itself and the role that maintains it, the governance structure and board reporting, how the framework is integrated, assessed and improved, how responsibility is assigned; the uses of AI that generate material financial transactions, consumer impact or financial-statement information; the policy for material vendors with their validation and testing; the oversight of AI used by professional service providers such as actuarial, claims and managing general agents; and any other aspects of the design. It also suggests asking how the company assesses “autonomy, reversibility, and reporting impact risk.”

The checklist option asks whether the company has adopted a written AI program, when and how often it is reviewed, the board's or management's role, and then fourteen items, each answered with a page reference to the governance documents: from residual risk of unfair trade practices, legal compliance and adverse consumer outcomes to data privacy, suitability for intended use, ERM and ORSA, the software development lifecycle, financial reporting, training, risk quantification, vendor standards, consumer complaints about AI and consumer disclosures. The tool adds that its questions are “not intended to be interpreted as creating new requirements.” Each item still needs a page number to point to. Our committee charter guide covers the governance documents that answer most of them.

Exhibit C: the systems you call high risk, in 13 questions

Exhibit C turns on the insurer's own classification. The tool says the exhibit “relies on company assessments of the risks and materiality of its AI System(s),” including which systems the company itself calls high risk, and that regulators may ask how the company assesses risk and for its model inventory. A tier rule written in one sentence, and applied consistently, is therefore part of the answer.

The thirteen questions run: name and version; model type; implementation date; internal or third-party development with the vendor's name; risk classification; risks and limitations; AI type (automate, augment, support); how outputs were tested for drift, accuracy, unfair trade practices, unfair discrimination and degradation, how the model was validated before deployment and how it is monitored; last date of testing; use cases and purpose; effect on financial statements, risk assessment or controls; review for compliance with unfair trade practices and claims settlement laws; and any regulatory actions taken against the company for its use of the model. That is a model card, a test file and a compliance note per system.

Exhibit C from a model card: one High-risk vendor system in AI Governance & Model Inventory

Exhibit D: which data, from whom

Exhibit D lists data elements and asks, for each one used in training or test data, the type of AI system (machine learning or generative), how the company uses it across its operations by line, and whether it is sourced internally or from a third party, with the vendor's name. The list is pointed: aerial imagery; age, gender, ethnicity or race; consumer or insurance risk scores; crime statistics; criminal convictions; driving behaviour; education level; facial or body detection; geocoding and geo-demographics; household composition; image and video analysis; income; job history; loss experience; medical and genetic information; natural catastrophe hazard; online social media; personal financial information; reasonable accommodations; telematics; vehicle data; voice analysis; weather; and other non-traditional elements.

Answering it needs a map from each system to its inputs and their sources, which most inventories do not keep. It also needs care with vendor models trained on pooled data: in the sample, the reserve recommender was trained on de-identified claims pooled from the vendor's insurer clients, so its data row names the vendor, and the insurer's own claims, as sources.

How should an insurer prepare before the vote?

Prepare as if Exhibit A and Exhibit C arrive in the next exam, because in pilot states they may already have. The tool lets a company say it has already answered another state's request and lets regulators accept a prior submission “if the prior response is still current and applicable,” which rewards insurers who keep one current answer rather than writing a new one each time.

This is where the register earns its keep. In the sample, a fictional market conduct exam's AI questionnaire of ten questions arrived with a 14 November deadline; on 12 October every answer had been drafted from the register, six were approved, and the answers the reserve recommender touched were marked updated when it was added. The product's design target is one day to answer such a questionnaire from the register. Our software guide shows how to measure that from the first quarter.

High-risk vendor evidence for Exhibits C and D in AI Governance & Model Inventory

Is the draft tool binding, and where are the NAIC's AI principles?

It is not binding. It is a draft, its exhibits are optional for states, pilot states could adapt the questions, and its governance exhibit says it creates no new requirements. What gives it weight is what stands behind it: the regulator's existing examination authority, the unfair trade practices and claims settlement laws it cites, and the model bulletin's own list of documents a department may request (NAIC Model Bulletin, Section 4). The NAIC's working group page lists the document as “AI System Evaluation Tool 4.0”; this guide read the clean PDF version.

The NAIC's Principles on Artificial Intelligence, adopted in 2020, sit behind both texts. The bulletin calls them “an appropriate source of guidance” and summarises them as fairness and ethical use, accountability, compliance with state laws and regulations, transparency, and a safe, secure, fair and robust system. The tool's Exhibit B checklist is, in effect, a test of whether each principle has a page in the insurer's governance documents.

Questions people ask

Which states piloted the NAIC AI Systems Evaluation Tool?

California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia and Wisconsin, from March to September 2026, according to the NAIC's pilot summary. The pilot focused on domestic insurers in property and casualty, life and health.

Is Florida part of the NAIC AI evaluation tool pilot?

Florida is one of the twelve pilot states the NAIC lists. The pilot summary says states would use the tool in market conduct exams and reviews, financial analysis and financial exams, and that each could adapt questions to its needs.

Has the NAIC adopted the AI Systems Evaluation Tool?

Not as of late October 2026. The NAIC's plan was to update the tool after the pilot and consider it for adoption at the Fall National Meeting in November; the working group listed a session on 16 November 2026.

Does the tool apply to vendor AI?

Yes. Exhibit C asks whether each high-risk system was developed internally or by a third party and the vendor's name, Exhibit D asks for third-party data sources by vendor, and the governance exhibit asks for the policy on material AI vendors and their validation.

What is the difference between the NAIC AI bulletin and the evaluation tool?

The bulletin, adopted in December 2023, sets out what regulators expect of an insurer's AI program and what they may request. The tool is a draft questionnaire regulators can send to collect that information in a standard form. Our bulletin guide covers the expectations themselves.