Disparate impact analysis tool: agents run the tests, people decide

A disparate impact analysis tool for an insurance model takes a scored portfolio, estimates protected-class membership, measures how outcomes differ by group, finds which inputs drive any difference and tests fairer versions of the model, then writes up what it found. That is the mechanical part of the three-step assessment New York's Circular Letter No. 7 describes, and agents can do all of it on every model version. What a tool should not do is decide whether a difference has "a legitimate, lawful, and fair explanation," or whether a fairer model meets the insurer's business needs: those are judgments the rules leave with the insurer. Botminds' AI Bias & Fairness Testing is built on that line. Six agents run the analysis; a model owner decides what changes and a model risk validator signs.

The line matters to a buyer because it decides what the tool's output is worth. A tool that only produces numbers leaves weeks of work undone. A tool that produces conclusions produces something an examiner will not accept from a vendor. This guide sets out the steps, the three decisions that stay human, and what to ask before buying.

"It is up to insurers to determine the appropriate sufficiency thresholds and standards of proof based on the product and the particular use of ECDIS or AIS."

New York DFS, Insurance Circular Letter No. 7 (2024)

What is a disparate impact analysis for an insurance model?

It is a test of whether a model that never sees race or another protected trait still produces worse outcomes for a protected group, and if so, whether that is justified. The American Academy of Actuaries describes the concern as a protected group "disproportionately impacted by policies or practices that on the face appear to be neutral" and observes that "Insurance companies have not historically attempted to measure disparate impact" (Academy, 2024). The rules that now ask for it were written for algorithms and external data.

New York's version is the most explicit. A "comprehensive assessment" of an underwriting or pricing guideline built on external data or AI should at a minimum go through three steps (NY DFS, 2024):

Step 1 asks "whether the use of ECDIS or AIS produces disproportionate adverse effects" for similarly situated insureds or a protected class. With no prima facie showing, the insurer may stop.Step 2 asks "whether there is a legitimate, lawful, and fair explanation or rationale for the differential effect." Without one, the insurer modifies the model and starts again. Step 3 is a documented search for a less discriminatory alternative; if one exists, the insurer adopts it and tests again, and if none does, it repeats the search every year.

Colorado's statute defines the same idea in one sentence: external data or a model correlated with a protected class, whose use leaves that class with a worse outcome than its link to the insurance practice can justify (C.R.S. § 10-3-1104.9(8)(e)).

How to conduct a disparate impact analysis, step by step

Conduct it in the order the evidence builds: know the model, estimate the groups, measure, find the driver, try the alternatives, write it down, then decide. The flow below is the order AI Bias & Fairness Testing follows. Each agent's "never" line is a limit the solution enforces, not a promise in a brochure.

The middle of the chain is where analyses usually stall by hand. A failing ratio says nothing about why. Finding the input means refitting or re-scoring the model once per input, and each alternative means another refit and another full set of tests. New York even names techniques for the driver step, from sensitivity analysis to Shapley values and regression coefficients (NY DFS, 2024). That is computation, and computation is what agents should take.

Three decisions a disparate impact analysis tool should never make

Three judgments stay with people: what counts as a failure, whether a gap is justified, and whether to change the model. Each is written into the rules as the insurer's own call, and an insurer cannot hand it to a supplier. New York is direct about that: "An insurer may not rely solely on a third-party's claim of non-discrimination or a proprietary third-party process to determine compliance with anti-discrimination laws" (NY DFS, 2024).

The threshold. No adopted insurance rule sets a pass mark for these metrics (Colorado's draft life rule would); the 0.80 adverse impact ratio analysts quote comes from federal employment guidelines. New York leaves "sufficiency thresholds and standards of proof" to insurers. In the solution the thresholds are a versioned standard approved by the chief actuary; an agent applies them and cannot change them. The justification. Whether an input's effect is "legitimate, lawful, and fair" turns on loss evidence, actuarial standards and the line of business. An agent can assemble the evidence; an actuary decides whether it holds. The change. Step 3 asks whether an alternative would still meet the insurer's legitimate business needs, and adopting one changes premiums for real customers. That is the model owner's decision, made with the filing in view.

Independence is the reason for the second column, not a formality. NIST's AI Risk Management Framework expects people who did not develop a system, or outside assessors, to take part in its regular assessments (NIST AI RMF, MEASURE 1.3), and New York wants testing and validation open to independent review and challenge. A validator who signs a test she did not build is that challenge.

Disparate impact analysis calculator, template or tool: what each can carry

The three answer different questions. A calculator turns two outcome rates into a ratio; a template lays out what the write-up should contain; a tool runs the whole analysis on the model itself. Which one is enough depends on whether group membership is known, how many inputs the model has and how often it changes.

A template earns its place too: the headings New York's documentation expectation implies (method, data, results, explanation, alternatives, decision) are worth fixing before any test runs. How to calculate an adverse impact ratio with probability weights shows why a two-number calculator gives the wrong answer once race is estimated rather than reported.

One run, on a fictional rating model

A fictional sample run shows the division of labour on a real-sized problem. A fictional mutual insurer's personal auto rating model, version 7.2, added a telematics driving score and rebased its credit-based insurance score tiers. It is tested on 184,260 New York quotes from April to June 2026. Black-probable drivers reach the preferred tier at 30.1% against 41.8% for White drivers, an adverse impact ratio of 0.72, and four of the six tests in the insurer's standard fail.

Six agents, 38 minutes, and the validator last

In the fictional run, the proxy detector traced 6.0 of the 11.7-point gap to five-digit ZIP code territory. Removing territory outright closed half the gap but dropped calibration for Black-probable drivers to 0.93 and kept only 96.4% of the model's lift, under the 98% floor in the insurer's standard. The alternative searcher's density-banded territory, county by five population-density bands, passed all six tests and kept 99.3%. The agents stopped there. The pricing actuary who owns the model adopted it for the filing; the validator then signed.

Adoption is a person's act, with the consequences on screen

Disparate impact testing for claims and underwriting models, not only pricing

A tool scoped to rating plans misses part of the exposure. New York's circular letter covers underwriting and pricing only, and says so. But Colorado's statute defines insurance practice as "marketing, underwriting, pricing, utilization management, reimbursement methodologies, and claims management" (C.R.S. § 10-3-1104.9(8)(c)), and the NAIC bulletin reaches AI systems that support any regulated insurance practice, claims settlement included (NAIC, 2023). California's claims regulation already bars settlement practices based on a claimant's race, gender or national origin, or on the territory insured (10 CCR 2695.7(a)).

The tests change shape for claims. A fraud referral model's adverse outcome is a referral, so the natural metric is a referral rate or odds ratio by group; in the fictional insurer's inventory, its auto claims fraud referral model fails one test on an odds ratio of 1.31 for Hispanic claimants and has a remediation date. Validating a fraud risk scoring model covers the other questions such a model raises, from biased labels to how often to retrain.

Disparate impact analysis after the 2025 federal changes: is it still needed?

For insurers, yes: the obligations in this guide come from state insurance law, and the federal changes did not repeal them. In April 2025 an executive order set a federal policy against disparate-impact liability and told the Attorney General to "determine whether any Federal authorities preempt State laws, regulations, policies, or practices that impose disparate-impact liability" (Executive Order 14281, § 7(a)). That review is a federal question about the future. The state texts are still in force: Colorado amended its governance regulation in August 2025, after the order, effective 15 October 2025 (3 CCR 702-10), and New York's letter was still posted as current guidance in October 2026.

Lenders face a different picture, because the federal banking agencies themselves stepped back. Disparate treatment versus disparate impact in lending sets out what each agency changed and what it kept.

What to ask before you buy a disparate impact analysis tool

Ask to see each answer on one of your own models, not the vendor's sample. A tool that cannot show the inputs behind a failing number, or that cannot show who decided what, will not survive the first examiner's question.

Where a tool fits, and where it does not

A disparate impact analysis tool makes testing cheap enough to run on every change. It does not make the hard questions easier. It needs a scored test portfolio and the model's documentation; without them there is nothing to test. Its group estimates are only as good as the names and addresses on file, and some groups cannot be estimated well at all. Its decomposition shows association, not cause, which is why the explanation of risk stays with an actuary. And small books give small samples: a gap in a few hundred policies may be real and still not significant.

The economics of running it on every change are in cost per model-year, and the way to trust a new tool is to pilot it on a model you have already tested. The six agents, their limits and the signed report are on the AI Bias & Fairness Testing page.

Questions people ask

Is there a disparate impact analysis calculator for insurance?

A simple calculator divides one group's favourable-outcome rate by another's. It works where group membership is reported, as in employment. For insurance models, where race is estimated, it needs probability-weighted counts, and it still will not show which input drives the gap or whether a fairer model exists.

What goes in a disparate impact analysis template?

At least the method and data, including how groups were estimated; results for every group and metric; the drivers of any gap; the explanation of each driver's relation to risk; the alternatives tested; and the decision and who made it. That mirrors New York's three steps and its expectation that the reasoning behind the testing is documented.

How long does a disparate impact analysis take?

By notebook and spreadsheet, Botminds estimates eight to twelve weeks of actuarial and data-science time per model, mostly spent finding drivers and refitting alternatives. The solution's design target is one day for a complete test with its written explanation; the fictional sample run above took 38 minutes before the people's part began.

Does the analysis change our filed model?

It should not, and in AI Bias & Fairness Testing it cannot. The agents never change a filed model or move a threshold; the alternative searcher proposes fairer refits, the model owner decides whether to adopt one, and the validator signs the test of whatever version is filed.

Can we rely on a vendor's disparate impact analysis?

Not on its conclusion alone. New York holds the insurer responsible for third-party tools whatever the vendor claims, and expects contracts with vendors to give audit rights "where appropriate and available." Use a vendor's tool to run the work; keep the judgment and the signature in-house.

What is an adverse impact analysis tool?

The same thing under its employment name: software that compares selection rates by group, usually against the four-fifths rule in the federal Uniform Guidelines. Insurance testing borrows the ratio but adds what employment tools rarely need: estimated group membership, several metrics, and a search for a less discriminatory model.