A bodily injury evaluation tool is software that estimates what an injury claim is worth, and the only test that shows whether one works on a particular insurer's book is a replay on that insurer's own closed files: give the tool each file as it stood at its first valuation, hide the outcome, and see where the amount actually paid falls in the range the tool returns. Demonstrations on a vendor's sample data prove the interface; a replay proves the numbers, the venue logic and the handling of old dollars and changed laws. Botminds' Bodily Injury Claim Valuation, which values an injury file from comparable closed files and published verdicts in every county where the suit could be filed, is built to be judged that way, and the first step with any carrier is to run it on that carrier's own files.
The plan below is a buyer's method, not a vendor's script: it works for any comparable-based tool. Where it shows the solution, the people, carrier and cases are fictional sample data.
A four-week pilot: settings, blind replay, shadow, decision
- Extract closed files, set rulesWeek 1
- Blind replay of ~100 closed filesWeek 2
- Shadow run on open filesWeek 3
- Measures and reviewWeek 4
- Managers set thresholds and authorityWeek 1
- Three adjusters value the same files blindWeek 2
- Desks value as usual; notes comparedWeek 3
- VP decides on the measuresWeek 4
What is a bodily injury evaluation tool, and what are you actually testing?
You are testing three things at once: whether the tool finds the right comparable cases, whether its range is calibrated to what your files actually paid, and whether a manager can check its work in minutes. A tool can pass one and fail the others. A range that is right on average but cites nothing cannot support authority; a well-cited range that runs 20% low on one venue will cost money quietly for a year before anyone notices.
The categories differ in what they need from you. An injury score needs coded inputs and rules. A verdict search needs nothing but access. A comparable-case tool needs your closed injury files, with the injury, treatment, venue, plaintiff firm and amount paid, and that dependency is exactly why a pilot must use your files. A vendor's demonstration on its own data says nothing about how your files, your venues and your desks will behave.
How do you test claim valuation software? Replay closed files blind
Take closed files whose outcome you know, rebuild each as it stood on the date of its first valuation, and run the tool on that version, with the outcome and everything after that date hidden. Then compare where the amount actually paid sat in the tool's range. That is a blind replay, and it is the claims equivalent of testing a model on data it was not trained on.
The trap is leakage: a replay that lets the tool see the future is flattering and useless. Three leaks are common. The file's own closing note must be hidden. The comparable pool must exclude cases that closed after the replay date, since a comparable from next year knows how this year's juries behaved. And the dollar adjustment must restate amounts to the replay date's dollars, or to a common date for every file, not to today's for some and the replay date's for others. Ask any vendor how its replay mode handles each one; an answer of "we use all your data" means the test is not blind.
Choose the files with care too. A good replay set mixes files that settled early, files that settled after suit, and a few that went to verdict, across the venues your desks handle, with at least one injury type where you suspect the desks disagree. Leave out files with coverage disputes or unusual liability, which test something other than valuation.
How many closed files are needed to test a valuation tool?
About a hundred files give a usable answer; twenty give an anecdote. The arithmetic is the margin of error on a share. If a tool's middle half is well calibrated, about half of actual outcomes should fall inside it, and with 100 files the 95% margin on that share is roughly ten points either way. With 40 files it is about fifteen points, with 20 about twenty-two, which is too wide to tell a well-calibrated tool from a poor one.
Regulators work at the same scale. In a 2024 market conduct examination of an auto insurer, Florida's Office of Insurance Regulation drew one of its samples from 14,295 non-PIP claims closed with payment, and its examiners reviewed 109 claims chosen at random. A pilot on about a hundred files is a test of the size an examiner would apply to your claim handling. If your injury book is small in one venue, the tool should say so rather than invent precision. In the sample configuration the solution requires twelve comparable cases per venue and, below that, says so and widens to neighbouring venues.
How do you compare a tool's value to what was paid?
Place each file's actual payment in the tool's range and read the pattern, not the individual misses. One file settling above the range proves nothing. Thirty files settling above it in the same county is a finding about that county. Six measures, read together, separate a tool that is calibrated from one that only looks right.
Six measures that decide a valuation pilot
| Measure | How to compute it | What a good result looks like |
|---|---|---|
| Middle-half coverage | Share of replayed files whose amount paid falls inside the tool’s middle half | Close to half; far above means the range is too wide to guide, far below means it is too narrow or biased |
| Direction of misses | Of files outside the middle half, how many fell above and how many below | Roughly even; a lopsided split is bias |
| Median error | Median of the gaps between the tool’s median and the amount paid, as a percentage | Small and similar across venues; one venue far worse points to a venue problem |
| Venue split | All of the above, county by county | No county where the tool is consistently low; that is where suits will go |
| Desk spread | Ratio of highest to lowest first authority on matched files, before and with the tool | Narrower with the tool, measured on your own desks |
| Checkability | Minutes a manager needs to verify a range from its cited cases | Minutes, with every amount opening to its file or report |
The first measure is the one most pilots skip and the most informative. A tool can have a small average error and still be useless if its middle half captures 90% of outcomes, because a range that wide cannot tell an adjuster whether $95,000 or $165,000 is the better offer. Conversely, a middle half that captures 20% of outcomes is confidently wrong. Calibration, not precision, is what makes a range safe to put in front of the person who signs authority.
Dollars from different years: test the adjustment
A comparable from 2021 is not comparable until it is restated in today's dollars, and the pilot should check that the tool does it with a named index and shows the factor on each case. Bodily Injury Claim Valuation uses the Consumer Price Index for medical care and keeps the amount actually paid beside the restated one. On one sample comparable, a Cook County file paid $311,100 in 2025 appears at $322,000, a factor of 1.035.
How much a closed file's amount moves when restated to September 2026 dollars
Three checks are worth making in the pilot. First, which month the tool restates to: it cannot be a month whose index has not been published yet, and a tool that claims otherwise is guessing. Second, how it handles a missing month such as October 2025. Third, whether medical prices are the right index at all for a settlement that is mostly general damages. The medical care index rose about 13% from January 2017 to January 2022, while the Insurance Research Council, an insurer-supported non-profit, found average bodily injury payments rising 7.8% a year over almost the same period. A medical-price factor restates an old case conservatively; it does not capture the severity trend, which is a reason to prefer recent comparables over heavily restated old ones.
Tort reform cutovers: the date-of-loss test
A pilot should include closed files on both sides of every recent change in the law of damages in its states, and check that the tool does not mix them. Florida's 2023 reform, HB 837, took effect on 24 March 2023 and, except where it says otherwise, applies to causes of action filed after that date: among other changes it limits evidence of past medical bills that have been paid to the amount actually paid and bars recovery by a claimant more than 50% at fault. Georgia's 2025 act, SB 68, changed how medical expenses are proved for causes of action arising on or after its effective date in April 2025, according to the Georgia Senate Research Office.
The test is simple to run: replay a Florida file filed in 2024 and look at its comparables. If most of them were filed before March 2023, when billed amounts and a different fault rule applied, the tool is valuing the file under the wrong law. The details of the Georgia cutover, and why it splits a set of closed files by date of loss rather than date of filing, are worked through in Georgia tort reform 2025 and medical bills.
The unit's own rules: what counts as comparable, how dollars move, who grants
Week by week: what each stage of the pilot proves
Week one: rules before results. Agree the match threshold, the window of closed years, the minimum cases per venue, the dollar index and the authority levels before seeing any output, so the pilot cannot be tuned to flatter itself. Extract the closed files with the fields the tool needs, and list the cutover dates in your states.
Week two: the blind replay. Run about a hundred closed files as they stood at first valuation. In parallel, give twenty of them to three adjusters from different desks, without the outcome, so the tool's spread and calibration can be set beside your desks' own.
Week three: the shadow. On open files, desks value as they always do; the tool runs alongside, and the notes are compared. This is where the checkability measure is earned: can a claim manager read the tool's range and its cited cases and decide faster than from the desk's own note?
Week four: the decision. Read the six measures by venue and by desk. Examine, as an examiner would, a random handful of the tool's notes for whether every amount opens to its source. Decide on the measures you agreed in week one, not on how the demonstration felt.
What results show a valuation tool works?
A tool works when its ranges are calibrated on your files, its misses are not concentrated in one venue, your desks agree more with it than without it, and a manager can verify any range from its sources in minutes. The design target for the solution, six minutes from open file to cited range against one to two days by hand, is a target to verify in the shadow week, not a promise to take on trust.
Pass marks to agree before the pilot starts
- Middle-half coverage near one half
On about a hundred replayed files, with the 95% margin in mind (roughly ten points).
- No venue consistently low
Every county your desks handle checked separately; one low county is where plaintiffs will file.
- Cutovers respected
No Florida file filed after March 2023 valued mainly from earlier filings; the same for Georgia’s 2025 change by date of loss.
- Every amount opens to its source
Closed file number or verdict report with court and case number, and the original amount beside the restated one.
- Desk spread narrower
First authority on matched files, measured on your own desks before and with the tool.
- People still decide
No agent can set authority, contact the claimant or save to the claim; the NAIC’s AI bulletin expects insurers to know how far humans are involved in the final decision.
The last item is not a technicality. The NAIC model bulletin on insurers' use of AI asks insurers to scale their controls partly by how far people are involved in the final decision, and a pilot is the moment to document that the tool computes and people decide. What the agents may and may not do is set out in personal injury software for case evaluation.
Where Bodily Injury Claim Valuation fits in a pilot, and where it does not
The solution fits a pilot on contested bodily injury files with a history of closed files behind them, in states whose venue statute is set in its rules. It reads the open file, searches closed files and published verdicts, builds the range in every county where suit could be filed and drafts the cited note, so the replay and shadow weeks test its whole chain, not a single score. Each comparable can be opened against the file it came from and excluded with a reason.
What a reviewer checks in a replay: one comparable against the file, point by point
It does not fit everywhere. A carrier with few closed injury files in a state will see the minimum-case warning more often than a range. It is not a medical bill review tool and not a jury forecaster: the venue analyst describes each county and never predicts a jury or a judge. Botminds builds the solution from a carrier's authority levels, note standards and a sample of its closed and open injury files, and the product's own answer to how a buyer starts is the right one for any vendor: show it on your own files first.
Questions people ask
What is a bodily injury evaluation tool?
Software that estimates what an injury claim is worth, from coded inputs (an injury score), from published outcomes (verdict research), or from comparable closed files and verdicts in each venue (comparable-case valuation). Bodily Injury Claim Valuation is the third kind.
How long should a valuation pilot run?
Four weeks is enough for one line and a few venues: a week to agree rules and extract files, a week of blind replay on about a hundred closed files, a week shadowing open files, and a week to read the measures and decide.
How many closed files do you need to test a valuation tool?
About a hundred. At that size the 95% margin of error on the share of outcomes inside the tool's middle half is roughly ten points; with twenty files it is about twenty-two, too wide to judge calibration. Florida's examiners reviewed a random sample of 109 claims in a 2024 auto market conduct examination.
How do you adjust old settlements for inflation?
With a named price index and a factor shown on each case. Restated with the BLS medical care index, a payment from January 2021 is multiplied by about 1.137 to reach September 2026 dollars; the solution keeps the original amount beside the restated one.
What is the biggest mistake in a valuation pilot?
Letting the tool see the future: running replayed files against comparables that closed after the replay date, or against the file's own closing note. A test that is not blind will look accurate and prove nothing.
Should a pilot include files that went to suit?
Yes. Files that settled after suit, or went to verdict, are where valuation errors cost most, and they show whether the tool's range in the county of suit matches what those files actually paid.