FNOL software takes a claim's first notice of loss, from a phone call, web form, app, email or agent, and turns it into a structured claim, ideally with the policy matched, the severity sized and the claim routed. Demonstrations look alike because the happy path is easy: a clear caller, a simple collision, a form that fills itself. What separates products is what happens off that path, when the injury is mentioned late, the value was never said, the caller speaks Spanish, or the model is updated. The fifteen questions below are grouped by the risk each one tests. Botminds' FNOL & Claims Triage Automation is one of the products a buyer might put them to, and its answers are given where they help; the questions work for any vendor, including us.
Examples come from the solution on a fictional carrier, Tallowmere Mutual, with sample data. Rules and research are linked where they appear.
Why the questions are about governance, not features
29
jurisdictions listed as having adopted the NAIC model bulletin on insurers’ use of AI, by October 2026
17–33%
how often commercial legal research tools hallucinated in a study by Stanford and Yale researchers, despite citing sources
15
questions below, in five groups: evidence, injury, limits, reach and control
What FNOL software does, and what the questions test
FNOL software, at its best, does the clerical half of a first-notice job: transcribing, filling the loss notice, finding the policy, scoring severity, picking a desk and an adjuster, and drafting the acknowledgement. The words buyers search for (FNOL AI, an FNOL agent, an FNOL portal, FNOL services) describe parts of that: an AI that listens or reads, an agent that does one task, a portal where customers report, or a service that takes the calls for you. The questions apply to all of them.
Regulators have made the governance question the buyer's. The NAIC's model bulletin expects insurers' decisions made with AI to meet the same legal standards as any other, and its expectations extend to systems bought from vendors as well as those built in-house. Washington's amended claims rule says "Insurers are responsible for the accuracy of their evaluations including, but not limited to, evaluations made on their behalf". Whatever FNOL software a carrier buys, its routing and its messages are the carrier's acts.
Evidence: does every field cite its source?
The first group asks whether a person can check what the software wrote. A loss notice is the basis of every later decision, so each value needs a source a reviewer can open: the sentence and its call time, the form field, the page or the photo. Citations are necessary, not sufficient. A study of commercial legal research tools by Stanford and Yale researchers found they "each hallucinate between 17% and 33% of the time" despite being built to cite sources. That was legal research, not claims, but the lesson carries: a citation lets a person check; it does not mean the check was done.
Questions 1–3: evidence
| Question | A good answer | A red flag |
|---|---|---|
| 1. Does every field on the loss notice show where it came from? | Yes: the sentence and call time, the form field, the page or the photo, one click away. | A confidence score instead of a source; "the model extracted it". |
| 2. Can we replay the call at the moment a value was said? | Yes, from the field itself, and the transcript stays with the claim. | Audio kept by the vendor only, or deleted after transcription. |
| 3. What does the claim’s log record, and can an examiner read it? | Each step with its time and source, ending with the named person who confirmed. | A log of system events with no link to the decision it made. |
The solution's answer: each value on the notice carries the call time of its sentence, or the form field, page or photo, and one click plays the transcript at that point. The claim's run log records each agent step with its call time and ends with the specialist's confirmation. Regulators expect a claim file from which the insurer's handling of the claim can be rebuilt step by step (NAIC Model 902, section 4); a log that does not link to the decision does not allow that.
Evidence on the claim: every point, every route, every step
Injury: does an injury always win the route?
The second group tests the error that costs most at first notice: an injury that does not reach the injury desk. Ask for a rule, not a weight. A score that rises when an injury is mentioned can still fall short of a threshold; a rule that sends any injury, to anyone, to the injury unit cannot. Then ask for the evidence on your own data, because no vendor's accuracy on someone else's calls tells you about your callers.
Questions 4–7: injury and routing
| Question | A good answer | A red flag |
|---|---|---|
| 4. If an injury is mentioned, does the claim always go to the injury desk? | Yes, by a rule that overrides the score, which no setting can switch off. | Injury raises the score; the threshold decides. |
| 5. What is the injury recall on our own past calls and notices? | A seeded test result: injuries planted late, by passengers, in Spanish and in writing, with every miss explained. | An overall accuracy figure from another carrier’s data. |
| 6. What happens when the injury is said after the claim was routed? | The route changes the moment it is said, and the contact clock starts then. | Routing happens once, at the end of the call or the next morning. |
| 7. How does it choose the adjuster, not just the desk? | Open files against a cap and a licence in the loss state; over-cap needs a supervisor. | Round-robin, or "the desk assigns it". |
The solution's answer: rule R-07, any injury to anyone, always wins and is locked in settings. On the sample call, the route changed at 01:01, the second the caller said the other driver was taken by ambulance, and a 24-hour contact clock started. The router chose an adjuster with room under the cap and a licence in Ohio. The test design behind question 5 is set out in claims triage solutions: plant 200 injuries, count what comes back.
Question 6 answered on screen: the route changes mid-call
What should FNOL software never do?
FNOL software should never decide coverage, fault, fraud or payment, never create a claim or send a message without a person, and never guess a value it did not hear. Those limits are what make the rest safe to automate. A vendor that offers a coverage decision at first notice has moved into adjusting; one that offers a fraud score that changes the route has moved into accusation; one that fills blanks with typical values has made the notice unreliable in a way nobody can see.
Questions 8–10: limits
| Question | A good answer | A red flag |
|---|---|---|
| 8. Does it confirm or deny coverage, decide fault or call a claim fraud? | No. It lists coverages that may apply; review signals go quietly to special investigations. | A "coverage decision" or a "fraud score" that changes the route. |
| 9. Can it create a claim or send a message without a person? | No. The specialist confirms the claim and every message. | Auto-send is on by default and can be turned off. |
| 10. What does it do with a value nobody said? | Leaves it empty, lists it as missing and asks for it. | Fills it from a typical value or the last claim on the policy. |
The solution's answer: the policy matcher lists coverages but never confirms or denies them; the severity scorer never decides fault and never calls a claim fraudulent, and review signals send a quiet copy to special investigations without changing the route or the customer's experience; the notice reader leaves a field empty and asks for it. The wider debate about AI and claim denials is covered on a sibling solution's page, insurance companies using AI to deny claims.
Fairness and language: whose calls was it tested on?
The fourth group asks who the software works for. Speech recognition is not equally accurate for every speaker: a 2020 study in PNAS of five commercial systems found error rates almost twice as high for Black speakers as for white speakers (Koenecke et al.). Systems have improved since, which is why the question is about testing, not about a vendor's general claims. Ask for results by language and caller group, on your own recordings.
Questions 11–13: reach
| Question | A good answer | A red flag |
|---|---|---|
| 11. Was it tested in our languages and on callers like ours? | Results by language and by caller group, on our own audio. | One accuracy number, English only. |
| 12. Can a customer report without the app or portal? | Phone, email, web form, agent, post: all read the same way. | Photos and documents only through the portal. |
| 13. Does it work with our recording announcement in every state we write? | Live transcription starts only on a recorded call with the announcement played. | "Consent is the carrier’s problem." |
Channels belong in this group because a portal can exclude as surely as a transcript can. For auto claims, Washington's amended rule forbids requiring a claimant or repair shop to use only an internet-based application to send claim information, and requires the insurer to accept email, post, fax and delivery in person. Ask how the software reads what arrives on each. On recording, federal law allows one-party consent, while some states require every party's; what a carrier brings on day one sets out the announcement Washington's statute accepts. The solution transcribes in English and Spanish; how a Spanish first notice should be handled is covered in file a claim in Spanish.
Control: who approves a rule change?
The last group tests who is in charge after go-live. Routing rules, severity signals, bands and clocks will change; the questions are who may change them, how a change is tested before it applies, and whether a past claim can be explained under the version that routed it. A vendor that tunes a model between releases is making the carrier's routing decisions for it.
Questions 14–15: control
| Question | A good answer | A red flag |
|---|---|---|
| 14. Who approves a change to a signal, a rule, a band or a clock? | A named role at the carrier, after a replay on last week’s notices. | The vendor tunes the model between releases. |
| 15. Can we see the version of the rules that routed any past claim? | Yes: every claim names the signal and rule versions it was scored under. | Only the current version exists. |
The solution's answer: signals are published and versioned (v3.2 on the sample carrier); any change can be tried on a single sentence, replayed on last week's notices and applies only when the director of claims operations approves it. Route overrides and assignments over an adjuster's cap need the intake supervisor. Every claim shows the signal version it was scored under.
Change control in one screen: try, replay, send for approval
How to evaluate FNOL software in a pilot
Evaluate FNOL software on your own first notices, with the vendor's settings frozen, against measures you choose before the pilot starts. A demo on the vendor's sample shows the happy path; a pilot on your calls shows your failure modes. Run written notices first if recordings are slow to arrange, then live calls with every claim confirmed by a specialist.
A pilot scorecard for FNOL software
- Injury recall on a seeded set
Every planted injury on the injury desk; every miss read and explained.
- First desk against final desk
Weekly, with each move classified: new information, or a first-notice error.
- Fields filled with a source, and fields left empty
An empty field with a request is a pass; a filled field without a source is a fail.
- Specialist overrides
How often the person changed the route or the message before confirming, and why.
- Results by channel and language
Phone, web, app, email and agent; English and every other language you serve.
- Time from notice to the right desk
Measured from the first notice, not from when someone opened it.
Set the pass marks before the run, and keep the run's settings with its results. The sample carrier's insights screen shows the measure to keep after go-live: the desk each claim was first routed to against the desk it ended on, 4,833 of 4,912 notices staying put over 30 days in the sample.
Red flags in an FNOL software demo
Some red flags show before any pilot. A demo that only uses a calm caller with a two-car collision and no injury has avoided the hard cases. A screen that shows a score without its reasons cannot be governed. A product that sends the acknowledgement automatically "to save time" has removed the one check that protects the customer from a wrong message. And a price that depends on "automation rate" rewards the vendor for removing people from the steps where they matter.
The useful counter is to bring your own hard cases to the demo: a recording where the injury comes out at minute two, a Spanish-speaking caller, an email with the injury only in an attachment, a sentence like "nobody was hurt" that a keyword match would misread. Ask the vendor to run them live. On the solution, the rules screen's sentence box exists for exactly this; a buyer can type what a caller said and see the score, the signals and the rule.
Questions people ask
What is FNOL software?
Software that turns a first notice of loss into a structured claim and, in fuller products, matches the policy, sizes the claim and routes it. It works across phone, web, app, email and agent channels. The best test of it is not a demo but its injury recall and evidence on your own past notices.
What is FNOL AI?
AI used at first notice: speech recognition and language models that transcribe calls, read written notices and fill the loss notice. In the NAIC's survey of large auto insurers, write-in uses of AI in claims already included speech analysis and listening to voice calls. The governance expectations of the NAIC's model bulletin apply whether the AI is built in-house or bought.
What is an FNOL agent?
Either a person who takes first notices, or a piece of software that does one first-notice task, such as transcribing, matching the policy or routing. In the solution described here, six software agents each do one task and a first-notice specialist confirms the result. Ask any vendor what each of its agents may never do.
Should FNOL software use a customer portal?
A portal helps, but it cannot be the only door. Washington's 2026 motor-vehicle rule forbids requiring claimants or repair shops to use only an internet-based application and requires insurers to accept email, post, fax and delivery in person. FNOL software has to read what arrives through each.
What do FNOL services include?
Usually a third party taking first-notice calls on the carrier's behalf, sometimes with software. The questions above still apply, because the notices and messages remain the carrier's. Washington's rule, for example, holds insurers responsible for evaluations made on their behalf.