One underwriting question on homeowners insurance, two answers

An underwriting question on homeowners insurance is any question whose answer decides whether a home can be written, referred or declined: the roof's age and covering, the heating, the distance to the coast, a dog or a pool, the losses, the rebuilding cost. The uncomfortable truth about these questions is that the same one, asked of two experienced underwriters on the same desk, can come back with two answers, and few desks measure how often. That rate of disagreement, not perfection, is the baseline any assistant has to beat. Botminds' Underwriting Guidelines Assistant answers each question from the carrier's own guidelines, quotes the clause and its revision, applies state law and gives the same answer to the same question, so the agent's result no longer depends on who picked up.

The examples on this page come from that fictional carrier, Kestrel Point Mutual, and its desk as they run through the solution; every person, agency, address and quote is sample data. The worked agreement test further down is illustrative arithmetic, labelled where it appears.

What is an underwriting question on a homeowners policy?

On a homeowners policy, an underwriting question is one the carrier asks, or an agent asks the carrier, to decide whether the risk fits its rules and on what terms. Customers meet them on the application ("When was the roof last replaced?", "Is the home ever rented?"); agents meet them when the answer is not on the quick-reference card and they have to ask the desk. Both kinds are answered from the same source: the carrier's underwriting guidelines, as modified by its bulletins, its agents' binding authority and the state's law.

What makes a question an underwriting question is that a different answer changes the outcome. "Is the fireplace masonry or a wood stove insert?" is one: on the sample carrier the first is eligible and the second needs proof of how it was installed. "What colour is the house?" is not. A desk's consistency problem lives entirely in the first kind, which is why it can be measured: there is a finite list of topics, and each has a right answer on a given date.

Why do two underwriters give two answers to the same question?

Two underwriters give two answers because they are reading two different rules, not because one of them is careless. On the sample desk the conflicts that produced different answers had four sources, and each one is a document problem before it is a people problem.

The first is two copies of one rule. The carrier's roof section changed on 1 October; the portal carried the new text that morning, but the June quick-reference card and the agents' printed manuals still carried the old one. An underwriter who learned the rule in the summer and one who read the bulletin give different answers to the same 17-year shingle roof. The second is a memo stricter than the manual: an appetite memo stopped new business within one mile of the coast in one South Carolina county while the manual's coastal band was 1,000 feet, and nine questions that month fell in the gap between the two. The third is a state exception page that never caught up with a bulletin. The fourth is memory: the person who wrote the rule answers from what it meant, and the person who reads it answers from what it says.

Each of those is invisible from inside one conversation. The agent hears one confident answer and has no way to know a colleague would have said something else. Where each rule lives maps the five sources an eligibility answer draws on and the error each one tends to cause.

How often do underwriters disagree? Measure it before you blame anyone

Measure disagreement the way researchers measure agreement between two raters: put the same set of questions to two underwriters independently and compare their verdicts. The simplest number is percent agreement, the share of questions both answered the same way. It is easy to read and easy to flatter, because two people will agree on some answers by chance alone.

That is why the standard measure corrects for chance. "In 1960, Jacob Cohen critiqued use of percent agreement due to its inability to account for chance agreement," Mary McHugh writes in her widely cited lesson on the statistic; his kappa compares the agreement observed with the agreement two raters would reach by guessing in the same proportions (McHugh, Biochemia Medica, 2012). The formula is short: κ = (observed agreement − chance agreement) ÷ (1 − chance agreement), where chance agreement is the sum, across the answer categories, of the two raters' shares multiplied together.

McHugh's reading of the result is stricter than the one most people learn. Cohen's original bands call 0.41 to 0.60 moderate; McHugh argues that "any kappa below 0.60 indicates inadequate agreement among the raters and little confidence should be placed in the study results," and notes that many texts recommend 80% as the minimum acceptable percent agreement. On an underwriting desk, an answer the agent acts on is the study result.

A worked agreement test: forty questions, two underwriters

Here is the arithmetic on a small test, set out so a desk manager can repeat it on real questions. Two underwriters each answer the same 40 eligibility questions, drawn from a month of agent questions, without seeing each other's answers. Each gives Yes, Refer or No.

The cells off the diagonal are the interesting ones. Seven of the eight disagreements were one underwriter referring while the other gave a firm answer, and which of them was right depends on the text in force. If the rule required a referral, the firm Yes put a risk on the books outside the guidelines; if it did not, the Refer cost the agent a day and perhaps the customer. The eighth sits in the corner that should stay empty: a Yes from one underwriter and a No from the other on the same home. A desk that counts only agreement treats all eight alike.

A matching verdict is not always the same answer

Two underwriters who both say Refer have agreed on a verdict, but if one refers for the roof and the other for the coastal distance, the agent hears two different next steps. A useful agreement test therefore scores two things: the verdict, and the clause that decided it. On the sample carrier the clause is the section and revision of the manual, the appetite memo, the binding authority or the statute the answer rests on.

Regulators push the same way from the consumer's side. Florida requires the specific reasons for a denial, including the specific underwriting reasons (Fla. Stat. §627.4091), and Delaware's Department of Insurance told insurers in 2025 that general reasons such as claims experience or underwriting judgment are not acceptable explanations on cancellation and nonrenewal notices (Delaware Bulletin No. 150). An answer that cannot name its clause cannot become a specific reason later.

Where homeowners underwriting questions cluster, and so where disagreement costs most

Disagreement costs most where the questions are, and on a homeowners desk the questions bunch into a few topics. Over a month the sample desk took 1,286 questions; two topics, the roof and wind restrictions, took two questions in five, and the four largest took nearly two thirds.

That shape tells a desk manager where to spend an agreement test. Forty questions drawn at random will be mostly roof and wind, which is right, because that is where most agents' customers are waiting. But the smaller topics are where rules differ most by state and change least often, so a second test of ten questions on dogs, heating and rentals finds the disagreements nobody has looked at for a year.

Every question, its verdict and the passage it rested on

Consistency is an examination standard, not a matter of style

Inconsistent answers are a regulatory exposure as well as a service problem, because examiners test whether a carrier applies its own rules the same way to the same risks. The NAIC's property and casualty examination standards include this one: "Credits, debits and deviations are consistently applied on a non-discriminatory basis" (NAIC examination standards, chapter 21). The underwriting standards also ask whether the carrier follows its own guidelines in selecting risks.

Texas writes the substance into statute. Personal auto and residential property underwriting guidelines must be sound, actuarially justified or otherwise in line with the risk, and may not be unfairly discriminatory (Tex. Ins. Code §38.002(e); what Texas makes public is in are underwriting guidelines public record). Two answers to one question about the same risk cannot both be commensurate with it. Even when neither answer was meant unfairly, a pattern of different answers for similar homes is the kind of thing an examiner's file review looks for.

The baseline for an assistant is your desk, not perfection

Judge an assistant against the agreement your own underwriters reach with each other, not against an imagined desk that never disagrees. If two senior underwriters reach a kappa of 0.67 on a test set, an assistant that matches the adjudicated answers at 0.85 is better than either of them on that set, even though it gets some questions wrong. One that reaches 0.55 is worse than the desk, however fluent it sounds.

Adjudication is the step most tests skip. After two underwriters answer independently, the guideline owner settles each disagreement against the text in force, and that settled answer becomes the key. The Underwriting Guidelines Assistant is built to be scored this way: every answer names its clause and revision, so a disagreement with the key is either a misread passage, a wrong revision or a missing state rule, and each of those is a fix with an owner. Where its own sources disagree, as in the coastal memo, it does not pick one: it refers the question and shows both texts. The KPIs that pay turns the same counts into the numbers a head of underwriting reports.

The same roof questions, answered under two revisions

How to run an agreement test on your own underwriting desk

An agreement test needs a week and three people: two underwriters who answer, and the guideline owner who adjudicates. It produces two numbers a head of underwriting can track quarter by quarter, and a list of clauses to rewrite.

Questions people ask

What are homeowners insurance underwriting questions?

They are the questions a carrier uses to decide whether a home fits its rules: roof covering and age, construction, heating, distance to the coast, rebuilding cost, prior losses, occupancy and liability features such as pools, trampolines and dogs. On the sample desk, roof questions alone were about a quarter of a month's 1,286.

What questions are asked for homeowners insurance?

Applications typically ask about the property (year built, construction, roof, heating, plumbing and wiring), its use (owner-occupied, rented, vacant), its exposures (pool, trampoline, dog, home business) and its history (losses and prior cover). The specific list is the carrier's, and in Texas the guidelines behind it are filed with the Department.

What is a good agreement rate between underwriters?

There is no regulatory figure. Research practice treats 80% percent agreement as a common minimum, and McHugh argues a kappa below 0.60 shows inadequate agreement; a desk should know its own figure before judging any tool against it.

Why do underwriters give different answers to the same question?

Usually because they are reading different versions or sources of the rule: an old printed manual against a new bulletin, an appetite memo against the manual, a state page that missed an update. On the sample desk, 23 questions in a month were answered two ways before the solution.

Are underwriting questions and answers the same for every insurer?

No. Each carrier writes its own guidelines within state law, so the same home can be a Yes for one carrier and a referral for another. What is the same everywhere is that the answer should come from the carrier's written rule in force on the quote date, not from memory.