Clause comparison means reading two versions of a clause side by side, finding exactly which words changed, and deciding whether the change alters the cover. The first part is mechanical and can be made exact. The second is judgment, and experienced readers, including judges with long insurance practice, regularly reach different conclusions on the same words. That matters for anyone comparing wordings at volume, and for anyone deciding whether to trust a tool to help: the fair test of a comparison, by a junior, a senior or software, is how it agrees with two experienced readers who have also been measured against each other. Botminds' Clause Library & Precedent Search is built on that split. Its difference marker shows every changed word and labels the change as scope, trigger, sanctions regime or cosmetic, and never says whether a change is acceptable; that decision stays with the underwriter and the head of wordings.
This guide shows how to compare two insurance clauses in layers, what the English courts' own decisions say about expert disagreement, and how to run a two-reader agreement test on a sample of past clause changes, with the arithmetic worked through. Comparing a whole broker or manuscript wording against a carrier's standard form is a different job, covered in the policy wording comparison tool guide. Product examples come from the solution's fictional sample book.
Six versions of one clause, fanned: added words lit, removed words struck, regimes marked
How do you compare two insurance clauses?
Compare them in three layers, and keep the layers apart. The first is the text: every word added, removed or moved, against a named base, which should be the version immediately before, the carrier's standard and the broker's draft in turn, because each comparison answers a different question. The second is the kind of change: does it alter who or what is covered (scope), what sets the clause off (trigger), which law or regime it refers to, or nothing of substance (cosmetic)? The third is the effect: would the change alter the outcome of a plausible claim?
The layers get harder as they go. Two careful people, or a careful person and a good tool, should agree almost perfectly on the first: a word is either there or not. They should agree well on the second, given written definitions. On the third they will sometimes disagree, and the disagreement is information, not a mistake to be stamped out. Most comparison failures in practice come from collapsing the layers, for instance a redline that shows the words but is read as if it showed the effect, or a summary that states the effect without showing the words.
A single changed word can carry the whole effect. The English Commercial Court read "would expose" in the London market's standard sanctions clause as requiring that payment actually be prohibited, not merely risky, and said that if the parties had meant risk it would have expected them to say so (what "would expose" means sets out the case). A comparison that marks "would" changing to "may" as cosmetic has made a judgment, whether it meant to or not.
What makes a clause change material?
A change is material when it could change whether, when or how much a claim is paid, or which law decides that. There is no statutory definition for wordings work, and carriers write their own; a working test asks five questions of each change. Does it change the insured interest or the parties? Does it change the trigger, or the standard of proof the trigger needs? Does it add, remove or narrow an exclusion? Does it change what happens over time, such as suspension, cancellation or notice? Does it change the law, forum or regime named?
The test sounds objective until two people apply it to the same pair of clauses. Is a sanctions regime added to a list a scope change or a trigger change? Is replacing "manifested" with "diagnosed" cosmetic, or a change in the proof a policyholder must produce? Each question has a defensible answer either way, and that is why the courts' history of reading insurance clauses is the best evidence a wordings team has about how much readers disagree.
Do wording specialists agree on materiality? The courts' own decisions
Not reliably, if the most experienced readers available are a guide. The Financial Conduct Authority's business interruption test case was tried by a court of two judges: "Both judges have extensive knowledge and experience of insurance law," the Supreme Court noted (FCA v Arch Insurance (UK) Ltd [2021] UKSC 1, para 41). They read a representative sample of 21 business interruption policy types, and found that "most of the disease clauses and certain prevention of access clauses (12 policy types from the sample of 21, issued by six insurers)" provided cover (FCA).
The Supreme Court dismissed the insurers' appeals, "for different reasons from those of the High Court", and allowed parts of the FCA's appeal, adding two QBE policy types to those that provide cover (FCA). On the disease clauses themselves, the Justices said that their reasons "also lead us to reach a different conclusion from the court below about the correct interpretation of other sample disease clauses" (para 81). The outcome for many policyholders was the same; the reading of the words was not.
The same disease clauses, read by two expert courts
High Court, September 2020: Cover for the disease itself, wherever it occurred, from the date of the first case inside the radius.
Supreme Court, January 2021: Cover only for cases of illness inside the radius; the wider outbreak is dealt with as a question of causation, not of what the clause covers.
High Court, September 2020: The words are adjectival: they describe which diseases count, and cover follows once one case is manifested in the area.
Supreme Court, January 2021: The words are "adjectival but not conditional": they are part of the description of the risk, so only cases in the area are covered.
High Court, September 2020: Read as the insurers argued, so no cover on these wordings.
Supreme Court, January 2021: The lower court read these two correctly and "was wrong not to interpret the other disease clauses in a similar way". The FCA still won cover on them, through the court’s approach to causation.
High Court, September 2020: Cover under most disease clauses in the sample.
Supreme Court, January 2021: Insurers' appeals dismissed "for different reasons"; more policyholders with valid claims.
Read the middle row twice. The High Court and the Supreme Court agreed on every word of the QBE 1 clause and disagreed on its grammar: whether a phrase describing where a disease is manifested limits the cover or merely identifies the disease. That is precisely the kind of difference a wordings specialist marks as "cosmetic" or "scope" on an ordinary afternoon. Expert readers split on it on the way to the UK's highest court.
Ten years later, the same judges read it differently
Disagreement is not only between people; it is also within one person over time. In the same judgment, the Supreme Court overruled the Orient-Express Hotels decision on business interruption and trends clauses, concluding "on mature and considered reflection" that it "was wrongly decided and ... should be overruled" (para 308). The two Justices who wrote the lead judgment added: "In reaching a different conclusion now to that which we both reached over ten years ago", they would invoke "whatever ways by which we may 'gracefully and good naturedly' surrender 'former views to a better considered position'" (paras 311–312).
For a wordings team the lesson is practical, not philosophical. A precedent that says "we agreed this in 2016, and the head of wordings considered it cosmetic" captures a reading, not a truth about the clause. When the same question comes back, the earlier reading is evidence to weigh, alongside what has happened since: a court decision, a claim, a change in the regime the clause refers to. A precedent search that shows the reason with the version, and the claim outcome after it, lets the reader see whether the old reading held up.
Judges disagree even about what evidence helps. Explaining why pre-contract negotiations are usually excluded when a court interprets a contract, Lord Hoffmann wrote: "Of course judges may disagree over whether in a particular case such evidence is helpful or not", adding that "there is nothing unusual or surprising about such differences of opinion" (Chartbrook v Persimmon Homes [2009] UKHL 38, para 33). A comparison process that assumes one right answer to every materiality question is assuming something the courts do not.
How do you measure agreement between reviewers?
Have two readers judge the same set of clause changes independently, then count how often they agree and correct the count for agreement that would happen by chance. Percent agreement is "calculated as the number of agreement scores divided by the total number of scores", but "it does not take account of the possibility that raters guessed on scores" (McHugh, Biochemia Medica, 2012). Cohen's kappa was "developed to account for the possibility that raters actually guess on at least some variables due to uncertainty", and it is the standard correction.
How agreement levels are usually read, and what each means for a wordings team
| Kappa | Level of agreement | What it means for clause comparison |
|---|---|---|
| 0 to .20 | None | The readers are not applying the same test. Rewrite the definitions before comparing anything else. |
| .21 to .39 | Minimal | The categories overlap; expect arguments on most changes that are not plainly cosmetic. |
| .40 to .59 | Weak | Common for materiality calls without written examples. Usable only with a second reader on every change. |
| .60 to .79 | Moderate | A workable level for change-type labels. Disagreements cluster on a few clause families; look at those. |
| .80 to .90 | Strong | What a team should expect on change type once definitions and worked examples exist. |
| Above .90 | Almost perfect | Expected on the text layer (which words changed); anything less there means a process fault, not a judgment call. |
What is inter-rater reliability? Worked on forty clause changes
Inter-rater reliability is the degree to which two people applying the same test to the same items reach the same result, beyond what chance would produce. Take forty changes from past broker drafts, two experienced specialists, and one question each: is this change material? The figure shows a plausible outcome; the numbers are illustrative, not measured.
Two specialists, forty clause changes: 80% agreement, but a kappa of 0.60
14
Both say material
5
Only reader A says material
3
Only reader B says material
18
Both say not material
Reader A called 19 of 40 material.
Reader B called 17 of 40 material.
They agreed on 32 of 40: 80%.
Chance alone would give about 50% agreement at those rates.
Kappa 0.60: agreement beyond chance is real but modest.
Eighty percent sounds like a team in agreement, and it meets the minimum many texts recommend. The kappa tells a less comfortable story: because each reader called about half the changes material, half their agreement would have happened anyway. The eight disagreements are where the work is. Expect them to cluster: a few clause families (sanctions, cyber, disease) and a few kinds of change (a regime added to a list, a defined term replaced) produce most of them.
Run the test twice a year, on fresh changes, with the readers blind to each other and to who agreed the change originally. Report the kappa by clause family, not only in total. A team-wide 0.60 can hide a 0.85 on property and a 0.40 on sanctions, and the second number is the one that tells the head of wordings where written guidance is missing.
How should AI be compared with human reviewers?
The same way the humans are compared with each other. Score the tool against each reader separately and set its agreement beside the readers' agreement with one another. If the tool agrees with each specialist about as well as they agree with each other, it is inside the human spread on that question; if it falls well below, it is not ready for that question, whatever its overall accuracy figure says. Where the two specialists disagree, the tool should not be marked wrong for siding with either.
The researchers who built CUAD, a widely used benchmark for reading contract clauses, took this seriously. The CUAD dataset's annotators trained for 70 to 100 hours, followed more than 100 pages of rules, and "Each annotation was verified by three additional annotators to ensure that the labels are consistent and correct" (Hendrycks et al., CUAD, NeurIPS 2021). That is the level of care needed to produce a reference answer a tool can be scored against. A vendor that quotes an accuracy figure on wordings should be able to say who the reference readers were and how well they agreed.
Keep the layers separate in the test as well. A tool should be held to near-perfect agreement on which words changed, since that layer is exact. Change-type labels can be scored with kappa against written definitions. Materiality should stay with people, which is how the solution's difference marker is set up: it labels the change and never judges whether it is acceptable. A forty-question test for AI answers about precedents uses the same two-reader scoring.
One version against the one before it: three words changed, with the decision and the outcome
How do you settle a disagreement on a clause?
Write the disagreement down, look at what the carrier decided before and what happened next, and give it to the person with authority to decide, with the reasons kept. The checklist turns that into a routine.
Settling a disagreement between two wordings readers
- State the disagreement in one sentence each
Reader A: "adds a condition to the trigger". Reader B: "describes the same trigger". Most disputes shrink once they are written down.
- Check the text layer first
Confirm both readers compared the same two versions against the same base. A surprising share of disagreements are about different bases.
- Pull the precedents for the same change
How did the carrier decide this change before, for whom, and on what reasoning? An earlier decision is evidence, not a ruling.
- Pull the outcomes
Was a version with this change argued at claim? One outcome line (loss, result, days in dispute) can settle what months of debate cannot.
- Decide at the right level
A change bound before with approval can be decided by the class underwriter; one refused before or never bound goes to the head of wordings.
- Keep the reason with the version
Write why the change was treated as material or not, so the next reader sees the reasoning and can tell whether it still holds.
- Feed the disagreement back into the guidance
If the same kind of change splits readers twice, add a worked example to the definitions and re-test.
The last step is the one that raises the kappa. Teams disagree most where their definitions are silent, and every settled disagreement is a worked example for the next reader. Over time the clause families where readers split become the families with the richest written guidance, which is the right way round. Keeping the decision and its reason alongside the text is the difference between a clause library and a precedent bank. To see the difference marker on a sample book, start from Clause Library & Precedent Search.
Questions people ask
What is the difference between a redline and a clause comparison?
A redline shows which words changed between two documents. A clause comparison goes further: it compares one clause against the right base (the version before, the standard, the broker's draft), labels what kind of change it is, and asks whether it alters the cover. A redline is the first layer of a comparison, not the whole of it.
Can two experienced underwriters read the same clause differently?
Yes, and the courts show it. In the FCA business interruption test case, a High Court of two judges with long insurance experience and the Supreme Court read several of the same disease clauses differently, even where they reached the same practical result. Two of the Justices also overruled a reading they had both reached more than ten years earlier.
What is a good kappa for wording reviewers?
For which words changed, above 0.90. For the type of change, against written definitions, 0.80 or more is achievable. For materiality, expect less without worked examples: the illustrative test above lands at 0.60, and McHugh's scale calls 0.60 to 0.79 moderate, and many texts treat 80% raw agreement as the minimum acceptable.
Should a tool decide whether a clause change is material?
It can propose a label, but the decision should stay with a person who has authority, because experts disagree and the carrier must be able to explain its choice. In Clause Library & Precedent Search the difference marker labels each change as scope, trigger, sanctions regime or cosmetic and never judges whether it is acceptable.
How many clause changes do you need for an agreement test?
Forty is enough to see whether readers broadly agree and where they split; a hundred gives a more stable kappa by clause family. Draw the changes from real broker drafts across several families, and make sure some are plainly cosmetic so the test is not all hard cases.