Covenant scoring system: when two analysts score one indenture

A covenant scoring system turns an indenture's protections into a number: each covenant area — restricted payments, investments, debt, liens, asset sales, change of control and the rest — gets a sub-score on a fixed scale, the sub-scores are weighted, and the total ranks the deal against others. Its weakness is hidden in its strength. The number looks objective, but it is only as reliable as the agreement between two analysts scoring the same document, and that agreement is rarely measured. Botminds' Bond Covenant Analysis scores each deal on a firm's own versioned rubric with every sub-score tied to quoted clauses, and this guide shows how to test any scoring system — including that one — by having two people score one indenture and reading where they split.

The worked example uses sample data on fictional companies: the $650 million of 7.25% senior notes due 2031 that Dunmore Cinemas Holdings is marketing, scored on the sample firm's rubric in the solution. The two-analyst comparison is our illustration built on that deal. Research and statistical guidance is linked where it appears.

What is a covenant scoring system, and what is it for?

A covenant scoring system is a rubric that converts an indenture's covenants into comparable numbers, so a portfolio manager can tell a loose package from a tight one without reading 400 pages. It is for comparison: this deal against recent deals, this bond against the issuer's last bond, the held book against itself after an amendment. A score that cannot be compared is just an opinion with a decimal point.

Academic finance has scored covenants this way for decades. A Journal of Finance study built "firm-level indices of bondholder covenant protection" for public debt and found that "covenant protection is increasing in growth opportunities, debt maturity, and leverage" (Billett, King and Mauer, 2007). An index built from a covenant database can show which covenants a bond carries; a practitioner's rubric also grades how wide each covenant's exceptions are, which is where judgement — and disagreement — comes in.

Why do covenant scores differ between two analysts?

Covenant scores differ because the rubric leaves room for judgement exactly where the documents are hardest: how to count a builder that started before the issue date, whether one basket can be used for another's purpose, and how much a missing blocker should cost. Two careful analysts can read every clause correctly and still score differently if the rubric does not say how to translate a reading into a number.

Take the sample deal and two analysts, A and B, scoring it independently on the same rubric. They agree on five of eight areas. They split on three: restricted payments, investments and liability management blockers — the areas that hold the builder, the general basket and the unrestricted subsidiary basket. The weighted totals come out at 2.1 and 2.7.

How are bond covenants scored? Weighted sub-scores on a house rubric

Bond covenants are usually scored area by area on a short scale, then weighted by how much each area matters to recovery and to value leaving the group. In the sample firm's rubric, version 3.2, restricted payments and investments each weigh 20%, liability management blockers and debt 15% each, liens and asset sales 10% each, and change of control and reporting 5% each, on a scale where 1 is loosest and 5 tightest. The head of credit research owns the weights; changing them needs her approval and goes on the audit trail.

The weights tell an analyst where precision pays. The two heaviest areas are the ones where a single defined term — what counts as a restricted payment, whether a basket reloads, when a builder starts — swings the dollars. A rubric that weights them heavily without defining how to read those terms is building its largest error into its largest weights.

Where two analysts split: builder start dates and reloading baskets

The split sits in the two places this indenture is hardest to read, and both are questions of definitions, not of effort. The builder counts 50% of net income from January 1, 2022, almost five years before the notes are issued, so on day one it holds $79.1M rather than its $40.0M starter. Analyst B scored the builder as if it started at issuance. The general restricted payments basket of $57.9M can fund investments, because the definition of a restricted payment includes a restricted investment; analyst B scored it as a dividend basket only.

The third split is the one counsel had to decide on the real deal: whether returns on investments reload the $75.0M unrestricted subsidiary basket above its original size. The sample firm's counsel ruled that returns restore the basket up to $75.0M but do not enlarge it. A rubric that says nothing about reloading leaves each analyst to decide alone, and the score depends on who happened to read the deal. How restricted payment baskets stack works the builder arithmetic line by line.

The consequence is not academic. At 2.1 the sample firm's settings call for a second approval from the head of credit research; at 2.7 they do not. A disagreement about two definitions decides whether the most senior reviewer ever sees the deal.

How to measure inter-rater agreement on covenant scores

Measure agreement the way any rating study does: have two analysts score the same set of indentures independently, then compare their sub-scores. The simplest measure is percent agreement — the share of sub-scores on which they match. Cohen's kappa corrects that share for the agreement two raters would reach by chance, which matters when a rubric uses few scale points.

A guide to kappa published in Biochemia Medica explains the benchmarks and warns against reading them generously: "Cohen suggested the Kappa result be interpreted as follows: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement. However, this interpretation allows for very little agreement among raters to be described as 'substantial'." It adds that many textbooks set 80% agreement as the lowest acceptable level between raters (McHugh, 2012; the same test applied to contract wordings is in clause comparison: when expert readers disagree). The statistics behind such benchmarks go back to Landis and Koch's 1977 work on "generalized kappa-type statistics" for observer agreement (Biometrics).

The last row is the one most firms miss. Two analysts can land within 0.2 of each other on the total while disagreeing by two points on restricted payments and two in the opposite direction on asset sales. Totals hide the split; sub-scores show where the rubric needs words.

What is covenant quality? A position against peers, not a point on a scale

Covenant quality is how much protection an indenture gives relative to comparable deals, so a score means something only against a peer set: same rating band, region, sponsor status, sector and vintage. A builder worth 20% of EBITDA is loose in one market and ordinary in another, and the rubric should say which peers define "ordinary".

In the sample firm, the peer set for Dunmore is 318 deals from its own library: US high yield, B-rated, sponsor-backed, consumer and leisure, issued 2024–2026. Against them, the unrestricted subsidiary basket at 19.4% of EBITDA is looser than 88% of peers, against a median of 10.4%, and the nearest comparable deals without a J.Crew blocker score 1.7 to 2.3. Every percentile in the solution shows the peers behind it, so an analyst who disagrees with a sub-score can see which deals produced it.

Bond Covenant Analysis: covenant scoring against the firm's own library

Are covenant scores priced? What the research says

Yes: covenants affect what a borrower pays, which is why scoring them is worth the effort. In a large sample of privately placed corporate debt, Bradley and Roberts report "a negative relation between the promised yield on corporate debt and the presence of covenants," and conclude "that covenant structure and the yield on corporate debt are determined simultaneously" (Quarterly Journal of Finance, 2015). A 2026 study measuring covenant prices from US corporate bond trading finds that "covenant prices vary significantly over time" and "increase with the riskiness of bonds" (Handler, Jankowitsch and Weiß, Review of Asset Pricing Studies).

The weak point is the measurement. Ivashina and Vallée, who built their own covenant-weakness measures from 1,240 loan agreements, note that industry scoring services had begun to score loan contracts and that "the extent to which such measures actually capture covenant weakness, and whether they complement previously used measures is yet unproven" (NBER w27316). The same is true of a house rubric until someone tests it: priced or not, a score that two analysts cannot reproduce cannot be priced reliably either.

Calibrating a covenant scoring system before anyone trusts it

A firm can test its covenant scoring system in a few weeks with documents it already holds. The steps below work whether the scores come from people or from a tool, and they should be run again whenever the rubric changes.

Bond Covenant Analysis keeps the parts of that study a firm needs every day. The Benchmark agent "never changes the rubric", and every rubric change goes to the head of credit research for approval and can be rolled back to an earlier version. The agents' sub-scores carry their quotes, so a calibration split can be traced to the clause and the reading behind it. And because each new agent or model version is tested on a golden set of 240 annotated documents before going live in the sample firm, the tool's own consistency is measured the same way an analyst's would be. What a firm brings to start — its past scores, rubric versions and library — decides how quickly that test can run.

Bond Covenant Analysis settings: the thresholds that turn a score into an approval route

Questions people ask

What is a covenant quality index?

A covenant quality index is a score or scale that summarises how protective a bond's or loan's covenants are, so packages can be compared across deals and over time. Research indices can be built from which covenants a debt contract includes; practitioners' rubrics also grade how wide each covenant's exceptions are. Either is only as reliable as the rules for reading the hardest clauses.

What weights should a covenant scoring system use?

Weights should follow where value can leave and where recoveries are decided: payouts, investments and transfers to unrestricted subsidiaries, debt and liens usually carry most of the weight. The sample rubric puts 40% on restricted payments and investments together, 30% on debt and liability management blockers, 20% on liens and asset sales and 10% on change of control and reporting. Whatever the weights, version them and note who approved each change.

How do you compare covenant packages across deals?

Put every basket in dollars and as a share of EBITDA, then rank it against a defined peer set: rating band, region, sponsor status, sector and vintage. Compare sub-scores area by area, and show which peers produced each percentile. A headline score alone hides whether two deals are loose in the same place.

Can a covenant scoring tool be more consistent than analysts?

It can be more consistent with itself, because it applies the same reading every time, but that is only useful if the reading is right. Test it against two analysts on indentures they have already scored; where the analysts themselves disagree, fix the rubric first. In Bond Covenant Analysis a disputed reading goes to legal counsel rather than being decided by the tool.

How many indentures do you need to test inter-rater agreement?

Enough to produce at least 30 comparisons, the floor the kappa guide cited above gives for a usable estimate; with eight sub-scores per deal, a few indentures reach that count, but 20 to 30 deals are needed to cover the hard clauses. Include deals with known routes and disputed readings, because agreement on easy documents tells you little.