An AI credit analyst is software that reads an issuer's documents and answers a credit analyst's questions about them in plain words: how much secured debt the credit agreement allows, when the revolver matures, what management said about a bond. The answer is worth something only if each sentence can be traced to the page that supports it, because a credit memo is signed by a person who must be able to defend it. That is why, in Botminds' AI Credit Analysis, the agent that decides what reaches the analyst is not the one that writes but the Citation Checker: it re-reads every cited passage, recomputes every figure, and removes any sentence it cannot tie to a page.
The example is one sample answer as it runs through the solution, about a fictional issuer, Halvorsen Packaging, held by a fictional fund group, Quillfield: sample data on fictional people and companies. The research on how often AI answers are cited correctly comes from peer-reviewed studies, linked in the figure captions.
Fluent answers, weak citations: what researchers measured
51.5%
of sentences from four popular generative search engines fully supported by their citations, on average
74.5%
of those engines’ citations actually supported the sentence they were attached to
50%
of the time, even the best models tested lacked complete citation support on one long-answer dataset
79
distinct carve-outs in the average leveraged loan credit agreement, each a defined term an answer may need
What is an AI credit analyst?
An AI credit analyst is a set of models that do the reading and arithmetic a credit question needs, while a person makes the call. The useful definition is narrower than the marketing one: software whose every sentence can be traced to a page, whose every figure can be recomputed, and which says what it did not read. Anything less produces a draft that someone still has to verify from scratch.
In AI Credit Analysis eight agents share the work. A Wall agent decides what the person may read; a Planner splits the question and waits; Indenture, Filings and Call agents read; a Cross-reference agent reads documents against each other; a Memo agent writes the approved sentences into a credit memo. Between the readers and the analyst stands the Citation Checker, and its rule is locked on: no uncited sentence reaches an answer.
Why citations are the hard part for AI
Language models write fluent sentences far more reliably than they attach the right source to them. When researchers audited four popular generative search engines, they found that "a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence" (Liu, Zhang and Liang, Findings of EMNLP 2023). A benchmark built to test cited answers found the same gap from the other side: on one of its datasets, half of the answers from the strongest models were not fully supported by the sources they cited (Gao et al., EMNLP 2023).
"even the best models lack complete citation support 50% of the time"
Gao et al., Enabling Large Language Models to Generate Text with Citations, EMNLP 2023
Both studies tested 2023 systems on general questions, and models have improved since. The lesson for a credit desk does not depend on the year: a citation printed next to a sentence is a claim, not proof. A system that wants its citations trusted has to check them itself, sentence by sentence, before a person reads them.
Can AI do credit analysis?
AI can do the reading and the arithmetic of credit analysis; it cannot be left to do the judgement alone. FINRA's 2026 oversight report lists the risks of agents in plain words, and two of them describe credit work exactly: reasoning spread over many steps can leave an outcome hard to trace or explain, which makes it hard to audit, and an agent built for everything may lack the specialist knowledge that complex, industry-specific tasks need, and so perform them unevenly (FINRA, 2026 Annual Regulatory Oversight Report; the report's full list is set against working controls in AI earnings call analysis: why the reader is never the checker).
Credit documents make both risks concrete. Ivashina and Vallée found that "the average credit agreement includes 79 distinct carve-outs" across the 1,240 agreements they studied (NBER Working Paper 27316), and an answer about debt capacity may turn on any of them. A general model that reads a covenant without its carve-outs, or an agreement without its amendments, writes a confident wrong answer. The fix is not a better writer; it is a reader that applies amendments in order and a checker that will not pass a figure it cannot recompute.
One answer, from question to signature
The Citation Checker runs twice: once before the analyst sees the answer, and again when the senior analyst opens the memo to sign. The rest of the time on the clock belongs to people. In the sample, the analyst asks whether Halvorsen can put new debt ahead of the fund's 2029 notes; the agents answer in 96 seconds, and the memo is signed 52 minutes after the question.
Of 52 minutes from question to signature, the agents took less than two
- Agents read; the checker removes 1 sentence and recalculates 196 s
- Held: sentence 7 below 0.85 waits for the analyst6 min
- Analyst decides, acknowledges the house-note conflict, submits10 min
- In the senior analyst’s queue30 min
- Checker re-reads; senior analyst checks 3 flags and signs4 min
Two holds sit on that clock, and both are the checker's doing. Sentence 7 scored 0.83, under the 0.85 line, so it could not go into a memo until the analyst decided it. And the answer conflicted with the house's own note of June 12, which had put secured capacity under $150 million; the memo push stayed blocked until she acknowledged it. Neither hold is a failure. Both are the system saying what a careful colleague would say: this one needs you.
Eight sentences, each pinned to the tranche it concerns
Eight sentences, and what the checker did to each
The checker does not grade an answer; it grades sentences. Each one either matches its passage, is corrected, goes to a person, or never appears. In the sample it removed one sentence that had no source and corrected one figure that had used EBITDA from a press release rather than the figure the credit agreement defines.
The sample answer, sentence by sentence, as the Citation Checker left it
| Sentence | What it says | Checker outcome |
|---|---|---|
| 1 | Incremental first-lien debt of the greater of $350M and 57% of EBITDA, raised from $250M by Amendment No. 2 | Cited (passages 1–3); confidence 0.97 |
| 2 | About $128M more of ratio debt: 4.00 × $612M less $2,320M already secured | Recalculated: the draft had used a press-release EBITDA |
| 3 | The 2029 indenture allows credit-facility debt up to the greater of $2,300M and 3.75x EBITDA, $730M unused | Cited (passages 6–8); confidence 0.94 |
| 4 | New first-lien debt ranks alongside the 2030 secured notes, not ahead of them | Cited (passages 9 and 10); confidence 0.96 |
| 5 | The revolver springs to December 14, 2028 if more than $150M of the 2029 notes remain | Cited to credit agreement §2.05(c), p.57; the link to the 10-Q proposed by the Cross-reference agent and confirmed by a person |
| 6 | The CFO is "evaluating all options for the 2029s" | Quotation cited to the call transcript (passage 12); a quotation is never edited |
| 7 | A Norwegian non-guarantor owns the barrier-film patents and may borrow up to $153M | Confidence 0.83, under 0.85: sent to the analyst |
| 8 | In all, about $631M could rank ahead; 3.85x–4.83x becomes 4.88x–5.86x | Computed from sentences 1, 2 and 7; flagged for the checker |
| (9) | A draft sentence with no passage behind it | Removed, with the reason shown |
How does AI cite its sources, and how do you know a citation is real?
A citation is real when a second process, not the one that wrote the sentence, opens the cited page and finds the claim there. Most AI tools attach sources at generation time and stop. The checker treats the writer's citation as a proposal and runs four tests on it before anyone reads the sentence, and again when the memo is opened for signature.
Four tests every sentence must pass before a person reads it
- The passage says what the sentence says
The checker re-reads the cited passage at its page and scores the match. Below the 0.85 threshold the sentence goes to the analyst to confirm, edit or remove with a reason code.
- Every figure recomputes
Each number is recalculated from its inputs. A difference above the 0.5% arithmetic tolerance is corrected and shown as recalculated, never silently replaced.
- The document is the current version
Citations carry the document version read. When the memo is opened for approval, every passage is re-read against the versions in force, so an amendment filed in between is caught.
- Quotations stay verbatim
The Call agent quotes management exactly, by speaker, and never states intent as fact. The checker never edits a quotation, so a quote cannot be smoothed into something the speaker did not say.
Can AI read an indenture?
It can, if it reads the way a lawyer does: the document as amended, definitions first, carve-outs with the covenant they cut. In the sample the Indenture agent read a 318-page credit agreement conformed through Amendment No. 2, both indentures and the intercreditor, and traced "Incremental Cap" to "Consolidated EBITDA" before computing anything. Undefined or conflicting terms are flagged, not guessed.
What it does not do is decide what a definition leaves open. Sentence 7 turned on whether the Norwegian subsidiary's barrier-film patents count as "Material Intellectual Property", a judgement the credit agreement leaves to the borrower. The agent showed the definition and the annual report's figure, barrier film at 41% of group sales, and left the call to the analyst. Questions about lender protections in loan documents belong to AI agents for credit agreement review, a separate solution.
How accurate is AI credit analysis?
Accuracy depends on the documents, the questions and the controls, so no published figure transfers to your desk. What a desk can measure is how its own system behaves: how many sentences the checker removed, how many figures it recalculated, how many sentences fell below the confidence line, and how many of those the analyst changed. In the sample house, 6.4% of sentences reach a person today.
Those rates are the honest substitute for an accuracy claim, and they should be tested on your own past questions before purchase. A two-week pilot on your own notes sets out the protocol, including a golden set of questions with known answers; in the sample house, every new agent version runs against 312 of them before it is used.
What a reviewer can ask of an AI-written sentence
A reviewer can ask the same question of an AI sentence as of any research input: is it sound, and can you show me why? The CFA Institute's standard on diligence says that members who "rely on secondary or third-party research" must "make reasonable and diligent efforts to determine that such research is sound" (CFA Institute Standard V(A)). A sentence that carries its page, its document version and its checker outcome can be judged sound in minutes; one that does not has to be re-researched.
The SEC's examiners say the technology supplements their own staff's judgment rather than replacing it, and they examine firms on the same footing: the Division is looking harder at how the technology shapes outcomes and at whether firms still meet obligations that apply whatever tool is used (SEC, FY2027 Examination Priorities; what that means for agents' authority is set out in AI stock alerts: what agents may decide). For a credit desk that means the analyst's reasonable basis, the second signature and the books and records are judged as before; the answer just arrives with its evidence attached.
The analyst decides the one sentence the checker would not pass
What does an AI credit analyst cost, and which claims should you believe?
Vendors don't publish prices for AI credit research, so the cost question becomes a measurement: hours saved per question against the licence, on your own documents. The claims question is sharper. In March 2024 the SEC settled charges against two advisers for statements about AI they did not use, with penalties of $225,000 and $175,000, and the SEC's chair at the time put the rule plainly: an adviser must not tell the public it uses an AI model when it does not (SEC press release 2024-36; the cases since are set out in AI washing: what the SEC charged).
That cuts both ways. A fund that tells its investors its research is AI-assisted must be able to show how; a vendor that promises accuracy must be able to show it on your files. The arithmetic of the purchase is in three ways to cover 54 issuers, priced, and the full set of agents is on the AI Credit Analysis page.
Questions people ask
Can AI replace a credit analyst?
Not in this design. The agents never approve a memo, cross the wall or decide a sentence below the confidence threshold; the analyst decides and a senior analyst signs. The wider question of which tasks move to software is argued in will AI replace credit analysts.
Does AI hallucinate in finance?
Yes, which is why the checker exists. In a 2023 human audit of four generative search engines, only 51.5% of sentences were fully supported by their citations. A credit system has to remove or route unsupported sentences before a person reads them, not after.
What is a confidence threshold in AI credit analysis?
The score below which a sentence is not allowed to stand without a person. In the sample it is 0.85, settable from 0.70 to 0.95 with a second signature; sentence 7 scored 0.83 and went to the analyst. Translation has its own line, 0.90, for foreign-language passages.
How does the checker handle an amendment filed after the answer?
It re-reads every citation against the current document versions when the memo is opened for approval, and a watched answer is re-run when a new amendment or filing arrives. In the sample, Amendment No. 4 at another issuer changed a watched answer overnight and put it back in the analyst's queue.
Is an AI credit analyst a model under bank model risk guidance?
Bank model risk rules do not settle it either. SR 26-2, the April 2026 successor to SR 11-7, expressly excludes generative and agentic AI models, and it is written for banks rather than asset managers. How banks are treating the question is covered in SR 26-2 and AI.