SearchCollections, Recovery & Special Assets

Evaluate · citations you can rely on

AI hallucination citations: the test a cited loan history must pass

A fake case is only one of four ways a citation fails, and it is the easiest to catch. This guide sets out the other three, the check that catches each, and the afternoon test a buyer can run on a closed file before relying on a machine-written history in a negotiation.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

An AI hallucination citation is a reference an assistant produces that does not support what it is attached to — and the phrase covers four quite different failures, only one of which is the invented case everybody has read about. The source may not exist. The source may exist and say something else. The source may exist and say it, but on a different page from the one named. Or the sentence may carry no citation at all, which in a long answer is the easiest of the four to miss. Asking "does it cite?" catches the first and almost none of the rest. The useful test is whether every sentence has been read back against the page it names, and what happens to the ones that fail. Special Assets File Review is built around that test: it answers questions about one troubled loan's file, one claim per sentence, each with the page it rests on, and a sentence its citation checker cannot support is removed before anyone sees it.

The worked example is the sample file the solution ships with, loan 70-418226 to Ridgeline Tower Partners LLC, $15,976,240 unpaid and rated 7 Substandard, with a first meeting on 20 October. Every person, bank, firm, borrower and property in it is fictional. Linking every answer and every fact to its source page is a design target, not a claim that the solution cannot be wrong; the whole point of the check below is that it sometimes is, and says so.

What is an AI hallucination citation?

It is a citation that looks like evidence and is not. The failure is usually described as a model inventing a source, because that is the version that made the news, but the underlying problem is more general: the text and the reference are produced by the same process, and nothing in that process requires the reference to have been read. A citation generated alongside a sentence is a prediction about where the sentence might be supported. A citation checked after the fact is a finding about where it actually is. Only the second is worth anything in a negotiation.

That distinction has a name in the risk literature. The AI Risk Management Framework separates the two things a reader needs: "Explainability refers to a representation of the mechanisms underlying AI systems' operation, whereas interpretability refers to the meaning of AI systems' output in the context of their designed functional purposes" (NIST AI 100-1, January 2023). A citation is the second kind of thing. It tells the reader what the answer means in the context of this file, and it is only useful if a reader can open it and arrive where the sentence said they would.

Four ways a citation fails, and only one of them is a fake case

Of the four, the invented source is the easiest to catch, because its absence is a fact about the world — and over a closed file it can be ruled out by construction. The other three are facts about the relationship between a sentence and a passage, and only a reader who opens the passage finds them.

The four failure modes, and what catches each

How the citation failsWhat a reader noticesWhat catches it before a reader doesWhat it costs in a negotiation
The source does not existNothing at all, until they try to open it. A plausible title, a plausible date, no document.A closed file makes this one structural: every citation resolves to a document already in the file or the sentence cannot be written.Total. One unopenable reference in a memo ends the discussion about everything else in it.
The source exists but says something elseOnly on reading it. The citation opens, the document is real, the sentence it is said to support is not in it.Reading the sentence back against the cited passage and asking whether the passage entails it.The worst of the four: the claim survives a spot check and fails under cross-examination by the borrower’s counsel.
Right source, wrong page or sectionWhen they go to the page and have to hunt. Usually forgiven, and usually a sign nothing was verified.Citing the passage the sentence was drawn from rather than the document, and opening at it.Time, and credibility. A reader who has to search twice starts checking everything.
The sentence has no citation at allRarely. In a seven-sentence answer the uncited one reads like a summary, and summaries are where conclusions hide.A rule that an uncited sentence does not ship, applied before the answer is shown rather than after.Highest of all in a workout: the uncited sentence is usually the legal conclusion nobody is entitled to draw.
Our classification, drawn from what each failure requires in order to be detected. The third column describes checks a buyer can ask any supplier to demonstrate; the fourth is our assessment of the consequence on a troubled loan file.

The fourth row deserves its place at the bottom of the list. In the sample file the question "is the borrower right that we promised another extension?" has a correct, fully cited answer and one obvious continuation — that the bank is therefore free of any obligation — which is a legal conclusion about waiver and course of dealing that no passage in the file supports. The answer agent does not write it, and if it did, the citation checker would remove it.

What a court said when fake citations reached a filing

The clearest statement of what this failure costs is a sanctions opinion, and it is worth being precise about its subject: it concerns lawyers, a federal court filing, and the rules that govern attorneys. It is not about banks or credit memos. It is useful by analogy, because the mechanism it describes — a professional relying on produced text without opening what it cited — is exactly the mechanism at work when a workout memo goes to a borrower's counsel.

In Mata v. Avianca the court found that the respondents "abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question" (S.D.N.Y., 22 June 2023). The second half of that sentence is the part most readers skip and the part that matters: the court said the record would have looked quite different had the lawyers come clean when the cases were first questioned, so the sanction turned on the standing-by as much as on the first mistake.

"Many harms flow from the submission of fake opinions. The opposing party wastes time and money in exposing the deception. The Court's time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents."

Mata v. Avianca, Inc., Opinion and Order on Sanctions, 22 June 2023

Read that list with a workout file in mind and the analogy writes itself. The opposing party is the borrower, who spends their counsel's time exposing an error in the bank's chronology. The court's time is the credit committee's. And the client deprived of arguments is the bank, which had a good point about what the amendment actually said and lost the room's attention to a bad one about something else.

Has anyone been sanctioned for fake AI citations?

Yes — lawyers have, in filings, and that same opinion is the best-known instance. What the court did not say is as important as what it did. It saw nothing wrong in principle with a lawyer using a reliable AI tool for help; it held that the existing rules still make the lawyer the gatekeeper for the accuracy of a filing. The tool was not the respondent. The duty did not move.

Carry that across to a bank and the question for a buyer stops being "can I trust the model" and becomes "what does this product do to make my gatekeeping cheap enough to actually perform". A product that produces a fluent history with footnotes makes it more expensive, because every footnote now has to be checked by hand. A product that opens each citation at the passage it was drawn from, and refuses to ship sentences it cannot support, makes it cheaper. Both can be called cited. Only one of them reduces the work.

How do you check AI citations? Read the sentence back against its page

One claim per sentence, one passage per claim, and a second pass that reads the sentence against the passage and asks whether the passage entails it. That is the whole method, and its value comes from the two constraints rather than the checking: a sentence carrying three claims cannot be checked against one page, and a claim spread over three sentences cannot be checked at all.

The life of one sentence in a cited answer

  1. Agent

    File reader

    Splits every document, email and call note into passages carrying page and section, so a citation can point at a passage rather than a file name.

    Never: Never edits or deletes a source.

  2. Agent

    Answer agent

    Drafts the answer one claim to a sentence, each matched to the passage it was drawn from.

    Never: Never answers a legal question, and never recalculates an amount — figures are quoted as the document states them.

  3. Agent

    Citation checker

    Reads each sentence back against its passage and asks whether the passage supports it.

    Never: Never rewrites the sentence to fit the page.

  4. Agent

    Citation checker

    Removes any sentence it cannot support, before the answer is shown.

    Never: Never shows an uncited sentence to a reader.

  5. A person decides

    The officer

    Reads the answer and opens the citations that matter, landing on the highlighted passage itself.

    Never: Never has to take the answer on trust: every sentence opens.

  6. Agent

    Conflict checker

    Re-reads every fact against its page when a new document arrives, and reports what now disagrees.

    Never: Never corrects a document, and never quietly changes an answer a person has relied on.

  7. A person decides

    Counsel and the chief credit officer

    Rule on the legal questions and approve the strategy, with the pages in front of them.

    Never: Never delegated: an agent flags, a person decides.

How a sentence reaches a reader in Special Assets File Review. The two person steps are where judgement stays; the removal step is the one a buyer should ask any supplier to demonstrate on their own file.

What should happen to an uncited sentence

It should not reach the reader. That is a stronger rule than flagging, and the difference is the whole argument: a flagged sentence is still in the memo, still readable, still quotable by someone who skims past the flag, and in practice it survives into the next draft. A removed sentence is a gap the writer has to fill honestly or leave empty.

The cost of the strict rule is real and worth stating. Answers get shorter. Questions the file cannot answer come back as "the file does not say", which reads as a failure and is in fact the most valuable output the system produces, because it is the thing an officer would otherwise discover in the meeting. The sample file's answer to the extension question ends by naming the open legal question rather than resolving it; that is the rule working, not the system falling short.

One answer, sentence by sentence, with the one that was removed

1

In the First Amendment of 9 April 2024 the bank promised to consider in good faith a further extension of up to 12 months, not to grant one.

First Amendment §2.4 · p.4

2

The promise was conditional on Cobalt Health renewing for at least five years by 30 June 2025.

First Amendment §2.4 · p.4

3

It was also conditional on fiscal year 2024 debt service coverage of at least 1.20 to 1.00.

First Amendment §2.4 · p.4

4

Cobalt Health renewed for three years, on 18 March 2025.

Lease renewal · p.1

5

Fiscal year 2024 debt service coverage was 1.04x.

FY2024 financials · p.6

6

The same amendment would have reduced one guaranty from 25% to 10% on that renewal.

First Amendment §4.2 · p.6

7

The amendment states that it supersedes all prior negotiations, correspondence and understandings.

First Amendment §9.4 · p.11

8

The bank therefore has no further obligation to the borrower under the 2024 email.

no passage supports this sentence

Fictional sample file 70-418226: the shape of the answer to 'what did we promise the borrower when we extended in 2024?'. The first seven are cited sentences of the kind the solution gives in that answer, each carrying its page; the eighth is the legal conclusion no passage supports, shown here struck through to make the rule visible. On the page itself a reader never sees it.

Note what the eighth sentence is. It is not false, it is not careless, and a human analyst might well write it. It is a conclusion about whether an earlier promise binds the bank, given an entire-agreement clause — which is a question of waiver and course of dealing, and therefore one the solution sends to workout counsel with its pages rather than answering. The rule that removes it is the same rule that keeps the other seven honest.

Why a closed file is the easier problem

Most writing about hallucinated citations concerns open-domain questions, where the model is asked about the world and the set of things it could cite is everything ever published. A troubled loan file is the opposite: 1,550 sources, every one of them in the file, every one already split into passages with a page and a section. Every citation must resolve to something that was collected, so the first failure mode — the source that does not exist — stops being a matter of vigilance and becomes a matter of construction.

That does not make the other three go away, and a buyer should be sceptical of anyone who implies it does. A real document can still be cited for a claim it does not make. A passage on page 6 can still be cited as page 4. What a closed file buys is that every one of those failures is checkable in principle, because the thing the sentence should have rested on is sitting there to be compared against. It is also why the honest unit of trust is the passage and not the document: on a fourteen-page amendment, "First Amendment" is not a citation, and "§2.4, p.4" is.

Two answers to the same question, and what each survives

Cites a documentCites a passage, and was read back against it
What the reader gets

Cites a document: "Per the First Amendment, the bank agreed to extend the loan."

Cites a passage, and was read back against it: "In the First Amendment of 9 April 2024 the bank promised to consider in good faith a further extension of up to 12 months, not to grant one." §2.4, p.4.

Opening the citation

Cites a document: Lands on a fourteen-page document. The reader hunts for the clause, or gives up and believes it.

Cites a passage, and was read back against it: Lands on the passage, highlighted, with the amendment’s other clauses around it.

If the document says something subtly different

Cites a document: Nobody finds out until the borrower’s counsel does.

Cites a passage, and was read back against it: The check fails before the answer is shown: "consider in good faith" is not "agreed to extend".

When a new document arrives

Cites a document: The answer stands unchanged, and is now wrong.

Cites a passage, and was read back against it: The fact is read back against its page again, and what disagrees is reported rather than quietly corrected.

Six months later, under scrutiny

Cites a document: An assertion with a document name attached.

Cites a passage, and was read back against it: A question, an answer, the passages behind each sentence, and a date — all still on the record.

The left column is the common pattern in document assistants that cite at file level; the right is what the solution does on the sample file. Both are described here as what a reader can check, not as a measure of accuracy.

Seven sentences, five events lit, every citation openable

A cited answer in Special Assets File Review for loan 70-418226. The question what did we promise the borrower when we extended in 2024 sits above an answer marked 1,550 sources, 64 passages, all checked, 8.6 seconds, with a Steps link. Each sentence of the answer carries a small numbered citation chip, and buttons below offer Open First Amendment, Handover and Counsel. To the right the relationship timeline across six lanes shows the five cited events highlighted and numbered while the other events are dimmed, including the First Amendment extension, the FY2024 coverage breach at 1.04x, the Cobalt renewal of three years, the Darrow email and the servicing letter.
The sample file in Special Assets File Review. The answer shows how many passages were read and that every sentence was checked against its page; the numbered citations open the passage itself rather than the document.

How to test it on your own file in an afternoon

Nothing above has to be taken on trust, and a buyer who accepts a demonstration on the supplier's own data is testing the demonstration. The test that matters uses a file whose answers you already know, because it is closed, and it takes an afternoon. The framework's language for this is ordinary and worth repeating at a steering committee: once a system is in use, its validity and reliability are typically judged by continuing testing or monitoring that shows it still performs as intended (NIST AI 100-1). Ongoing, and on your own material.

The afternoon test, on one of your own closed files

  1. Ask five questions whose answers you already know

    Pick ones where the obvious answer is wrong — a promise that was conditional, a waiver that was narrower than people remember. Score whether the answer names the condition, not whether it sounds right.

  2. Open every citation

    All of them, on at least one answer. Count how many land on the passage, how many land on the document, and how many land somewhere you have to search.

  3. Change one figure in one document and ask the question again

    The answer should move, and should name the changed page. If it does not move, the answer was not resting on the page it cited.

  4. Remove a document the answer relies on and repeat

    The system should say what it can no longer support. An answer that survives the removal of its source is not reading the source.

  5. Add a document that contradicts an earlier one

    Watch whether the earlier fact is re-checked and reported as a conflict, or silently overwritten. Reporting is the behaviour you want; a quiet correction destroys the audit trail.

  6. Ask a legal question and see what comes back

    Waiver, course of dealing, release scope. An answer is a finding against the product; a flag to counsel with the pages attached is the right behaviour.

  7. Ask something the file genuinely does not contain

    The right answer is that the file does not say, with what is missing named. Anything else means uncited sentences are reaching readers.

Our acceptance test for a cited document assistant, written so a credit or technology buyer can run it without the supplier present. Seed the file first, and keep the original.

What has to survive six months, and an examiner

A cited answer is a working document on the day it is produced and a piece of evidence afterwards. Six months later the questions are different: who asked this, what did the file hold when they asked, which passages did the answer rest on, and has anything arrived since that changes it. Those are questions about records, not about models, and they are answered by keeping the question, the answer, the passages and the date together rather than keeping a PDF of the result.

The supervisory expectation behind that is unglamorous. The interagency safety and soundness standards ask institutions to maintain internal controls and information systems that provide for "timely and accurate financial, operational and regulatory reports" (12 CFR part 30, appendix A). An answer whose basis cannot be reconstructed is not a timely and accurate report about anything. In the sample file's own settings, every question, answer, flag to counsel and signature goes to the audit trail, and the question log is kept for seven years — a setting, so each bank's own retention schedule governs it.

There is a supervisory consequence to thin documentation too, set out in the 2023 policy statement on commercial real estate workouts: when a bank cannot support its loan modifications with adequate analysis and documents, examiners form their own view of the classification until the bank produces the support for its grades. The bank that can show its work keeps the argument. The bank that cannot gets the examiner's own view in the meantime.

The rule that removes an uncited sentence, as a setting

The settings screen for the special assets team at Alder Point Bank. Under How answers are written, three switches are on: every sentence carries its page, with the note that a sentence the citation checker cannot support is removed; legal questions go to counsel, with waiver, course of dealing, oral promises and release scope flagged and never answered; and quote amounts, never recalculate, with figures shown as the document states them. A fourth, include other loans to the same guarantors, is off so answers stay inside this loan's file. Other panels list the four connected systems a file is read from, what is sent to counsel automatically, who signs the handover, and a question log kept for seven years.
The sample file in Special Assets File Review. The removal rule, the counsel rule and the quote-never-recalculate rule are settings a bank owns, not fixed behaviour, and each one is visible to the people relying on the answers.

What a cited history is for, once it holds, is the subject of what a document reading agent may decide on a loan file, and what it may not, and what it is worth is worked through in what reading a troubled loan file actually buys. To see an answer, its passages and the removal rule on one file, start from Special Assets File Review.

Questions people ask

Can an AI make up a page number?

Yes, and it is the failure mode that survives a careless check, because the document is real and the quotation may be nearly right. The structural answer is to cite the passage the sentence was drawn from rather than the document it came from, so the citation and the evidence are the same object. If a product cites page numbers as text rather than opening at the passage, assume the page number is unverified until you have opened several.

What is the difference between a citation and a grounded answer?

A citation is a reference attached to a sentence. A grounded answer is one whose sentences were drawn from passages and then read back against them, with the ones that fail removed. The second implies the first; the first does not imply the second. The question to ask a supplier is not whether answers are cited but what happens to a sentence whose citation does not support it.

Can an AI cite a real document for a claim it does not make?

Easily, and this is the most expensive of the four failures because it survives a spot check. The example on this page is exact: an amendment in which the bank agreed to consider an extension in good faith can be cited for the claim that the bank agreed to extend. Both sentences point at the same real clause; only one of them is what it says. Checking means comparing the sentence with the passage, not confirming the document exists.

Does a human have to check every citation?

Not every one, but the duty to check does not move to the tool. The court that sanctioned fake citations in a filing saw nothing improper in using a reliable tool, and held that the professional stays the gatekeeper for accuracy. The practical reading for a bank is to make checking cheap — citations that open at the passage, a rule that removes unsupported sentences — and then to check the sentences that carry the argument.

How do you audit an AI answer six months later?

By keeping the question, the answer, the passages each sentence rested on and the date together, so the answer can be reconstructed rather than merely reread. A PDF of the memo is not an audit trail; it has no way of showing what the file held at the time. In the sample file the question log is kept for seven years as a setting, and every flag to counsel and signature sits on the same trail.

What happens when a new document contradicts an answer that was right?

The right behaviour is to report, not to correct. A fact that was properly supported when it was written stays as it was, the new document is shown beside it, and the disagreement is marked for a person to resolve. Silent correction is worse than being wrong: it breaks the link between what someone was told and what they acted on, which is exactly what an auditor or an examiner will later want to see.

Special Assets File Review.
See it on your own files.

An agentic solution from Botminds. A troubled loan's whole history on one timeline, each answer opening the page it rests on. Book a demo, or explore how it works.