AI procedure writing is useful to a bank when the tool changes the right sentences in the procedures you already have, and nothing else, and the only way to know is to test it on documents where you already know the answer. Fluent prose proves nothing: a model can write a convincing hold procedure that quotes last year's threshold. The test this guide proposes is simple. Take copies of your own procedures, plant ten known errors drawn from a real rule change, run the tool, and count what it found, what it missed and what it changed that it should have left alone. Botminds' Policy & Procedure Management is an agentic solution built for this job: agents find every document a rule change touches, draft each redline in the document's own words with the rule passage beside it, and compare every number in the owner's approved version with the rule before Compliance signs.
The seeds come from the Regulation CC amounts that rose on July 1, 2025, a change every bank and credit union branch network had to make. The run described alongside them is the solution on a fictional sample credit union with 1,240 procedures and job aids.
What a real rule change hides in a real library
5
Regulation CC amounts changed on July 1, 2025, and nothing else in the rule
7of 17
affected sample documents quote an amount without ever naming the rule
3
plausible matches that must not be changed: two look-alike amounts and one policy
$6,275
a human typo for $6,725, made after the drafting was right
Can AI write policies and procedures?
Yes, in the sense that a large language model will produce a readable procedure on request. But a bank seldom needs a new procedure from a blank page. It needs the procedures it already has changed correctly when a rule moves: the right numbers in the right sentences, in the owner's words, by the effective date, with every other sentence left exactly as it was. That is a narrower task, and it fails in ways a demonstration does not show.
The first failure is the model's own knowledge. Researchers who traced the knowledge in open language models found that the cutoff a model's maker reports is often not the date its knowledge actually reflects: "effective cutoffs often differ from reported cutoffs", in part because of "temporal biases of CommonCrawl data due to non-trivial amounts of old data in new dumps" (Cheng et al., Dated Data, 2024). The old Regulation CC amounts were in force for five years, from July 2020 to June 2025, so the web a model learns from is full of them. A tool that drafts from what it remembers rather than from the rule text in front of it will reproduce $5,525.
The second failure is that no supervisory test exists for this kind of tool. The revised interagency guidance on model risk management that replaced SR 11-7 in April 2026 states that generative and agentic AI models are outside its scope (Federal Reserve SR 26-2). Whatever a bank decides about governing such tools, the evidence that a drafting tool works on its procedures has to come from the bank's own testing. The seeded test is one way to produce it.
The seeded test: ten errors planted in copies of your procedures
A seeded test plants errors whose right answer you know, runs the tool, and scores each one as found, missed or wrongly changed. Its value is that it measures the failures that matter in a procedure library, not the ones that are easy to see. The scorecard below is the one we suggest; the last column shows which seeds the sample run exercised and which a buyer has to plant.
The seeded-error scorecard: what each seed tests, and what a pass looks like
| Seed | Planted in | A pass | A fail | In the sample run |
|---|---|---|---|---|
| 1. Cited amount: $225 next-day | A hold procedure that cites 12 CFR 229.10(c)(1)(vii) | Changed to $275, citation kept | Missed, or changed with a new citation that is wrong | Found: 7 documents cite the rule |
| 2. Uncited amount: $450 cash withdrawal | A teller script that never names Regulation CC | Found and changed to $550 | Missed because the rule is not named | Found: 7 documents quote an amount without the rule |
| 3. Uncited threshold: $5,525 large deposit | A contact center script for hold questions | Found and changed to $6,725 | Missed | Found, in the same middle ring |
| 4. Format variant: "$5525" or "5,525.00" | A form or a table | Found despite the format | Missed because the search expects "$5,525" | Not shown: plant it |
| 5. Dependent document with no number | A quick card that says "see the large deposit procedure" | Listed for the owner to review; no text invented | Ignored, or given an invented amount | Found: 3 dependent documents |
| 6. Look-alike: $450 daily ATM limit | A card procedure; the limit is the institution's own | Left unchanged, with a reason | Changed to $550 | Left out with the reason |
| 7. Look-alike: $5,525.18 payoff example | A loan payoff procedure | Left unchanged, with a reason | Changed to $6,725.18 | Left out with the reason |
| 8. Untouched sentence: the 5:00 p.m. cut-off | The same paragraph as seed 2 | Word for word as before | Reworded, shortened or "improved" | Only the numbers changed; wording kept |
| 9. Withdrawn authority | An overdraft procedure citing CFPB Circular 2022-06 | Citation flagged for the owner; no rule number changed | Passed over, or the citation silently deleted | Not shown: plant it |
| 10. Human typo after approval: $6,275 | An owner's edited version | Set aside before sign-off, with the rule passage | Approved with the rest | Caught; returned with a reason code |
Seeds 6, 7 and 8 are the ones demonstrations skip. A tool that changes every $450 it sees will score perfectly on seed 2 and wrongly on seed 6, and the wrong change is worse than the miss: it rewrites an ATM limit the institution chose for itself into a number that has nothing to do with it. Seed 8 tests restraint. A model asked to update a paragraph tends to tidy it, and a tidier paragraph with a changed cut-off time is a second, unapproved change that nobody asked for.
Seed 9 tests something no rule-change feed announces. On May 12, 2025 the CFPB withdrew 67 guidance documents in one notice, among them Circular 2022-06 on unanticipated overdraft fee assessment practices and Circular 2023-02 on reopening deposit accounts that consumers previously closed (90 FR 20084). The underlying rules did not change, so no number moves. A procedure that cites the circular as its authority still needs its owner to decide what now stands behind it.
How to plant the seeds in your own procedures
Use copies of real documents, not samples written for the test, because the test is of the tool on your library: your formats, your cross-references, your habits of quoting numbers. Keep the answer key sealed until the run is scored.
Planting a seeded test in an afternoon
- Pick 30 to 50 documents across owners
Procedures, quick cards, call scripts and at least one form, from deposits, branches and the contact center.
- Restore the old Regulation CC amounts in copies
Put $225, $450 and $5,525 back where your current versions say $275, $550 and $6,725, in the formats your writers use.
- Remove the rule's name from some of them
A quick card that quotes the amount but never says "Regulation CC" is the case that separates tools.
- Add the look-alikes
Your own ATM limit, a worked payoff example, a fee that happens to share a digit pattern.
- Add one dependent document and one withdrawn citation
A quick card that points to a procedure; a procedure that cites a withdrawn CFPB circular.
- Type one wrong number into an "approved" version
Transpose two digits of a new amount, as a tired owner would, after the drafting step.
- Seal the answer key, run the tool twice
A tool whose answers change between identical runs needs a deterministic check behind it.
- Score every seed and every unplanned change
An unplanned edit anywhere in the set counts against the tool, even if every seed passed.
How accurate is AI at writing compliance procedures?
No public study measures AI drafting of bank procedures, so accuracy has to be measured on your own documents, and it has three parts that should never be averaged into one score. Found: of the seeds that should change, how many did. Restraint: of the seeds that must not change, look-alikes and untouched sentences, how many stayed exactly as they were. Caught: of the errors introduced after drafting, how many were stopped before approval.
Scoring the run: three numbers, each with its own pass line
| Measure | How to count it | Pass line we suggest |
|---|---|---|
| Found | Seeds correctly changed ÷ seeds that should change (1–5, plus 9 flagged) | All of them; one miss is a document staff will read wrong on the effective date |
| Restraint | Seeds left exactly as they were ÷ seeds that must not change (6–8) | All of them; any wrong change is worse than a miss |
| Unplanned edits | Changes anywhere in the set that no seed called for | Zero |
| Caught | Post-approval errors set aside before sign-off (seed 10) | Caught, with the rule passage shown |
| Explained | Exclusions and flags that carry a reason a reviewer can check | Every one |
The strict pass lines are the point. A tool that finds nine of ten affected documents still leaves one teller card quoting the old amount at every branch that uses it. A test with a lower bar measures the tool's prose, not its fitness for a library that examiners read.
In the sample run, one candidate in seven was a trap
21 candidates the three searches turned up
- Changed: 17 procedures, quick cards and scripts · 81%
- Member disclosure, redrafted as a notice · 5%
- Set aside: 2 look-alike amounts, 1 policy with no amount · 14%
Job aids and scripts that quote the number but never name the rule
Does AI know the latest Regulation CC thresholds?
Sometimes, and that is the problem. What a model answers depends on what it saw in training and on how the question is put, and a confident $5,525 reads exactly like a confident $6,725. The safe design never asks the model what the threshold is. It gives the drafting step the rule's own table of amounts and the section each comes from, and it checks every number in the result against that table after the last human edit.
That is how the drafting works in the solution. Each redline carries the rule section behind it and why the agents propose it; the owner sees, for example, that $225 is the minimum next-day amount in 12 CFR 229.10(c)(1)(vii) and becomes $275 on July 1, 2025. Below 0.80 confidence the Impact agent asks a person instead of proposing. The whole change traced from the Federal Register to the teller line shows the same redline in its sequence.
The rule's own table of amounts is the source every redline cites
What should a human check in an AI-written procedure?
Two people check different things. The owner reads the changed sentences for meaning: whether the redline says what the counter actually does, whether a sentence the agents left alone should change too, and whether a set-aside look-alike really is one. Compliance checks that every number and citation in the owner's final version matches the rule, after the owner's edits, because the owner's edits are where new errors appear.
The sample shows why the order matters. The drafting was right; the owner of the hold-question procedure edited by hand and typed $6,275 for the large deposit threshold, which the rule sets at $6,725 (12 CFR 229.13(b)). The Rule check agent compares on exact match only and never approves, so it set the document aside with the rule passage. Compliance approved the 16 that matched in one action and returned the 17th with the reason code "Number does not match the rule".
Every number in the owner's version, checked against the rule
Who is responsible for an AI-drafted procedure?
The institution, through the owner who approves the text and the Compliance officer who checks it. Using a tool does not move the responsibility. The FDIC's compliance manual says it plainly for any outside provider: "ultimately the institution's Board and management are responsible for identifying and controlling compliance risks arising from third-party relationships, to the same extent as if the third-party activity was handled within the institution" (FDIC Consumer Compliance Examination Manual, II-3.1).
That is why the design matters more than the model. In the solution the agents find, draft and check; they never approve, publish or close a rule change, and every proposal can be overridden with a reason code that stays on the record. Each agent runs at a pinned version with its confidence threshold, approved by the Compliance Officer, so an examiner can see which version drafted which redline. A procedure that a person approved, a second person checked and a log can reconstruct is a procedure someone is responsible for.
Where Policy & Procedure Management fits, and where it does not
It fits the work the seeded test measures: changing existing procedures, job aids, scripts and member notices when a rule changes, finding the documents that quote a number without naming the rule, and checking every number before sign-off. Its design target is that every document quoting a changed rule's numbers is found; the seeded test is how a buyer checks that target on their own library before relying on it.
It does not write a new policy for a new product from nothing, decide whether a rule applies (Compliance confirms every applicability, and only Compliance can close a change as not applicable), or judge whether a procedure describes what staff really do; that stays with the owner. Seeds 4 and 9 were not part of the sample run, so plant them. Twelve questions to ask any policy and procedure management software covers the rest of an evaluation, and Policy & Procedure Management can be run on a sample of your own procedures against a rule change you have already handled.
Questions people ask
Can ChatGPT write bank procedures?
It can draft one, but a general chatbot drafts from what it learned in training, which may be out of date, and it does not know your library, your look-alike amounts or which version is in force. For the 2025 Regulation CC change the old amounts had been in force for five years, long enough to fill years of published disclosures and guides a model may have learned from. Any draft needs the rule text supplied and every number checked against it.
What is the best AI for procedure writing?
The one that passes a seeded test on your own documents. Look for three behaviours: it finds amounts that are quoted without the rule's name, it leaves look-alikes and untouched sentences exactly as they were, and it checks the final human-edited version against the rule. A tool that only produces fluent text cannot show any of them.
How do you test an AI policy writer?
Plant known errors in copies of real procedures, seal the answer key and score each seed as found, missed or wrongly changed. Include look-alike amounts, a sentence that must not change and a typo introduced after approval. Count every unplanned edit against the tool, and run the same set twice to see whether the answers change.
How many documents does a seeded test need?
Thirty to fifty copies across owners is enough to hold ten seeds and the noise around them, including documents that should come back untouched. The sample library had 1,240 documents, and one rule change reached 17 of them; a test set should keep that proportion of untouched documents so restraint is measured, not assumed.
Should AI-drafted procedures be disclosed to examiners?
We know of no federal rule that requires a procedure to be labelled as AI-drafted, and the 2026 interagency model risk guidance leaves generative AI outside its scope. What an examiner can always ask is who approved and checked a version and on what basis, so keep the drafting agent's version, the owner's approval and the Compliance check on the record of every change.