SystemRisk Management & ALM

Decide · buying on evidence

Audit findings tracking software: test it on last quarter first

Audit findings tracking software is easy to demonstrate and hard to judge, because every product can list findings. The honest test is a replay: run last quarter's closure requests through it and compare its verdicts with your validators', above all on the ones they returned. This guide sets the data, the pass lines and the decision, using Audit Issue & MRA Tracking as the worked case.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

Audit findings tracking software keeps internal audit findings, regulatory findings such as MRAs, and issues the business found itself in one register, each with an owner, an action plan, milestones, a due date and the evidence, and follows each to validated closure. Every product in the category can do that much, which is why a demonstration on vendor data tells a buyer little. The test that separates them is a replay on your own past: take last quarter's closure requests, run them through the tool, and compare what it says with what your validators decided, especially for the requests they sent back. Botminds' Audit Issue & MRA Tracking is built to be judged that way, because its agents do the part a validator would otherwise do by hand: they read each closure request against its evidence, sample the system of record and write one cited verdict line, while people keep every decision.

The worked examples use a fictional sample bank as it runs through the solution: Corvane National Bank, an OCC-supervised regional bank with 640 open issues and 86 closure requests in September 2026, 14 of them returned at first validation. The pass lines and the 30-day plan below are our suggestions, not regulatory requirements.

Five tests for audit findings tracking software, decided before the replay starts

The testHow it is measuredPass line

1Validated requests: does it agree?

How it is measuredOf the requests your validators validated, the share the tool reads as ready.

Pass lineSet before the pilot, for example 9 in 10. Each disagreement is read, not counted.

2Returned requests: does it catch them?

How it is measuredOf the requests your validators returned, the share the tool reads as not ready, for the same reason.

Pass lineThe test that matters most. A tool that misses returns is checking documents, not operation.

3Every statement: can you open its source?

How it is measuredA spot check of verdicts: each sentence opens the passage, sample or system entry behind it.

Pass lineAll of them. One uncited claim in a verdict is a fail.

4Samples: can you redraw them?

How it is measuredFor sampled controls, the population, sample size and seed are shown, and a redraw gives the same items.

Pass lineAll of them.

5Authority: did it stay inside its limits?

How it is measuredCount of validations, signatures, sends or date changes made by the tool rather than a person.

Pass lineZero. Not a threshold, a condition.

Our suggested test card for a replay pilot. The bank sets each pass line in writing before the pilot and does not move it afterwards; the fifth is a condition, not a score.

What is audit findings tracking software?

It is the system of record for the bank's open findings: what was found, by whom, who owns the fix, what the plan is, when it is due, what proves it, and whether internal audit has validated it. Searches for an audit issue tracker, internal audit tracking software and bank audit findings tracking software all describe the same need. The professional standard does not require a particular tool. The Institute of Internal Auditors' Standard 15.2 says: "Internal auditors may use a software program, spreadsheet, or system to track whether management's action plans are implemented according to the established timelines" (IIA, Global Internal Audit Standards, 2024). What the standard requires is the methodology the tool supports, set out in internal audit follow up procedures under Standard 15.2.

That is why the buying question is not "can it track findings?" but "what does it add to a well-kept spreadsheet?" For many tools the honest answer is workflow: reminders, routing, role permissions, reports. Those matter. But none of them tells a validator whether a fix operated, and that is where the work, and the risk, sits.

All open issues in Audit Issue & MRA Tracking, one register across every source

All open issues screen of the fictional sample bank: internal audit, self-identified and regulatory issues, from opening to validated closure. Filters: All 640, Internal audit 410, Self-identified 190, Regulatory 40, MRAs 12; any status; search by ID, title, owner or unit; any rating. Table columns issue, source, owner and unit, rating, due, status, validator. Rows include MRA 2025-10 flood insurance compliance monitoring, waiting for a signature; MRA 2025-07 IRR model assumption governance, asked for closure; MRA 2025-08 contingency funding plan testing and discount window readiness, due soon; MRA 2024-02 internal audit coverage of technology risk, asked for closure, validated by Helen Park; a row shown only as Restricted: BSA team only.
Fictional sample bank. The register is the easy part; a SAR-related issue appears to everyone else only as restricted, with its number, date and status.

How do you evaluate audit issue software?

Against your own closed work, with the pass lines written down first. A vendor demonstration shows the product at its best on data built for it; a replay shows how it behaves on the documents, minutes and system extracts your owners actually attach. Write down beforehand what would make you say yes and what would make you say no, so the result cannot be argued into a different shape afterwards.

Five properties are worth testing. Does its reading of past closure requests agree with your validators' decisions? Does it catch the requests your validators returned, and for the same reason? Can every sentence it writes be traced to a passage, sample or system entry? Can its samples be redrawn? And did it stay inside its authority: no validation, signature, report or date change made by software. The test card above turns those into measures.

Can you test issue software on past closures?

Yes, and it is the fastest honest test there is, because the right answers already exist. Your validators decided every closure request last quarter; some were validated, some returned with a reason. Replaying the same requests, with the evidence the owners attached at the time and read-only access to the same systems, gives a direct comparison: where the tool and the validator agree, where the tool is stricter, and where it would have let through something a validator caught.

Reading a replay: four outcomes and what each means

Tool saysValidator validatedValidator returned
ReadyAgreement. Spot-check the citations anyway.A missed return: the most serious outcome. Find out what the validator saw that the tool did not.
Not readyThe tool is stricter. Read its reason: sometimes it found a gap the validator missed, sometimes its rule is wrong.Agreement, if the reason matches. A different reason is half a catch.
Our reading guide. A 'needs a look' verdict, where the tool asks a person to decide, is counted with the cases it flags for the validator; below the solution's confidence threshold of 0.80 a person decides by design.

The returned requests are the heart of the replay. At the sample bank, 8 of September's 14 returns were for sustained operation not evidenced: the fix was in, but the months of proof were not. A tool that reads only the attached documents will call most of those ready, because the documents say the right things. A tool that samples the system of record and counts the months will not. That single difference decides most replays.

What a replay should show for a request the validator returned

Register screen for Thursday, October 22, 2026, viewing as Issue Validation Lead Renata Cole: 9 issues ask you for closure and 14 you validate fall due in the next 30 days. Each request shows the Evidence agent's read: MRA 2025-07 not ready, 2 of 3 months of operation evidenced; IA-2026-045 Call Report reconciliation sign-offs, ready, Q1 to Q3 reconciliations signed by preparer and reviewer; MRA 2025-06 privileged access reviews, ready; IA-2025-188 vendor SOC report reviews, not ready, 1 of 2 quarters of reviews on time; SI-2026-097 wire limit overrides, needs a look, 23 of 25 overrides had a second approver; IA-2026-077 Reg E provisional credit, not ready, 21 of 25 sampled claims credited within 10 business days.
Fictional sample bank. A useful verdict states what was counted, such as 21 of 25 sampled claims credited within 10 business days, so the validator can compare it with her own test line by line.

What data is needed for an issue management pilot?

Last quarter's closure requests as they were submitted, the decisions your validators made, and access to the systems the evidence comes from. Less than a migration and more than a spreadsheet export: the replay is only fair if the tool sees what the validator saw, no more and no less.

What to bring to a replay pilot of audit findings tracking software

  1. Every closure request from one recent quarter

    All of them, including the ones returned and later resubmitted, with the evidence attached at the time.

  2. The validators' decisions and reason codes

    Kept apart from the tool until the replay is done, so the comparison is blind.

  3. Your validation rules

    Months of sustained operation and sample sizes by rating, independence rules, the validation clock.

  4. Read-only access to the systems of record

    Or extracts as of the dates the validators tested: change logs, payments, core banking, model validation tracker.

  5. Minutes from the board portal

    ALCO, risk and audit committee minutes for the period, since many milestones are approvals.

  6. The register as it stood

    An export of open issues with owners, ratings, dates and date history.

  7. One recent exam letter and one audit report

    To test intake: does it split findings correctly and link to issues already open?

  8. Your roles

    Who owns, validates, signs and reports, so maker-checker can be set up as it would run.

Our checklist. Confidential supervisory information and SAR-related issues should follow the bank's own handling rules during a pilot exactly as in production.

How do you measure validation quality?

By agreement with independent judgment, by the returns caught, and by what happens after closure. The first two come out of the replay. The third takes longer: of the issues validated, how many reopened or came back as repeat findings? The OCC's audit booklet treats repeat issues as a signal worth reporting, and over a year they are the best measure a bank has of whether its validations held.

Measuring the people as well as the tool matters, because oversight that is never exercised is oversight in name only. The Financial Stability Board's 2026 sound practices on AI ask firms to build in feedback, including "tracking when humans override AI recommendations, analysing patterns in these interventions" (FSB, October 2026). In the solution every override of an agent's verdict carries a reason code from OV-01 to OV-03, so the bank can see whether its validators disagree with the agents rarely, often, or always about the same kind of control.

What is a good first-time validation rate?

High enough that returns are exceptions, and moving up. We know of no regulatory benchmark for the share of closure requests validated at the first attempt; the useful comparison is the bank's own rate over time. The sample bank validated 84% the first time in September 2026, 72 of 86, and its design target with the solution is 95%. The lever is not stricter or looser validation; it is the owner running the same test before asking, so the requests that reach validation are already complete.

A first-time rate can also be too high. If nearly everything is validated on first submission while repeat findings rise, the validation is not testing operation. Report the rate beside its reasons: at the sample bank, 8 of 14 returns for sustained operation tells the audit committee where owners need help.

How long does an issue management pilot take?

About a month for a replay on one quarter, if the data is ready. The plan below is our suggestion, and most of it is the bank's own time: assembling the requests, setting the rules, and reading the disagreements. Configuration of the solution for a replay uses the bank's own validation rules, reason codes and roles.

A 30-day replay pilot: most of the month is the bank reading results

Suggested plan30 days
Assemble last quarter5 days
Set rules and roles5 days
Replay, blind7 days
Read every disagreement9 days
Decide against the card4 days
  • Assemble last quarter5 days
  • Set rules and roles5 days
  • Replay, blind7 days
  • Read every disagreement9 days
  • Decide against the card4 days
Our suggested plan, in calendar days, not a commitment by any party. The replay is blind: validators' decisions are compared only after the tool's verdicts are in.

Keep the scope narrow. One quarter, all its closure requests, and the systems those requests depend on. A pilot that also tries to migrate the register, redesign the reports and onboard every owner will end with an opinion rather than a result. Intake of exam letters can be tested in the same month on one recent letter, as below.

A second test for the same month: one exam letter read into proposed issues

Intake screen of the fictional sample bank. OCC supervisory letter CRE-2026-3 of October 16, 2026, commercial real estate concentration review, confidential supervisory information. Highlighted passages: a Matter Requiring Attention, the bank's CRE stress testing does not capture refinance risk for office loans maturing in 2027 and 2028, $1.9 billion of commitments; an observation, management may consider showing maturities by property type in board CRE reports; and appraisal review turnaround of 41 days, which management had already identified. The Intake agent proposes one MRA with owner, root cause, rating High and due date June 30, 2027, at 0.94; one non-binding observation at 0.88; and a link to an issue already open, no duplicate, at 0.91.
Fictional sample bank. Each proposed issue cites the words in the letter that decide its label. The Issue Management Office registers it and the Chief Risk Officer confirms a new MRA; nothing is registered by the agent.

Pass and fail: written before the pilot, judged after

A pilot passes when every condition holds and every score clears the line the bank wrote down in advance. It fails on any missed condition, an action outside authority, an uncited claim, a sample that cannot be redrawn, whatever the scores. Between those, the decision is about the disagreements: if the tool's misses are explainable and its catches are real, the bank has learned what it would get. If it missed returns for reasons it cannot explain, the bank has learned that too, for a month's work instead of a year's.

What the pilot proves, and what only go-live proves

Replay pilotAfter go-live
Question

Replay pilot: Would the tool have read last quarter's requests the way our validators did?

After go-live: Do owners ask for closure only when the proof is there, and do validations hold?

Measure

Replay pilot: Agreement, returns caught, citations, redraws, authority.

After go-live: First-time validation rate, repeat findings, slips flagged before the date.

Who does the work

Replay pilot: Validators reading disagreements.

After go-live: Owners running the pre-check; validators deciding; the Chief Risk Officer signing.

What it cannot show

Replay pilot: Behaviour change: owners were not using it.

After go-live: A clean comparison: the requests themselves have improved.

Our comparison. Both halves matter: the replay tests the reading; go-live tests whether better reading changes what owners submit.

The supervisory stakes argue for doing it properly. The Federal Reserve's June 2026 report says that for large firms "supervisors have avoided duplicating validation work when a firm's internal audit function has been assessed to be effective and has validated remediation efforts" (Supervision and Regulation Report, June 2026). Whatever tool a bank chooses, its validations are increasingly what examiners rely on; for the OCC and FDIC, the new MRA rule takes effect November 2, 2026, and the OCC and FDIC MRA final rule guide covers what it changes.

Where Audit Issue & MRA Tracking fits after a replay, and where it does not

It fits a bank whose closure evidence lives in systems an agent can read, where validators spend their days re-performing tests by hand, and where returns for missing proof are common. It works with the bank's existing issue management system two-way and with its document management system, board portal and systems of record read-only. It does not replace the validators' judgment, write the bank's validation rules, or decide closures; and at a bank with a few dozen findings and a validator who knows each one, a well-kept register may serve as well. For the three broad kinds of product on the market and what each catches, see issue management software compared; for what the file behind a validated closure should hold, the MRA closure report guide. To run the replay on one of your own closure requests, start from Audit Issue & MRA Tracking.

Questions people ask

What is the difference between audit management and audit findings tracking software?

Audit management software runs the audit itself: planning, workpapers, testing and reports. Findings tracking picks up where the report ends, following each finding through the owner's action plan to internal audit's validation and closure. Some products do both; a bank evaluating either should test the part that carries its risk.

Can audit findings tracking software be tested without production data?

Only partly. Synthetic data can test workflow, permissions and reports, but not whether the tool reads your owners' real evidence the way your validators do. A replay on last quarter's actual closure requests, under the bank's own data-handling rules, is the test that answers the buying question.

How many closure requests does a replay need?

Enough to include returns. The sample bank's September had 86 requests and 14 returns; if every month looked like that, a quarter would give about 40 returns to test against. At a smaller bank, take two quarters, or every request since a recent exam.

Should validators see the tool's verdicts during the pilot?

Not before the comparison. A replay is blind: the validators' original decisions are fixed, and the tool's verdicts are compared afterwards. In production the opposite holds: validators see the verdict and the evidence together, and decide.

Does the IIA require software for tracking audit findings?

No. Standard 15.2 allows a software program, spreadsheet or system, and lists as evidence of conformance a routinely updated tracking system holding the finding, the action plan, its status and internal audit's confirmation. The choice is the bank's; the methodology is not optional.

Audit Issue & MRA Tracking.
See it on your own files.

An agentic solution from Botminds. Each issue a closure dossier: what was promised beside what proves it, tested by agents Book a demo, or explore how it works.