AI tax preparation tools: what the research actually measured

AI tax preparation tools are better than their critics say and worse than their advertising, and the peer-reviewed research shows why. A 2024 study of successive OpenAI models applying U.S. tax law found performance improving with every release and highest when the model was handed the relevant legal text and worked examples, reaching "high levels of accuracy but not yet at expert tax lawyer levels". A 2023 study found an earlier model answering simple questions about unfamiliar statutes poorly. Studies of legal questions found public chatbots inventing answers at least 58% of the time and commercial research tools built on retrieval doing so 17% to 33% of the time. None of them tested what a fund tax team does: computing and tying out hundreds of partners' K-1s from books, partnership agreements and foreign documents. The evidence therefore argues for a particular design, AI that works from the fund's own documents, cites every figure and stops when unsure, and for testing any tool on your own filed year before trusting it. That is the design of Botminds' K-1 Preparation.

Each study below is cited to its published version; one is an arXiv preprint and is labelled as not peer reviewed. The product examples are fictional sample data.

What have studies of AI and tax actually measured?

Mostly question answering. The studies with the most rigour set models questions with known answers, about tax provisions, statutes, court cases or accounting problems, and count how often the answer is right. That is a fair test of whether a model knows or can apply a rule. It is not a test of preparation, which is computation over a fund's own records, followed by a tie-out and a signature.

The distinction matters to a buyer because the failure modes differ. A model that answers a tax question wrongly produces a wrong sentence a professional can catch. A tool that prepares 214 K-1s wrongly produces 214 plausible documents, each priced at $340 if it reaches an investor with incorrect information. The research tells you how much to trust a model's reading of a rule; only a test on your own data tells you how much to trust a tool's output.

AI on tax law: improving, and best when it is given the right text

The most direct evidence is Nay and colleagues' study in the Royal Society's Philosophical Transactions. The authors chose tax law because it "allows us to set up automated validation pipelines across thousands of examples, requires logical reasoning and maths skills". Their experiments "demonstrate emerging legal understanding capabilities, with improved performance in each subsequent OpenAI model release", and giving the model the relevant legal authority and examples of question-answer pairs improved the most advanced model further (Nay and others, 2024).

The conclusion is the sentence vendors quote half of: models "particularly when combined with prompting enhancements and the correct legal texts, can perform at high levels of accuracy but not yet at expert tax lawyer levels". For K-1 work, the useful half is the condition. A model reasoning from memory is weaker than one working from the actual text, which for a fund means the actual LPA clause, the actual instruction and the actual withholding certificate.

Statutory reasoning: where models broke

A study presented at the 2023 International Conference on Artificial Intelligence and Law tested GPT-3 on SARA, a statutory-reasoning dataset built from U.S. tax law (described in arXiv 2005.05257, not peer reviewed). The model beat earlier published results, but the authors found it had an imperfect knowledge of the actual statutes behind the dataset, and when they wrote simple synthetic statutes it could not have seen in training, it answered straightforward questions about them poorly (Blair-Stanek, Holzenberger and Van Durme, ICAIL 2023).

The same authors later reported, in a preprint that has not been peer reviewed, that the best public models "currently perform poorly on basic legal text handling", such as "looking up the text at a line of a witness deposition or at a subsection of a contract", while fine-tuning on their training set "brings even a small model to near-perfect performance" (arXiv 2311.09693, not peer reviewed). An LPA is a long contract of numbered subsections. A tool that cannot reliably find §8.4 cannot be trusted to apply it.

How often models invent answers when they can be checked

Two studies from the same research group measured hallucination, answers inconsistent with the law, on questions whose answers can be verified. Neither is about tax, and both are about the behaviour that matters most in a preparation tool: confident output with no source behind it.

Dahl and colleagues add a finding that matters more than the rate: the models were poor at telling when they were wrong, and tended to accept a user's mistaken legal premise without challenge (Dahl and others, 2024). Magesh and colleagues found retrieval reduced hallucination relative to a general chatbot but did not remove it, and read their results as a reason for professionals to supervise and verify what such tools produce (Magesh and others, 2025). A preparation tool, in other words, needs to be checkable by design, and it needs a rule for when to stop.

And on accounting questions?

The largest test of a chatbot on accounting material, by more than 300 co-authors across 186 institutions, put ChatGPT as of January 2023 through 28,085 assessment and test-bank questions. It answered 56.5% correctly and a further 9.4% partly correctly, and it beat the student average on 15.8% of assessments (Wood and others, Issues in Accounting Education, 2023). That model is now several generations old; the study's lasting value is its method, scoring a tool on a large set of questions with known answers, which is exactly how a fund should score a tool on its own K-1s.

Will AI replace tax preparers?

The research does not support it, and the rules do not contemplate it. Treasury's Circular 230 requires a practitioner to exercise due diligence in preparing, approving and filing returns, and when relying on someone else's work presumes diligence only where the practitioner took reasonable care in engaging, supervising, training and evaluating that person (Circular 230, §10.22). The rule speaks of people, but it describes the right relationship with any tool: supervised, evaluated, and never the one who signs. Regulators of investment firms are moving the same way; the SEC's examination priorities for fiscal 2027 speak of "super intelligence (SI)" and put firms' policies for monitoring its use, back-office work included, on the examination list (SEC Division of Examinations, October 23, 2026).

What changes is the preparer's day. Reading a withholding certificate in Portuguese, rebuilding an allocation from a waterfall, re-keying lower-tier K-1 footnotes and tying 214 K-1s to Schedule K are tasks with checkable outputs, and they are where AI earns its place. Choosing a method when the LPA and the administrator disagree, deciding how much of an over-withheld tax is creditable, reviewing and signing are not. The split, task by task, is in K-1 preparation software: what agents take and what preparers keep.

Can AI prepare a partnership return?

It can do much of the preparation, if it is built as a set of tools that work on the fund's records rather than as a chat window. A partnership return is a computation: allocations under an agreement, tax capital rolled forward, K-1s that must sum to Schedule K, K-3s that must sum to K-2, withholding that must match payments made. A chatbot asked about the rules can be right or wrong; a preparation system has to be right every time and able to show why.

How to test AI tax preparation tools on your own fund

Use the method the studies used, on your data. Take last year's closed books, partnership agreements, partner file and incoming documents for a handful of partnerships with your hardest cases, run the tool, and compare its K-1s with what you filed, box by box. Then probe the behaviours the research warns about.

The first test is the one that matters most and is easiest to skip. It is also how a fund finds errors in last year's filed K-1s while they can still be dealt with, which for a partnership under the centralized audit regime means an administrative adjustment request rather than a corrected K-1; see administrative adjustment request vs amended return.

Where K-1 Preparation fits, and where it stops

K-1 Preparation is built on the conditions under which the research says AI does well: the exact text in front of it, a narrow task, an independent check and a person at the end. Seven agents each have one job and written limits. The K-3 agent reads portfolio documents in their own language beside an English translation and cites every figure to its page; in the sample season it read the board minutes and the withholding certificate in Portuguese for a Brazilian payment, and stopped on a Norwegian dividend at 89% confidence because its threshold is 90%. The Tie-out agent recomputes every K-1 without using the Allocation agent's numbers.

A foreign document read in its own language, every figure on its page

Where it stops is equally deliberate. The agents never release, send, change partner data, change an allocation method or report tax above a treaty rate as creditable, and below their confidence threshold they ask. The tax senior prepares, the tax manager reviews, the tax partner releases, and internal audit can read every step. It does not give tax advice, and it does not replace the tax software that files the return.

The question the agent would not answer alone

The solution, with every screen and agent, is at K-1 Preparation.

Questions people ask

What is the best AI tax preparation software for a fund?

The one that passes a replay of your own filed year with differences you can explain, and that cites its sources, stops when unsure and cannot release on its own. Published benchmarks test question answering on general tax material; none has tested a fund's K-1s.

Can ChatGPT prepare a K-1?

A general chatbot can explain what a K-1 box means; it is not built to compute 200 partners' allocations from a fund's books and tie them to Schedule K. Studies of public chatbots on checkable legal questions found hallucination at least 58% of the time, and models struggling to tell when they were wrong.

Do AI legal and tax research tools still hallucinate?

Less than general chatbots, but yes. A preregistered study published in 2025 found commercial AI legal research tools built on retrieval hallucinating between 17% and 33% of the time. Any tool used in preparation should make its sources checkable.

Is AI better at tax law than it used to be?

Yes. A 2024 peer-reviewed study found performance improving with each successive OpenAI model and rising further when the model was given the relevant legal text and examples, but still short of expert tax lawyers. Results for one model generation do not carry over automatically to the next.

Who is responsible for a return prepared with AI?

The same people as before: the partnership for furnishing correct K-1s, the partner who signs Form 1065, and any paid preparer for due diligence under Circular 230. A tool changes who does the mechanical work, not who answers for the result.