Can AI reconcile a bank statement?
It depends on the model. On 16 November 2026, 11 of the twelve current AI models I tested got this right three times out of three, and 1 of 36 answers had at least one error. The cheapest model that never failed was Claude Haiku 5.5, at $0.06 per 100 runs.
- Tested
- 16 November 2026
- Result
- 11 of 12 models, every time
- Answers
- 35 of 36 passed every check
- The task
- A bookkeeper’s, in bookkeeping
- Asked by
- Me, to open the record
The result, model by model
Each model was given the same message three times, and each square is one of its answers: in ink when it passed every check, in red when it failed at least one. The cost is what one run was billed through the API, and the time is how long the full answer took to arrive.
| Model | The three runs | Right | Cost of a run |
|---|---|---|---|
| Claude Haiku 5.5Anthropic | 3 of 3 | $0.0006 | |
| DeepSeek V4.1 FlashDeepSeek | 3 of 3 | $0.002 | |
| GPT-6.1 TerraOpenAI | 3 of 3 | $0.002 | |
| GPT-6.1 SolOpenAI | 3 of 3 | $0.003 | |
| Gemini 4 ArgonGoogle | 3 of 3 | $0.004 | |
| Qwen4 MaxAlibaba | 3 of 3 | $0.005 | |
| Claude Sonnet 5.5Anthropic | 3 of 3 | $0.008 | |
| Gemini 4 HydrogenGoogle | 3 of 3 | $0.012 | |
| Mistral Large 4Mistral | 3 of 3 | $0.013 | |
| Grok 4.7SpaceXAI | 3 of 3 | $0.014 | |
| Claude Fable 5.2Anthropic | 3 of 3 | $0.033 | |
| GPT-6 LunaOpenAI | 2 of 3 | $0.0003 |
Where the answers went wrong
What a right answer has to do, most often missed first. These are the places where a person still has to look: one of the five checks caught at least one wrong answer.
- Spots the cheque paid in on the 30th and the cheque written on the 31st, neither at the bank yetwrong in 1 of 36
- Finds the bank charges and the interest that were never entered in the cash booknever wrong
- Notices the direct debit the bank took twice on the same daynever wrong
- Catches the cheque entered as 384.60 and paid as 348.60never wrong
- Matches the one credit of 990.00 to the two invoices it paysnever wrong
A wrong answer, word for word
From GPT-6 Luna, the most expensive model that failed this task. It reads as finished work, which is the difficulty: nothing in it says which line to doubt.
Spots the cheque paid in on the 30th and the cheque written on the 31st, neither at the bank yet
- It answered
- ["C12"]
- Right
- ["C11","C12"]
{
"cash_book_not_on_statement": ["C12"],
"statement_not_in_cash_book": ["B08", "B11"],
"taken_twice": ["B09", "B10"],
"amount_differs": { "cash_book": "C03", "statement": "B04", "difference": 36.00 },
"paid_in_together": { "statement": "B03", "cash_book": ["C04", "C05"] }
}The task as the models saw it
A print shop closes October with a cash book and a bank statement that disagree by 951.38 pounds. The difference is made of six ordinary things, one of them the bank taking a direct debit twice. Every name and figure is invented. The models received the two texts below as one instruction and one message, and nothing else.
The documents
SABLE & FINCH PRINT LTD: CASH BOOK, OCTOBER 2026 Balance brought forward 1 October: 4,210.35 C01 02 Oct Receipt Harris & Co, invoice 1041 1,250.00 C02 03 Oct Payment Rent, standing order 950.00 C03 06 Oct Payment Cheque 300417, Baxter Supplies 384.60 C04 08 Oct Receipt Okafor Ltd, invoice 1043 672.00 C05 08 Oct Receipt Okafor Ltd, invoice 1044 318.00 C06 13 Oct Payment Card, fuel 64.18 C07 15 Oct Payment Wages, BACS 3,120.00 C08 20 Oct Receipt Lindqvist AB, invoice 1046 1,485.90 C09 22 Oct Payment Cheque 300418, Moor Print Finishing 217.45 C10 27 Oct Payment Electricity, direct debit 186.72 C11 30 Oct Receipt Harris & Co, invoice 1049, cheque paid in 940.00 C12 31 Oct Payment Cheque 300419, T. Reilly 150.00 Balance carried forward 31 October: 3,803.30 WESTMERE BANK: STATEMENT, BUSINESS CURRENT ACCOUNT, 1 TO 31 OCTOBER 2026 Opening balance: 4,210.35 B01 02 Oct Credit HARRIS & CO 1,250.00 B02 03 Oct Debit STANDING ORDER, KELSO ESTATES 950.00 B03 08 Oct Credit OKAFOR LTD 990.00 B04 09 Oct Debit CHEQUE 300417 348.60 B05 13 Oct Debit CARD, NORTHWAY FUELS 64.18 B06 15 Oct Debit BACS, WAGES 3,120.00 B07 20 Oct Credit LINDQVIST AB 1,485.90 B08 24 Oct Debit ACCOUNT CHARGES 12.50 B09 27 Oct Debit DIRECT DEBIT, BRIGHTLINE ENERGY 186.72 B10 27 Oct Debit DIRECT DEBIT, BRIGHTLINE ENERGY 186.72 B11 29 Oct Credit INTEREST 1.84 B12 30 Oct Debit CHEQUE 300418 217.45 Closing balance: 2,851.92
The instruction
You are the bookkeeper of Sable & Finch Print Ltd. Reconcile the October cash book with the October bank statement below. The cash book lines are numbered C01 to C12 and the statement lines B01 to B12.
Return only one JSON object, with no text before or after it and no Markdown, in exactly this shape:
{
"cash_book_not_on_statement": ["C00"],
"statement_not_in_cash_book": ["B00"],
"taken_twice": ["B00", "B00"],
"amount_differs": { "cash_book": "C00", "statement": "B00", "difference": 0.00 },
"paid_in_together": { "statement": "B00", "cash_book": ["C00", "C00"] }
}
"cash_book_not_on_statement": cash book lines that had not reached the bank by 31 October.
"statement_not_in_cash_book": statement lines for something the cash book has no entry for at all. Do not put the two lines of "taken_twice" here.
"taken_twice": the two statement lines where the bank took the same payment twice.
"amount_differs": the one payment entered in the cash book at a different amount from the one the bank paid, and the difference in pounds.
"paid_in_together": the one statement line that is two cash book receipts paid in as a single amount.Every answer of the run is in one file, under CC BY 4.0. How a test is run and what it does not show.
The other tests
Have a task of your own? Describe it and I will test it the same way. Send a task
- Can AI check an expense claim against the company policy?All twelve models, every time
- Can AI compare supplier quotes and pick the cheapest?All twelve models, every time
- Can AI find the cancellation deadline in a contract?All twelve models, every time
- Can AI shortlist job applicants against the requirements?All twelve models, every time
- Can AI write a staff rota that follows the rules?10 of 12 models, every time