Guglielmo Paesano AI engineer and founder of AIGROW

I build AI systems that do people's work, and I measure where they still need people.

contact@guglielmo.ioDiscord, 4K members

Can AI reconcile a bank statement?

It depends on the model. On 16 November 2026, 11 of the twelve current AI models I tested got this right three times out of three, and 1 of 36 answers had at least one error. The cheapest model that never failed was Claude Haiku 5.5, at $0.06 per 100 runs.

Tested
16 November 2026
Result
11 of 12 models, every time
Answers
35 of 36 passed every check
The task
A bookkeeper’s, in bookkeeping
Asked by
Me, to open the record

The result, model by model

Each model was given the same message three times, and each square is one of its answers: in ink when it passed every check, in red when it failed at least one. The cost is what one run was billed through the API, and the time is how long the full answer took to arrive.

ModelThe three runsRightCost of a runTime
Claude Haiku 5.5Anthropic3 of 3$0.00063.7 s
DeepSeek V4.1 FlashDeepSeek3 of 3$0.00216.5 s
GPT-6.1 TerraOpenAI3 of 3$0.0023 s
GPT-6.1 SolOpenAI3 of 3$0.0032.5 s
Gemini 4 ArgonGoogle3 of 3$0.0044.7 s
Qwen4 MaxAlibaba3 of 3$0.00512.6 s
Claude Sonnet 5.5Anthropic3 of 3$0.0084.1 s
Gemini 4 HydrogenGoogle3 of 3$0.0129.4 s
Mistral Large 4Mistral3 of 3$0.013151.4 s
Grok 4.7SpaceXAI3 of 3$0.01416.2 s
Claude Fable 5.2Anthropic3 of 3$0.0335.6 s
GPT-6 LunaOpenAI2 of 3$0.00034.9 s

Where the answers went wrong

What a right answer has to do, most often missed first. These are the places where a person still has to look: one of the five checks caught at least one wrong answer.

  1. Spots the cheque paid in on the 30th and the cheque written on the 31st, neither at the bank yetwrong in 1 of 36
  2. Finds the bank charges and the interest that were never entered in the cash booknever wrong
  3. Notices the direct debit the bank took twice on the same daynever wrong
  4. Catches the cheque entered as 384.60 and paid as 348.60never wrong
  5. Matches the one credit of 990.00 to the two invoices it paysnever wrong

A wrong answer, word for word

From GPT-6 Luna, the most expensive model that failed this task. It reads as finished work, which is the difficulty: nothing in it says which line to doubt.

  1. Spots the cheque paid in on the 30th and the cheque written on the 31st, neither at the bank yet

    It answered
    ["C12"]
    Right
    ["C11","C12"]
{
  "cash_book_not_on_statement": ["C12"],
  "statement_not_in_cash_book": ["B08", "B11"],
  "taken_twice": ["B09", "B10"],
  "amount_differs": { "cash_book": "C03", "statement": "B04", "difference": 36.00 },
  "paid_in_together": { "statement": "B03", "cash_book": ["C04", "C05"] }
}

The task as the models saw it

A print shop closes October with a cash book and a bank statement that disagree by 951.38 pounds. The difference is made of six ordinary things, one of them the bank taking a direct debit twice. Every name and figure is invented. The models received the two texts below as one instruction and one message, and nothing else.

The documents

SABLE & FINCH PRINT LTD: CASH BOOK, OCTOBER 2026
Balance brought forward 1 October: 4,210.35

C01  02 Oct  Receipt   Harris & Co, invoice 1041                 1,250.00
C02  03 Oct  Payment   Rent, standing order                        950.00
C03  06 Oct  Payment   Cheque 300417, Baxter Supplies              384.60
C04  08 Oct  Receipt   Okafor Ltd, invoice 1043                    672.00
C05  08 Oct  Receipt   Okafor Ltd, invoice 1044                    318.00
C06  13 Oct  Payment   Card, fuel                                   64.18
C07  15 Oct  Payment   Wages, BACS                               3,120.00
C08  20 Oct  Receipt   Lindqvist AB, invoice 1046                1,485.90
C09  22 Oct  Payment   Cheque 300418, Moor Print Finishing         217.45
C10  27 Oct  Payment   Electricity, direct debit                   186.72
C11  30 Oct  Receipt   Harris & Co, invoice 1049, cheque paid in   940.00
C12  31 Oct  Payment   Cheque 300419, T. Reilly                    150.00

Balance carried forward 31 October: 3,803.30

WESTMERE BANK: STATEMENT, BUSINESS CURRENT ACCOUNT, 1 TO 31 OCTOBER 2026
Opening balance: 4,210.35

B01  02 Oct  Credit  HARRIS & CO                                 1,250.00
B02  03 Oct  Debit   STANDING ORDER, KELSO ESTATES                 950.00
B03  08 Oct  Credit  OKAFOR LTD                                    990.00
B04  09 Oct  Debit   CHEQUE 300417                                 348.60
B05  13 Oct  Debit   CARD, NORTHWAY FUELS                           64.18
B06  15 Oct  Debit   BACS, WAGES                                 3,120.00
B07  20 Oct  Credit  LINDQVIST AB                                1,485.90
B08  24 Oct  Debit   ACCOUNT CHARGES                                12.50
B09  27 Oct  Debit   DIRECT DEBIT, BRIGHTLINE ENERGY               186.72
B10  27 Oct  Debit   DIRECT DEBIT, BRIGHTLINE ENERGY               186.72
B11  29 Oct  Credit  INTEREST                                        1.84
B12  30 Oct  Debit   CHEQUE 300418                                 217.45

Closing balance: 2,851.92

The instruction

You are the bookkeeper of Sable & Finch Print Ltd. Reconcile the October cash book with the October bank statement below. The cash book lines are numbered C01 to C12 and the statement lines B01 to B12.

Return only one JSON object, with no text before or after it and no Markdown, in exactly this shape:
{
  "cash_book_not_on_statement": ["C00"],
  "statement_not_in_cash_book": ["B00"],
  "taken_twice": ["B00", "B00"],
  "amount_differs": { "cash_book": "C00", "statement": "B00", "difference": 0.00 },
  "paid_in_together": { "statement": "B00", "cash_book": ["C00", "C00"] }
}

"cash_book_not_on_statement": cash book lines that had not reached the bank by 31 October.
"statement_not_in_cash_book": statement lines for something the cash book has no entry for at all. Do not put the two lines of "taken_twice" here.
"taken_twice": the two statement lines where the bank took the same payment twice.
"amount_differs": the one payment entered in the cash book at a different amount from the one the bank paid, and the difference in pounds.
"paid_in_together": the one statement line that is two cash book receipts paid in as a single amount.

Every answer of the run is in one file, under CC BY 4.0. How a test is run and what it does not show.