Can AI compare supplier quotes and pick the cheapest?
Yes. On 16 November 2026 all twelve current AI models I tested got this right three times out of three: 36 answers and not one error. The cheapest of them, GPT-6 Luna, costs $0.02 per 100 runs. The task, the documents and every result are on this page.
- Tested
- 16 November 2026
- Result
- All twelve models, every time
- Answers
- 36 of 36 passed every check
- The task
- A bakery owner’s, in purchasing
- Asked by
- Me, to open the record
The result, model by model
Each model was given the same message three times, and each square is one of its answers: in ink when it passed every check, in red when it failed at least one. The cost is what one run was billed through the API, and the time is how long the full answer took to arrive.
| Model | The three runs | Right | Cost of a run |
|---|---|---|---|
| GPT-6 LunaOpenAI | 3 of 3 | $0.0002 | |
| Claude Haiku 5.5Anthropic | 3 of 3 | $0.0005 | |
| DeepSeek V4.1 FlashDeepSeek | 3 of 3 | $0.001 | |
| GPT-6.1 TerraOpenAI | 3 of 3 | $0.002 | |
| GPT-6.1 SolOpenAI | 3 of 3 | $0.003 | |
| Qwen4 MaxAlibaba | 3 of 3 | $0.004 | |
| Gemini 4 ArgonGoogle | 3 of 3 | $0.005 | |
| Claude Sonnet 5.5Anthropic | 3 of 3 | $0.007 | |
| Mistral Large 4Mistral | 3 of 3 | $0.009 | |
| Gemini 4 HydrogenGoogle | 3 of 3 | $0.014 | |
| Claude Fable 5.2Anthropic | 3 of 3 | $0.023 | |
| Grok 4.7SpaceXAI | 3 of 3 | $0.024 |
What every answer got right
What a right answer has to do. All 36 answers did every one of these, so on this task there was nothing left for a person to correct.
- Castle: adds the delivery charge, and leaves out a discount the bakery does not qualify fornever wrong
- Dunmore: rounds the order up to whole packs of 100never wrong
- Dunmore: takes the VAT out of prices quoted with VAT includednever wrong
- Ashby: rounds the order up to whole packs of 250never wrong
- Ashby: adds the delivery and the one-off printing platenever wrong
- Names the supplier that is really cheapestnever wrong
A right answer, word for word
From GPT-6 Luna, the cheapest model that passed every check. No model gave a wrong answer on this task, so this is what all 36 answers said, each in its own layout.
{
"quotes": {
"castle": { "boxes_ordered": 1150, "total_ex_vat": 1004.00 },
"dunmore": { "boxes_ordered": 1200, "total_ex_vat": 980.00 },
"ashby": { "boxes_ordered": 1250, "total_ex_vat": 1087.50 }
},
"cheapest": "dunmore"
}The task as the models saw it
A bakery needs 1,150 printed cake boxes and has three quotes on the table. The quote with the lowest unit price is the most expensive one to accept. Every name and figure is invented. The models received the two texts below as one instruction and one message, and nothing else.
The documents
QUOTE 1: CASTLE PACKAGING LTD Printed cake box, 8 inch, one colour: 0.84 pounds per box, excluding VAT. Sold in packs of 50 only. Delivery: 38.00 pounds excluding VAT. Free delivery on orders of 1,000.00 pounds or more of goods, excluding VAT. Settlement discount: 3 per cent off the goods if the invoice is paid within 7 days. QUOTE 2: DUNMORE PRINT & PACK All our prices include VAT at 20 per cent. Printed cake box, 8 inch, one colour: 0.96 pounds per box. Sold in packs of 100 only. Delivery: 24.00 pounds per order. QUOTE 3: ASHBY BOX CO. Printed cake box, 8 inch, one colour, prices excluding VAT: 1 to 999 boxes: 0.92 pounds per box 1,000 to 1,999 boxes: 0.79 pounds per box The price band is set by the number of boxes ordered and applies to all of them. Sold in packs of 250 only. Delivery: 55.00 pounds per order, excluding VAT. Printing plate, charged once on a first order: 45.00 pounds excluding VAT. Fenwick & Rye Bakery has not ordered from us before.
The instruction
You help the owner of Fenwick & Rye Bakery choose a supplier. The bakery needs at least 1,150 printed cake boxes. It is VAT registered, so compare the quotes excluding VAT. It pays every supplier 30 days after the invoice.
For each supplier work out the cheapest order that delivers at least 1,150 boxes, and what the bakery pays for it in total, excluding VAT, with every charge the quote makes.
Return only one JSON object, with no text before or after it and no Markdown, in exactly this shape:
{
"quotes": {
"castle": { "boxes_ordered": 0, "total_ex_vat": 0.00 },
"dunmore": { "boxes_ordered": 0, "total_ex_vat": 0.00 },
"ashby": { "boxes_ordered": 0, "total_ex_vat": 0.00 }
},
"cheapest": "castle | dunmore | ashby"
}Every answer of the run is in one file, under CC BY 4.0. How a test is run and what it does not show.
The other tests
Have a task of your own? Describe it and I will test it the same way. Send a task
- Can AI check an expense claim against the company policy?All twelve models, every time
- Can AI find the cancellation deadline in a contract?All twelve models, every time
- Can AI reconcile a bank statement?11 of 12 models, every time
- Can AI shortlist job applicants against the requirements?All twelve models, every time
- Can AI write a staff rota that follows the rules?10 of 12 models, every time