Can AI shortlist job applicants against the requirements?
Yes. On 16 November 2026 all twelve current AI models I tested got this right three times out of three: 36 answers and not one error. The cheapest of them, GPT-6 Luna, costs $0.02 per 100 runs. The task, the documents and every result are on this page.
- Tested
- 16 November 2026
- Result
- All twelve models, every time
- Answers
- 36 of 36 passed every check
- The task
- A warehouse manager’s, in hiring
- Asked by
- Me, to open the record
The result, model by model
Each model was given the same message three times, and each square is one of its answers: in ink when it passed every check, in red when it failed at least one. The cost is what one run was billed through the API, and the time is how long the full answer took to arrive.
| Model | The three runs | Right | Cost of a run |
|---|---|---|---|
| GPT-6 LunaOpenAI | 3 of 3 | $0.0002 | |
| Claude Haiku 5.5Anthropic | 3 of 3 | $0.0006 | |
| DeepSeek V4.1 FlashDeepSeek | 3 of 3 | $0.0009 | |
| GPT-6.1 TerraOpenAI | 3 of 3 | $0.002 | |
| GPT-6.1 SolOpenAI | 3 of 3 | $0.002 | |
| Gemini 4 ArgonGoogle | 3 of 3 | $0.002 | |
| Qwen4 MaxAlibaba | 3 of 3 | $0.003 | |
| Claude Sonnet 5.5Anthropic | 3 of 3 | $0.006 | |
| Gemini 4 HydrogenGoogle | 3 of 3 | $0.010 | |
| Mistral Large 4Mistral | 3 of 3 | $0.020 | |
| Grok 4.7SpaceXAI | 3 of 3 | $0.021 | |
| Claude Fable 5.2Anthropic | 3 of 3 | $0.025 |
What every answer got right
What a right answer has to do. All 36 answers did every one of these, so on this task there was nothing left for a person to correct.
- Passes Tomasz, who meets all five requirementsnever wrong
- Stops Priya: her forklift licence expired in September and the refresher is after the start datenever wrong
- Passes Marcus: 14 months and 11 months of supervising add up to more than two yearsnever wrong
- Stops Aoife, the strongest application, because she cannot work nightsnever wrong
- Stops Kwame: he would need a visa the employer has said it cannot sponsornever wrong
- Stops Hannah: her licence is for a reach truck, not a counterbalance trucknever wrong
- Stops Li Wei: January 2025 to October 2026 looks like two years and is under 22 monthsnever wrong
A right answer, word for word
From GPT-6 Luna, the cheapest model that passed every check. No model gave a wrong answer on this task, so this is what all 36 answers said, each in its own layout.
{"fails":{"A1":[],"A2":["M2"],"A3":[],"A4":["M4"],"A5":["M1"],"A6":["M2"],"A7":["M3"]}}The task as the models saw it
Seven people apply for a warehouse team leader job with five stated requirements. Two meet all five. A wrong shortlist either wastes an interview or turns away someone who qualifies. Every name and figure is invented. The models received the two texts below as one instruction and one message, and nothing else.
The documents
CALDER VALLEY FOODS LTD: WAREHOUSE TEAM LEADER Full time, starting 1 December 2026. The company cannot sponsor visas. Requirements. An applicant must meet all five. M1 The right to work in the United Kingdom on the start date, without sponsorship. M2 A counterbalance forklift licence that is in date on the start date. M3 At least 24 months of experience supervising a team, in total across all jobs. M4 Able to work the rotating shift pattern, which includes night shifts. M5 A Level 2 food safety certificate. APPLICATIONS A1 Tomasz W. Has settled status in the UK. Counterbalance forklift licence valid until 14 March 2027. Shift supervisor at a chilled distribution centre for three years, leading a team of nine. Has worked rotating shifts including nights throughout. Level 2 food safety certificate, 2025. A2 Priya N. British citizen. Counterbalance forklift licence expired on 30 September 2026; refresher course booked for 9 January 2027. Team leader at a bakery warehouse for four years. Happy with any shift including nights. Level 2 food safety certificate, 2024. A3 Marcus B., aged 58. British citizen. Counterbalance forklift licence valid until 2028. Supervised the goods-in team at a frozen foods depot for 14 months, then supervised a picking team at a supermarket warehouse for 11 months. Currently on rotating shifts with nights. Level 2 food safety certificate, 2026. A4 Aoife D. Irish citizen, living in Leeds. Counterbalance forklift licence valid until 2029. Warehouse supervisor for five years, team of 22, and site trainer for new team leaders. Level 3 food safety certificate (which covers Level 2). Writes: "I can work early and late shifts. I am not able to work night shifts." A5 Kwame O. Lives in Accra, Ghana. Writes that he would relocate and would need a Skilled Worker visa sponsored by the employer. Counterbalance forklift licence issued in the UK in 2023, valid until 2028. Supervised a team of six for three years in a UK warehouse. Has worked nights and is able to again. Level 2 food safety certificate, 2023. A6 Hannah S. British citizen, has a young child. Reach truck licence valid until 2028; has never held a counterbalance licence. Team leader for 30 months at a drinks distributor. Works rotating shifts including nights now. Level 2 food safety certificate, 2025. A7 Li Wei C. Indefinite leave to remain in the UK. Counterbalance forklift licence valid until August 2027. Team leader at a parcel hub from 6 January 2025 to 30 October 2026, her only supervising role. Has worked nights for two years. Level 2 food safety certificate, 2025.
The instruction
You help the warehouse manager of Calder Valley Foods Ltd sort applications for the job below. For each applicant, list which of the five requirements (M1 to M5) the application does not meet. Judge only against the five requirements, on what the application says. An applicant with an empty list goes to interview.
Return only one JSON object, with no text before or after it and no Markdown, in exactly this shape:
{
"fails": { "A1": [], "A2": [], "A3": [], "A4": [], "A5": [], "A6": [], "A7": [] }
}
Each list holds the codes of the requirements that applicant does not meet, for example ["M2"], or is empty.Every answer of the run is in one file, under CC BY 4.0. How a test is run and what it does not show.
The other tests
Have a task of your own? Describe it and I will test it the same way. Send a task
- Can AI check an expense claim against the company policy?All twelve models, every time
- Can AI compare supplier quotes and pick the cheapest?All twelve models, every time
- Can AI find the cancellation deadline in a contract?All twelve models, every time
- Can AI reconcile a bank statement?11 of 12 models, every time
- Can AI write a staff rota that follows the rules?10 of 12 models, every time