LO
CapabilityCognitive automation2024-09-13
Examiners found no significant difference in overall scores between ChatGPT and anaesthesiology fellows on sample oral board exams, but identified the machine's answers in 23 of 24 modules
Anaesthesiologist / anaesthetistoccupation page →
Event date / reported
2024-09-13
Evidence stage
CapabilityA demo, benchmark or paper shows the task can be done. Updates what the technology can do — not what employers will do.
Tasks this bears on
Pre-operative assessment and planning
Examining the patient, reviewing history, medicines and risks, and deciding the anaesthetic plan.
Being augmented≈ Platform inference
Where this applies
An exploratory study comparing four anaesthesiology fellows with ChatGPT on two sample US oral board examinations, which test judgement and adapting to unexpected clinical changes, scored blind by eight examiners using a voice replicator. Fellows scored better on module topics, with no significant difference in overall module scores, and examiners identified the ChatGPT answers in 23 of 24 modules. A small study of sample exams.
What this means
On an exam that tests judgement, the model scored close to trainees, and examiners could still tell it apart.
What it does not yet show
Four trainees and two sample exams; it does not measure care of real patients.
What you can check
Open the Anesthesia & Analgesia study (doi 10.1213/ane.0000000000006875) and find "23 of 24 modules".
Does it change the assessment?
No — and this stage does not move it either. A "Capability" record is real evidence, but it does not upgrade a task judgement on its own. The 1 linked judgement above stand where they were.
Source
Blacker et al. — An Exploratory Analysis of ChatGPT Compared to Human Performance With the Anesthesiology Oral Board Examination: Initial Insights and Implications, Anesthesia & Analgesia (published online 13 September 2024) · verified 2026-09-30 · Claude (VOLO agent) · interpreted 2026-09-30 · Claude (VOLO agent)
Primary source — published by the party that did this, or the authority of record. No co-signature needed.