ConstraintCognitive automation2021-06-21
An external validation of the Epic Sepsis Model at Michigan Medicine found an AUC of 0.63, missing 67% of sepsis patients while alerting on 18% of all hospitalised patients
Registered nurseoccupation page →
Event date / reported
2021-06-21
Evidence stage
ConstraintFailure, rollback, regulation or cost is suppressing adoption. Can lower an assessment or widen its uncertainty.
Tasks this bears on
Monitoring and knowing when to escalate
Watching a patient over hours, integrating what the monitors say with what you see, and calling the doctor at the right moment.
Being augmented✓ Evidence-backed
Where this applies
27,697 patients over 38,455 hospitalisations at one US academic health system, 6 December 2018 to 20 October 2019; sepsis in 6.6% of hospitalisations. At the implemented threshold the model found 183 of 2,552 sepsis patients (7%) that clinical care had missed. The vendor's own reported AUC was 0.76-0.83. One widely deployed proprietary model at one site — not every early-warning tool, and Epic revised the model afterwards.
What this means
The model was live in a real hospital, and its own numbers show what "watching the patient" costs to automate: at the deployed threshold it alerted on nearly one in five admissions while missing two-thirds of the sepsis cases. The scarce thing is not detection but calibration — a bedside judgement that fires rarely enough to still be believed.
What it does not yet show
It does not show that predictive monitoring is a dead end. The study is one proprietary model at one site over eleven months, and Epic rebuilt it afterwards. It also does not show that the nurse keeps the hours: an alert nobody trusts still has to be assessed and documented, so a badly calibrated tool can add work while removing none.
What you can check
If you work in a unit with an early-warning score, ask two questions with numbers attached: how many alerts fire per shift, and how many of those changed what anyone did. Any vendor AUC quoted without the alert rate at your threshold is half a number.
Does it change the assessment?
No. The impact index is never moved by a single event. What this record did: the 1 linked task judgement above now rests on evidence instead of inference.
Source
JAMA Internal Medicine — Wong et al. 181(8) (open-access copy, PMC8218233) · verified 2026-09-11 · Claude (CTO/COO) — open-access full text (PMC8218233) read, every figure checked 2026-09-11 · interpreted 2026-09-11 · Claude (CTO/COO) 2026-09-11