LO
Worker adoptionCognitive automation2024-03-31
In a managed-detection firm's SOC, 45 analysts sent 3,090 queries to GPT-4 over ten months; only 4% asked for a verdict such as 'is this malicious?', the rest sought explanation
Security analyst (SOC)occupation page →
Event date / reported
2024-03-31 · reported 2025-09-19
Evidence stage
Worker adoptionMeasured, large-scale use of a tool for real work, where the decision to use it was the worker's rather than an employer's. It is more than a capability record — the work is real, not a demo — and less than a deployment record, because no employer put it into production, required it, or built a process around it. Weighted `cautious`: `automating` means the machine can do the task AND there are adoption signs, and this is an adoption sign — but usage can be experimental, and much of the measurement comes from a party with a stake, so one record is never enough and two independent ones are. Note who is counting. Vendor telemetry sees this directly and sells the tool, so such a record names that stake in its scope; a statistics agency asking firms whether their workers use AI in tasks sees the same channel with no stake at all, and that is the better source where it exists.
Tasks this bears on
Deciding it is an incident
Calling it: waking people up, pulling a system off the network, telling the business it has a problem — on incomplete information and before you can be sure.
Still human-led✓ Evidence-backed
Where this applies
Researchers from Australia's national science agency, with co-authors from the security firm, analysing 3,090 queries that 45 analysts at eSentire — a company selling 24/7 managed detection and response — submitted to GPT-4 during live investigations from May 2023 to March 2024. They found the model used mainly to interpret low-level telemetry such as commands and to refine written communication, in short exchanges; only 4% of queries sought explicit recommendations such as "is this malicious?", which the authors read as a strong preference for keeping decision authority. It is one company's SOC, measures what analysts asked rather than how accurate the answers were, and the firm has a commercial stake.
What this means
Even inside a company that sells incident detection as a service, analysts with a model on hand almost never asked it whether something was malicious — they asked it to explain, and made the call themselves. The verdict is the part people are keeping.
What it does not yet show
One firm's analysts in 2023–24, what they asked rather than how well it worked; it does not show whether employers keep someone to make the call.
What you can check
Open arXiv:2508.18947 ("LLMs in the SOC") and find "Only 4% of queries sought explicit recommendations".
Does it change the assessment?
No. The impact index is never moved by a single event, and this stage does not move one on its own: a "Worker adoption" record counts toward a judgement but needs a second, independent record before the judgement rests on evidence. This one is counted; on its own it changed nothing.
Source
Ronal Singh, Shahroz Tariq, Fatemeh Jalalvand, Mohan Baruwal Chhetri et al. (CSIRO Data61 with eSentire) — "LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres", arXiv:2508.18947v2 (19 September 2025) · verified 2026-09-28 · Claude (VOLO agent) · interpreted 2026-09-28 · Claude (VOLO agent)
Primary source — published by the party that did this, or the authority of record. No co-signature needed.