LO
CapabilityCognitive automation2025-06-24
Language models mix first-print economic data with later revisions and often assume they have data that has not been released, a Federal Reserve Board staff study found
Economistoccupation page →
Event date / reported
2025-06-24
Evidence stage
CapabilityA demo, benchmark or paper shows the task can be done. Updates what the technology can do — not what employers will do.
Tasks this bears on
Forecasting and nowcasting
Producing forecasts of growth, inflation and other indicators, and estimating where the economy is now before official data arrive.
Being augmented✓ Evidence-backed
Data work and modelling
Building datasets, writing code, estimating models and checking that results hold up.
Being augmented✓ Evidence-backed
Where this applies
United States, Federal Reserve Board staff. Testing language models' recall of macroeconomic data, the authors find accurate recall overall but two important errors: mixing first-print data with later revisions and mixing data for past and future periods; on any given day the model is likely to believe it has data in hand that has not been released, which limits its use for historical analysis or to mimic real-time forecasters.
What this means
Language models get economic data roughly right but blur revisions and release dates, which matters for real-time forecasting.
What it does not yet show
One staff study of particular models; newer models may differ.
What you can check
Open FEDS 2025-044 (24 June 2025) and find "data in hand which has not been released".
Does it change the assessment?
No — and this stage does not move it either. A "Capability" record is real evidence, but it does not upgrade a task judgement on its own. The 2 linked judgements above stand where they were.
Source
Crane L, Karra A, Soto P — Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models, Finance and Economics Discussion Series 2025-044, Federal Reserve Board (June 24, 2025) · verified 2026-10-02 · Claude (VOLO agent) · interpreted 2026-10-02 · Claude (VOLO agent)
Primary source — published by the party that did this, or the authority of record. No co-signature needed.