# On the Finance Agent Benchmark — 537 expert-authored questions over recent SEC filings — the best model, OpenAI o3, reached 46.8% accuracy at $3.79 per query

Page: https://flyvolo.ai/en/changes/ev-20250520-financial-analyst-3
Date: 2025-05-20 · Stage: Constraint
Can this record move a task judgement? yes
Occupation: https://flyvolo.ai/en/careers/financial-analyst

## Scope

Nine task categories from information retrieval to financial modelling, authored with experts from banks, hedge funds and private equity; the agents were given Google Search and EDGAR access. Public-filing research only — not an analyst working inside their own firm's models and internal data.

## Source

- arXiv 2508.00828 (Bigeard, Nashold, Krishnan, Wu) — https://arxiv.org/abs/2508.00828 (primary source)

## What this means

The agents had the tools an analyst has — search and the filings database — and still got fewer than half the expert-written questions right, at nearly four dollars a query. That places the current line inside the job rather than around it: across nine task categories from retrieval to modelling, the best model was right less than half the time, and cost is a live constraint at volume.

## What it does not show yet

A benchmark is not an employment outcome, and this one is public filings only — it says nothing about an analyst working inside their own firm models and internal data, where the context is richer and the errors are more expensive. Nor does it hold still: it measures the models of its date, so treat 46.8% as a dated reading, not a ceiling.

## How to verify it yourself

Take one model you maintain and one assumption inside it — the growth rate, say. Ask a current model to derive that number from the filings alone, then check every figure it cites back to the document. What you are measuring is not whether it sounds right but how many of its numbers survive being traced.


More: https://flyvolo.ai/llms.txt