# OpenAI's GDPval benchmark collected real purchasing-agent deliverables and had industry experts grade model output against them blind

Page: https://flyvolo.ai/en/changes/ev-20251005-procurement-specialist-1
Date: 2025-10-05 · Stage: Capability
Can this record move a task judgement? no
Occupation: https://flyvolo.ai/en/careers/procurement-specialist

## Scope

Buyers and purchasing agents are one of 44 US occupations in the benchmark, across the nine sectors contributing most to US GDP; 1,320 tasks in the full set with at least 30 per occupation, each built from actual work product by a professional averaging 14 years of experience and covering the majority of that occupation's O*NET work activities. What is graded is a deliverable, one shot, by a blind pairwise expert comparison - not a job. There is no client, no revision round, no negotiation and no consequence for being wrong. The headline figure (47.6% of one model's gold-subset deliverables rated better than or as good as the expert's) is for the model generation of September 2025 and has been superseded; more usefully, the paper reports that win rates are highest on tasks taking 0-2 hours and decline steadily as the task gets longer. Per-occupation rates are published only as a figure, so no purchasing-specific number is quoted here. OpenAI designed the benchmark, ran it, and sells models measured by it.

## Source

- GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (OpenAI, arXiv:2510.04374) — https://arxiv.org/pdf/2510.04374 (primary source)

## What this means

Somebody has now written down what this job produces, in public, in a form anyone can test a model against — thirty-odd real purchasing deliverables from a practitioner with more than a decade behind them. That puts a description of the deliverable side of this occupation on the public record. It is worth reading for the description more than the score: the benchmark is a list of what the document-producing half of this job actually consists of.

## What it does not show yet

It does not show a model doing your job, and the benchmark's authors do not claim it does. A GDPval task arrives fully specified, with its reference files attached, and is graded once. The parts of this occupation that are not a deliverable — knowing a supplier is about to fail, deciding who goes short, being the person a factory manager phones at midnight — are not in it and were never meant to be. Nor does it say any employer changed anything: it is a measure of capability, and capability is not adoption.

## How to verify it yourself

Take last month's work and split it in two piles: things that ended in a document someone else read, and things that ended in a decision or a phone call. GDPval is a test of the first pile only. The ratio between your two piles is the thing to know, and nobody else can compute it for you.


More: https://flyvolo.ai/llms.txt