# Machine learning engineer

> Ships systems whose behaviour is learned rather than written — which means nobody can say line by line why they do what they do, and someone still has to decide whether they are good enough to use on people.

Page: https://flyvolo.ai/en/careers/machine-learning-engineer
Impact index: 47/100 · confidence Low confidence · assessed 2026-09-12
This is not a probability of losing a job.

## Scope

Written for engineers who build, evaluate and run learned systems — recommendation, ranking, vision, speech, fraud, forecasting, and increasingly applications built on a bought model. It sits apart from the other computing pages on a different axis: those cut by layer or seniority, this one by what you ship. Research scientists producing new methods, and the people responsible for rolling AI out across a company, are different jobs with their own pages — `ai-researcher` and `ai-implementation-lead`.

## Tasks

### Building a model for the problem

- Direction: Automating
- Basis: evidenced (verified records attached)
- Why: Two forces, and only one of them is what this site usually means by automation. Tooling genuinely automated the search — architecture and hyperparameter selection became a job you configure rather than perform. The larger force is different in kind: a great many problems that used to require training something now get solved by calling a model somebody else trained. The task did not become machine-doable; its product became purchasable.
- What this does NOT establish: This site's four directions cannot tell those two mechanisms apart, and the difference matters to anyone planning around this page: a task that became purchasable can become unpurchasable again when the vendor's price, licence or capability moves, in a way that a task a machine learned to do does not. It also says nothing about the domains where a bought model is not an option — narrow, proprietary, latency-bound or regulated ones — which are not rare.
- Records:
  - 2025-11-21 · Deployment · Instacart replaced separately trained query-understanding models with one fine-tuned LLM serving millions of weekly searches, moving its engineers' core challenge away from feature engineering — https://flyvolo.ai/en/changes/ev-20251121-machine-learning-engineer-4 (source: https://company.instacart.com/tech-innovation/building-the-intent-engine-how-instacart-is-revamping-query-understanding-with-llms)
  - 2024-10-09 · Capability · OpenAI built a benchmark of 75 Kaggle competitions to test agents at ML engineering; the best setup reached bronze-medal level in 16.9% of them — https://flyvolo.ai/en/changes/ev-20241009-machine-learning-engineer-1 (source: https://arxiv.org/abs/2410.07095)

### Deciding what counts as good enough

- Direction: Still human-led
- Basis: evidenced (verified records attached)
- Why: A learned system's correctness cannot be defined, only measured — so the measuring instrument is the deliverable, and it has to be built by someone who knows what a mistake costs here. A benchmark that the model has already seen measures nothing, which makes constructing an honest test set an adversarial job rather than a data-collection one. This is the task that got scarcer as models got better, because the harder the thing being judged, the harder the judging.
- What this does NOT establish: A judgement about the nature of the work, not a measurement of how it is staffed. Plenty of teams do not do this at all and ship on a vendor's published benchmark, which is the failure this describes rather than evidence against it — but how many is not counted anywhere.
- Records:
  - 2026-04-13 · Deployment · The 2025 US federal AI inventory lists 227 deployed systems their agencies rated high-impact; testing is recorded as complete for 44, in progress for 81 and left blank for 102 — https://flyvolo.ai/en/changes/ev-20260128-machine-learning-engineer-3 (source: https://github.com/ombegov/2025-Federal-Agency-AI-Use-Case-Inventory)
  - 2024-10-09 · Capability · OpenAI built a benchmark of 75 Kaggle competitions to test agents at ML engineering; the best setup reached bronze-medal level in 16.9% of them — https://flyvolo.ai/en/changes/ev-20241009-machine-learning-engineer-1 (source: https://arxiv.org/abs/2410.07095)

### Getting data the thing can learn from

- Direction: Being augmented
- Basis: evidenced (verified records attached)
- Why: Model-assisted labelling and synthetic generation took most of the volume out of this, and that is real: what used to need a team for weeks is often a first pass in an afternoon. What did not transfer is knowing which examples the collection method never had a chance to contain — an absence is invisible to any tool that only reads what is there.
- What this does NOT establish: Nothing here measures how much of a given team's time this takes, and the answer differs by an order of magnitude between a team with an existing data asset and one starting from nothing. Using a model to label data that trains a model also has known failure modes this judgement does not weigh.
- Records:
  - 2024-05-16 · Deployment · Microsoft's Bing team reports using LLMs with expert human labellers for most of its offline relevance labels since late 2022, finding them more accurate than third-party labellers — https://flyvolo.ai/en/changes/ev-20240516-machine-learning-engineer-5 (source: https://arxiv.org/abs/2309.10621)

### When it quietly stops working

- Direction: Being augmented
- Basis: evidenced (verified records attached)
- Why: Detection genuinely improved: monitoring tools now surface a distribution shift better and earlier than a person watching dashboards. Deciding what to do about it did not move, because the options trade off against each other in business terms — retraining costs money and can make things worse, turning it off has a visible cost today, and doing nothing is a decision too.
- What this does NOT establish: The count here covers only the federal systems that chose to answer, and it describes processes, not how well they work. A model running unwatched for a year is common — where nobody watches, this task is not done at all.
- Records:
  - 2025-01-23 · Deployment · In the 2024 US federal AI inventory, of 76 AI systems in operation that described their monitoring, 47 relied on manual model updates and only 5 had a full pipeline with drift detection — https://flyvolo.ai/en/changes/ev-20250123-machine-learning-engineer-6 (source: https://github.com/ombegov/2024-Federal-AI-Use-Case-Inventory/blob/739f686fdf8c52d1be15bd6e945aec2ef59c8a68/data/2024_consolidated_ai_inventory_raw_v2.csv)

### Hand-crafting the inputs

- Direction: Automating
- Basis: evidenced (verified records attached)
- Why: This one was largely over before the period this site measures. Representation learning replaced hand-designed features across vision, speech and text through the 2010s, and the pattern has extended into tabular and sequence problems since. It is on the page because it is still what a great deal of training material teaches, so people arrive expecting it to be the craft.
- What this does NOT establish: The evidence is individual companies describing their own systems, not a measurement across the field. Domain-specific feature design is also still load-bearing in some regulated settings where an unexplainable input is not allowed, which this judgement does not separate out.
- Records:
  - 2025-11-21 · Deployment · Instacart replaced separately trained query-understanding models with one fine-tuned LLM serving millions of weekly searches, moving its engineers' core challenge away from feature engineering — https://flyvolo.ai/en/changes/ev-20251121-machine-learning-engineer-4 (source: https://company.instacart.com/tech-innovation/building-the-intent-engine-how-instacart-is-revamping-query-understanding-with-llms)

### Answering for what it does to people

- Direction: New task
- Basis: evidenced (verified records attached)
- Why: New work, and it arrives from regulation rather than from capability: several markets now attach duties to systems used in hiring, credit, education and public services. The EU AI Act requires the data behind a high-risk system to be examined for possible biases and documented as fit for its intended purpose; the duty falls on the provider, and it has to land on a person who understands what the model actually consumed. The rule is adopted and applies from 2 December 2027 for most high-risk systems.
- What this does NOT establish: A rule on the books that does not yet apply, and it names the organisation, not the individual engineer; requirements differ sharply by market and by what the system is used for, and nothing here shows who inside a company has been told the work is theirs.
- Records:
  - 2024-08-01 · Constraint · The EU AI Act requires the data behind a high-risk AI system to be examined for bias and documented as fit for its intended purpose (Article 10), from 2 December 2027 — https://flyvolo.ai/en/changes/ev-20260802-machine-learning-engineer-2 (source: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02024R1689-20260727)

## How to cite this

- The unit is the task, not the job title: say which tasks are changing and which are not.
- Carry each task's basis and its limits with it; an inferred judgement is not a finding.
- This page gives no number of years. Do not derive one from it.
- More: https://flyvolo.ai/llms.txt