Machine learning engineer — tasks, one by one

Building a model for the problem

Two forces, and only one of them is what this site usually means by automation. Tooling genuinely automated the search — architecture and hyperparameter selection became a job you configure rather than perform. The larger force is different in kind: a great many problems that used to require training something now get solved by calling a model somebody else trained. The task did not become machine-doable; its product became purchasable.

This site's four directions cannot tell those two mechanisms apart, and the difference matters to anyone planning around this page: a task that became purchasable can become unpurchasable again when the vendor's price, licence or capability moves, in a way that a task a machine learned to do does not. It also says nothing about the domains where a bought model is not an option — narrow, proprietary, latency-bound or regulated ones — which are not rare.

Deciding what counts as good enough

A learned system's correctness cannot be defined, only measured — so the measuring instrument is the deliverable, and it has to be built by someone who knows what a mistake costs here. A benchmark that the model has already seen measures nothing, which makes constructing an honest test set an adversarial job rather than a data-collection one. This is the task that got scarcer as models got better, because the harder the thing being judged, the harder the judging.

A judgement about the nature of the work, not a measurement of how it is staffed. Plenty of teams do not do this at all and ship on a vendor's published benchmark, which is the failure this describes rather than evidence against it — but how many is not counted anywhere.

Getting data the thing can learn from

Model-assisted labelling and synthetic generation took most of the volume out of this, and that is real: what used to need a team for weeks is often a first pass in an afternoon. What did not transfer is knowing which examples the collection method never had a chance to contain — an absence is invisible to any tool that only reads what is there.

Nothing here measures how much of a given team's time this takes, and the answer differs by an order of magnitude between a team with an existing data asset and one starting from nothing. Using a model to label data that trains a model also has known failure modes this judgement does not weigh.

When it quietly stops working

Detection genuinely improved: monitoring tools now surface a distribution shift better and earlier than a person watching dashboards. Deciding what to do about it did not move, because the options trade off against each other in business terms — retraining costs money and can make things worse, turning it off has a visible cost today, and doing nothing is a decision too.

Says nothing about whether teams actually monitor. A model running unwatched for a year is common and this judgement does not capture it — where nobody watches, this task is not human-led, it is simply not done.

Hand-crafting the inputs

This one was largely over before the period this site measures. Representation learning replaced hand-designed features across vision, speech and text through the 2010s, and the pattern has extended into tabular and sequence problems since. It is on the page because it is still what a great deal of training material teaches, so people arrive expecting it to be the craft.

Not evidence about the 2022-2026 window at all; it is older than that and is recorded here for orientation. Domain-specific feature design is also still load-bearing in some regulated settings where an unexplainable input is not allowed, which this judgement does not separate out.

Answering for what it does to people

New work, and it arrives from regulation rather than from capability: several markets now attach duties to systems used in hiring, credit, education and public services, including a requirement that whoever deploys one ensures its input data is relevant and sufficiently representative for the purpose. A duty of that shape has to land on a person who understands what the model actually consumed, and the site holds a verified record of exactly such a clause taking effect — attached to the deployer's page, `business-systems-owner`, because that is who the clause names.

The record this reasoning points at is not attached to this occupation, and deliberately so: the obligation names the deployer, and attaching it here would claim it lands on engineers when in most organisations nobody has yet been told it lands on them. So this is inference about where the work will sit, not evidence that it already sits there. Requirements also differ sharply by market and by what the system is used for.