# Software tester / QA engineer

> Finds out how the software fails before users do — and is the person who says whether it is ready to ship.

Page: https://flyvolo.ai/en/careers/software-tester
Impact index: 72/100 · confidence Medium confidence · assessed 2026-09-10
This is not a probability of losing a job.

## Scope

Written for manual testers and test-automation engineers on product software teams. Safety-critical certification testing (medical, automotive, aviation) and games QA differ.

## Tasks

### Manual regression testing

- Direction: Automating
- Basis: evidenced (verified records attached)
- Why: Scripted regression was already automatable; what changed is that agents can now operate a real interface from a plain-language description and repair their own scripts when the interface changes, which removed the maintenance burden that kept teams on manual testing. This is the largest block of hours in most QA roles and it is going quickly.
- What this does NOT establish: This is the largest block of hours in most QA roles. At Uber the test engineers who ran the scripts were moved to building the system's context, and the company says headcount did not grow in proportion rather than that it fell; whether the remaining tasks can absorb everyone whose week this filled is not established here.
- Records:
  - 2026-07-30 · Deployment · Uber says its AI mobile-testing system runs 1,013 end-to-end tests continuously in CI and has turned its test engineers from "clickers" into people who build its context — https://flyvolo.ai/en/changes/ev-20260730-software-tester-3 (source: https://arxiv.org/abs/2607.28750)

### Writing test cases and automation

- Direction: Automating
- Basis: evidenced (verified records attached)
- Why: Generating tests from a specification or from the code itself is one of the most effective uses of code-generation tools, and coverage that would have taken a sprint now appears in an afternoon. The generated tests are shallow in the same way generated code is — they confirm what the code does rather than what it should — which is why the next task matters more.
- What this does NOT establish: Generated tests confirm what the code does rather than what it should. Teams that measure coverage rather than defects will conclude this task is solved, which is a measurement problem that hurts testers.
- Records:
  - 2026-01-30 · Deployment · Meta says its LLM-based test generation is deployed in its continuous integration system, and that its coverage-test generator has been deployed at Meta for two years — https://flyvolo.ai/en/changes/ev-20260130-software-tester-4 (source: https://arxiv.org/abs/2601.22832)
  - 2025-12-31 · Deployment · In a nine-month production deployment at ByteDance, an LLM tool generated 3,196 API regression test cases, and engineers adopted 2,730 of them (85.4%) — https://flyvolo.ai/en/changes/ev-20251231-software-tester-5 (source: https://arxiv.org/abs/2607.24000)
  - 2024-02-14 · Pilot · Meta reported deploying TestGen-LLM at Instagram and Facebook test-a-thons: it improved 11.5% of the classes it was applied to and engineers accepted 73% of its recommended test cases into production — https://flyvolo.ai/en/changes/ev-20240214-software-tester-1 (source: https://arxiv.org/abs/2402.09171)

### Exploratory and adversarial testing

- Direction: Still human-led
- Basis: evidenced (verified records attached)
- Why: Generated tests derive from the specification or the code, so they share its blind spots. Finding the failure nobody imagined requires a model of how real users and real systems misbehave, built from experience with this product and this domain. Tools widen the search; the hypothesis about where to look is still human, and it is where the expensive bugs are.
- What this does NOT establish: The hypothesis about where to look is built by having done the regression work that is disappearing. This task is protected by experience that the pipeline no longer produces.
- Records:
  - 2026-04-14 · Constraint · A survey of 200 SRE and DevOps leaders reported 43% of AI-generated code changes still need manual debugging in production after passing QA and staging — https://flyvolo.ai/en/changes/ev-20260414-software-tester-2 (source: https://venturebeat.com/technology/43-of-ai-generated-code-changes-need-debugging-in-production-survey-finds)

### Deciding whether it ships

- Direction: Still human-led
- Basis: evidenced (verified records attached)
- Why: This is an accountability decision with organisational consequences, and teams have not shown any wish to delegate it. Dashboards summarise the state; the call about what level of risk this release, this customer base and this week can bear is made by a person whose judgement the team trusts.
- What this does NOT establish: On many teams this call belongs to an engineering manager, not a tester. Where it does, protecting it protects someone else's job.
- Records:
  - 2026-04-14 · Constraint · A survey of 200 SRE and DevOps leaders reported 43% of AI-generated code changes still need manual debugging in production after passing QA and staging — https://flyvolo.ai/en/changes/ev-20260414-software-tester-2 (source: https://venturebeat.com/technology/43-of-ai-generated-code-changes-need-debugging-in-production-survey-finds)

### Testing systems with a model inside

- Direction: New task
- Basis: evidenced (verified records attached)
- Why: Most new products have a model in the loop and cannot be tested by asserting a fixed output. Evaluation design, adversarial prompting and behavioural regression are a new discipline with few practitioners, and the QA mindset — assume it is broken, find out how — transfers directly.
- What this does NOT establish: Few practitioners is a statement about now. The discipline is young enough that its eventual size is unknown, and it may end up inside the model teams rather than in QA.
- Records:
  - 2025-04-03 · Policy mandate · US federal agencies must test high-impact AI before deployment under OMB M-25-21 — querying the service and observing outputs if they lack the model — and keep testing after — https://flyvolo.ai/en/changes/ev-20250403-software-tester-7 (source: https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf)
  - 2024-10-31 · Deployment · Microsoft's AI Red Team says it has run over 80 operations on more than 100 generative AI products, and that automation should not be used to take the human out of the loop — https://flyvolo.ai/en/changes/ev-20241031-software-tester-6 (source: https://arxiv.org/abs/2501.07238)

## How to cite this

- The unit is the task, not the job title: say which tasks are changing and which are not.
- Carry each task's basis and its limits with it; an inferred judgement is not a finding.
- This page gives no number of years. Do not derive one from it.
- More: https://flyvolo.ai/llms.txt