AI Made QA More Important, Not Less

The pitch for AI in creative work is that it removes labor.

It does. A campaign's worth of variants in an afternoon. A hundred product descriptions before lunch. A landing page from a brief. Alt text for a four-year image library. A first draft of anything, instantly, at a quality that would have taken a competent person an hour.

What the pitch does not mention is what happens to everything downstream of production.

Because production was never the bottleneck in quality. Verification was. And AI has just multiplied the input to the verification step by an order of magnitude while leaving its capacity exactly where it was.

This is the eleventh article in this series.

Fluency Is Not Correctness

Every previous source of error in creative work had a tell.

A rushed draft reads rushed. A junior copywriter's uncertainty shows up in the prose. A developer who did not understand the requirement writes code that looks like someone who did not understand the requirement. Human error is usually correlated with visible signs of the conditions that produced it, which is why experienced reviewers develop an instinct for where to look.

Generated output breaks that correlation completely.

An LLM produces a confidently wrong statistic in exactly the same register as a correct one. The sentence containing a fabricated case study is as well-constructed as the sentence containing a real one. The code with the wrong edge-case behavior is formatted, commented and idiomatic.

There is no tell.

This is a genuinely new failure mode, and it defeats the review technique most teams actually rely on, which is scanning for things that look wrong. Nothing looks wrong. That is the product working as designed.

Fluency was historically a weak proxy for care. It is now a proxy for nothing at all.

Why It Errs at All

It is worth being precise about the mechanism, because the mechanism is what determines which controls can possibly work.

A language model does not retrieve an answer and then phrase it. At each step it produces a probability distribution over possible next tokens and samples from it. That sampling step is the source of most of what follows.

Three consequences matter for QA.

It is not deterministic. Give the same model the same prompt twice and you can get two different outputs. That is not a defect in the implementation; it is how the decoding works. Lower the temperature and the variance shrinks, but it does not become a different kind of system. And even with sampling turned off entirely, output is not reliably bit-identical run to run, because the arithmetic underneath is executed in parallel and floating-point addition is not associative. Change the batch size or the hardware and sums are accumulated in a different order, which occasionally flips a token.

There is no step where it checks anything. Nothing in that loop consults a price list, a calendar, or a fact. The model is optimizing for a plausible continuation given everything before it, and a plausible continuation is frequently a true one, which is exactly what makes this hard. Truth is a strong correlate of plausibility in the training data, not a constraint on the output. Retrieval and tool use change this materially, since the model can be handed the real number, but only for the facts you thought to retrieve, and only if the generated text actually reflects what came back.

Fluency and correctness are produced by the same process. They are not two stages, one of which could be weaker. The sentence containing an invented statistic is generated by exactly the mechanism that generated the sentence containing a real one, which is why it comes out equally well-formed.

The Defect You Cannot Regression-Test

That non-determinism has a consequence most creative teams have not absorbed, and it breaks the mental model that people carry over from software.

A conventional bug is deterministic. Same input, same wrong output. Which means you can reproduce it, write a test that fails, fix it, watch the test pass, and keep that test forever. The defect is caught once and stays caught.

A generation defect may not reproduce.

You review the output, it is fine. You run it again to check, it is fine. It fails on the run that went to production, or on the fourth of forty variants, or for one product in a catalogue of nine hundred.

This means a passing review proves something much weaker than teams assume. It proves the system can produce correct output. It does not establish that it will, and there is no test you can write that converts the first into the second.

The practical translation is uncomfortable but simple: with a generator in the pipeline, verification cannot be a property of the process that was validated once. It has to be a property applied to the specific artifact that is about to be published. Sampling five of forty assets and approving the batch is a reasonable technique against a deterministic process. Against this one, it tells you about five assets.

Technical Errors and Business-Logic Errors

The second distinction is the one that decides what escapes.

Technical errors are wrong as artifacts. Malformed markup. Invalid JSON. Code that does not compile. A URL that does not resolve. A referenced API that does not exist. An image path with no file behind it.

These are the errors everyone worries about, and they are almost the least dangerous ones, because they are machine-detectable. A parser, a compiler, a linter, a link checker, a build step finds them without anyone having to know anything about the client. They are cheap to catch and they tend to fail loudly.

Business-logic errors are wrong only in context.

The price is a real price. It is just not this client's price. The date exists. It is last quarter's. The discount was genuine. The promotion ended in June. The claim is grammatical, on-brand and confident. Legal never approved it. The code runs, passes its tests, and implements a rule the business does not have. The testimonial is well-written and belongs to a different client.

Nothing about any of these is malformed. There is no parser that objects, no linter that fires, no test that fails, because as artifacts they are perfect. They are wrong relative to a fact that lives entirely outside the artifact, in a contract, a calendar, a price list, or someone's head.

Which produces the asymmetry that matters:

Automation catches the technical class. So the class that survives all the way to publication is disproportionately the business-logic class, the expensive one.

That was already true before generated content. What changed is the mix. A human writer who does not know a price leaves it blank or asks; the failure is visible as a hole. A model produces a confident, plausible, correctly formatted number in the right currency, and the hole never appears.

And so the defect that reaches the public is not the sloppy one. It is the one that looked finished.

The Errors Are Different in Kind

It is worth being specific about what generated output gets wrong, because the categories do not overlap much with human error and existing checklists do not cover them.

Confident fabrication. Statistics, dates, citations, product capabilities, case studies, quotes. A human writer who does not know a number leaves a placeholder or asks. A model produces a plausible one.

Subtle factual drift. The generated copy says the plan includes something adjacent to what it actually includes. Nothing is invented outright; the claim has just moved a few degrees from the source, and it will pass any review that is not conducted against the source.

Silent propagation. Feed a model a brief containing one wrong price, and you get forty assets containing that wrong price, each one internally consistent and confidently formatted. Human error tends to be sporadic. Automated error is uniform, which makes it both more damaging and paradoxically harder to notice, since inconsistency is one of the strongest signals a reviewer has.

Voice that is nearly right. Brand voice violations from a model are rarely obvious. They are a register that is slightly too enthusiastic, a claim slightly stronger than legal would allow, a comparison to a competitor that the client has a policy against. These pass unnoticed for months and then arrive as a single uncomfortable client call.

Code that runs. Generated code that works on the happy path and fails on exactly the states that Red-Team the Work, Not the People was about: double submission, expired session, empty result, a name longer than the sample data.

Compliance and rights problems. Claims that need substantiation, imagery whose provenance nobody checked, content that reproduces something it should not.

Notice that most of these are invisible to proofreading, and that several of them are invisible to anyone who does not have the source document open next to the output.

Another Prompt Is Not a Control

The first thing every team tries is to solve this inside the generation step. Add a line to the prompt. Make sure all facts are accurate. Ask the model to check its own work. Run a second pass.

Assume Error, the first article in this series, made the general version of this point: the solution to unreliable output cannot be another instruction telling it to be reliable.

The specific version is worth stating, because it is where a lot of otherwise careful teams are currently placing their trust.

A model reviewing its own output is not an independent control. It shares the failure mode. If it did not know the real number when it wrote the sentence, it does not know the real number when it checks the sentence, and it will frequently confirm the fabrication with the same fluency it used to produce it.

Independence is the whole basis of layered verification. Two controls that fail for the same reason are one control. This is why a second model does not fully solve it either, and why the only verification that closes the loop is one that consults a source outside the generation process: the actual price list, the actual contract, the actual calendar, the actual staging environment.

Automated checks remain enormously valuable here, and they should be aggressive: link checkers, spell checkers, fact-extraction that pulls every number and date out of a draft into a list for a human to tick off against the source. That last one is one of the highest-value things a team can build right now, because it converts an unbounded reading task into a bounded checking task.

But the tick has to be made against something the model did not write.

The Volume Problem Is the Real Problem

Even if every individual generated artifact were slightly better than a human first draft, the arithmetic would still be against you.

No Rate Without a Denominator argued that defect rates are meaningless without volume. Run that in reverse. Hold the defect rate constant and multiply the volume by ten, and you have multiplied the defects by ten.

A team producing forty assets a week with a review process sized for forty assets a week does not get to produce four hundred with the same process and the same headcount. It gets to produce four hundred with a tenth of the verification per asset, which is a different decision than the one anybody consciously made.

This is where most of the current damage is happening, and it is not really an AI problem. It is a capacity problem created by an AI-shaped increase in throughput.

Which puts the decision back where A Blame-Free Culture Is Not an Accountability-Free Culture put it: if the workload makes proper review impossible, that is a capacity problem, and it belongs to whoever set the workload.

What Actually Changes in the Process

The adjustments that work are unglamorous and mostly involve information about provenance.

Mark what was generated. The reviewer needs to know. Not as a judgment, as routing: generated content needs a source check, human content needs a proofread, and those are different activities requiring different amounts of time. A review process that cannot tell them apart is applying the wrong technique to half its work.

Review against sources, not against the page. Reading generated copy for quality tells you whether it is good. It does not tell you whether it is true. Extract every checkable claim and verify each one against a document, and treat that as a separate pass from the editorial one.

Verify at the boundary, not at the artifact. The Trust Chain argued that control strength should follow the boundary being crossed. Generated content makes this more urgent, not less, because the volume tempts teams to review by sampling. Sampling is defensible for internal work. It is not defensible for the four assets going out under the client's name to a hundred thousand people.

Verify the input, hard. Silent propagation means a wrong number in the brief becomes a wrong number in every derived asset. The brief was always worth checking. It is now the highest-leverage artifact in the pipeline.

Track generated content as its own defect class. When Does a Mistake Become a Pattern? argued that patterns sorted by class point at processes. This is a new class with new failure modes, and you will want the data in a year.

The Part That Is Genuinely Good

None of this is an argument against using the tools, and it would be a strange argument coming from anyone who uses them daily.

The same capability that produces confident errors is extremely good at the mechanical half of review. Extracting every claim in a document. Listing every number and where it appears. Flagging inconsistencies between two versions. Generating the adversarial cases from the previous articles (the long names, the empty states, the extreme values), which is tedious work that models do tirelessly and well.

The productive arrangement is roughly the opposite of the intuitive one. Use generation for the parts where being wrong is cheap and obvious, and use the same technology to widen verification, while keeping the final tick against a source in human hands.

That still leaves you with more to check than before. The honest framing is not that AI makes QA harder. It makes the ratio worse, and the ratio was already the problem.

The Shape of the Claim

The argument of this article is narrow and does not require any position on how good the models are.

AI increased the rate at which unverified content can be produced. It did not increase the rate at which anything can be verified.

Any process whose quality depended on the natural slowness of human production has just had that protection removed, and most creative processes depended on it more than they realized.

The first article in this series argued that a resilient process assumes someone will eventually make a mistake. That is now true of a much larger population than the people on the team.

And the new members of that population make mistakes that read beautifully.