QA practice10 min read

Security and Quality Are Like an Onion

Defense in depth applied to creative quality control, and the full index to a thirteen-article series on QA, severity, escapes and adversarial review.

There is a scene in Shrek (2001) where Shrek tries to explain to Donkey that there is more to ogres than people assume.

He picks up an onion.

Donkey works through the possibilities enthusiastically and gets all of them wrong. Onions stink. They make you cry. Leave one in the sun long enough and it sprouts little white hairs.

Shrek, losing patience, gives him the answer: layers. Onions have layers. Ogres have layers.

"Ogres are like onions."

And then Donkey raises the objection that makes the scene funny, and that turns out to be the more interesting one: plenty of things have layers. Cake has layers. Everybody likes cake.

Shrek refuses to switch metaphors, and he is right to. Not because cake is worse than onion, but because Donkey has quietly identified the thing that actually matters, and it is not the number of layers.

This is the index to a thirteen-article series about quality control in creative work, written from a security background. The onion is where it starts, because the onion is not a joke in this field. It is the model.

Layers Are the Whole Idea

Security has spent decades arriving at an unglamorous conclusion: no single control works.

Authentication fails, because credentials get phished. Input validation fails, because somebody forgot one endpoint. The firewall fails, because the threat arrived through a browser. Encryption fails, because the key was in the repository. Monitoring fails, because the alert went to an inbox nobody reads.

The response was never to find the one control that does not fail. It was to stop expecting any of them to.

This is called defense in depth, and it is drawn, almost universally, as concentric rings. The onion, unironically, is the standard diagram.

The design assumption underneath is worth stating precisely, because it is the assumption the entire series rests on:

Every layer is expected to fail. The system is built to survive that.

Which is a strange thing to say out loud in a creative agency, where the operating assumption is usually the reverse: that competent people, doing their jobs properly, will produce correct work. And they usually will. That is not the problem. The problem is what the process does on the day they do not.

Donkey's Objection

Cake has layers.

He is not wrong, and this is exactly where most quality processes fail, so it is worth taking seriously.

Plenty of organizations have layers. There is a self-review. There is a peer review. There is an account manager sign-off. There is a client approval. On a slide, that is four layers, and it looks like defense in depth.

It usually is not. It is cake.

A cake's layers are stacked. They sit on each other, they were baked in the same oven, and if you drop the cake they all hit the floor together. An onion's layers each enclose the whole thing independently. You have to get through one to reach the next, and cutting the outer one does not open the inner ones.

The difference in a quality process is the same, and it comes down to a single property:

Layers only count if they fail for different reasons.

The peer reviewer who attended the same kickoff, read the same brief, and absorbed the same wrong price from the same deck is not a second layer. They are a second copy of the first layer. Both will fail simultaneously, for one reason, and the organization will be surprised.

Four dependent checks are one check with extra meetings. Two genuinely independent ones are worth more than all of them.

That is the test to apply to any review step you already have: if this fails, will it fail for the same reason the previous one did?

Why Quality Needs This More Than It Used To

Creative work now has a source of defects that did not exist five years ago, and it is not careless people.

Generated content is produced by sampling from a probability distribution. It is not deterministic. The same prompt can produce different output, and nothing in the process consults a price list or a calendar. What comes out is optimized to be a plausible continuation, and plausibility correlates with truth without being constrained by it.

Which means the output is fluent, well-formatted, confident, and occasionally wrong in a way that no proofreader can see, because there is nothing wrong with it as an artifact. It is only wrong relative to a fact that lives somewhere else.

The old protection against this was the natural slowness of human production. That protection is gone. Volume went up by an order of magnitude and verification capacity did not move.

An unlayered process was survivable when the work was slow. It is not now.

The Series

Thirteen articles, in order. Each one is a layer.

The premise: why perfection is not a design input

1. Assume Error: What Creative QA Can Learn From Cybersecurity People make mistakes, automated pipelines propagate them faithfully, and LLMs produce fluent output that is simply wrong. The goal is not to demand perfection but to design a process that does not depend on it. Creating a defect and allowing a defect to escape are not the same event.

2. Not Every Typo Is Low Severity Security has CVSS, a published scale that makes a vendor, a researcher and a regulator read the same number the same way. Creative QA has adjectives, assigned by whoever is talking. Worse, the three parties who evaluate a creative defect score it differently and sometimes in opposite directions. Where the five dimensions of impact come from.

3. The Trust Chain: Why Creative Errors Get More Expensive as They Travel Creator, team, agency, client, the client's customer, the public. The defect does not change as it crosses those boundaries; what changes is who absorbs it. Past the client, cost stops being measured in effort. Past the customer, it stops having a repair time.

The practice: how to find things, and what to do when you do

4. Red-Team the Work, Not the People Good security testing and good QA start from the same question: what are we assuming will always be true? Then you systematically make those assumptions false: in a checkout flow, and in a card that has only ever rendered short names. Adversarial review, visual fuzzing, abuse cases, and the cultural boundary that keeps the practice alive.

5. A Blame-Free Culture Is Not an Accountability-Free Culture One incorrect price has at least five different causes, and only one of them is a performance problem. You are not accountable for being fallible. You are accountable for the controls you agreed to run.

6. When Does a Mistake Become a Pattern? Two defects are not a pattern because they are two defects. They are a pattern when they share a class, a window fixed in advance, and a denominator. A recurrence metric is an alert, not a verdict.

The measurement: which numbers are real, and which ones corrode

7. Stop Counting Mistakes. Start Measuring Escapes. Team A found 100 defects internally and five escaped. Team B found 10 and six escaped. Everyone reacts to the 100. The number that should alarm you is 63%.

8. No Rate Without a Denominator Jane produced 10,000 assets and had 15 findings. John produced 10 and had 2. The raw counts say Jane has a problem. The rates say the opposite by a factor of 130, and John's rate is not really a rate at all.

9. Your Best Performer Should Be a Laboratory, Not a Weapon The difference is almost never talent. It is a private, undocumented control they invented after being burned once, that takes ninety seconds and lives entirely in their head. Extract it or keep paying for it forever.

10. Metrics Can Destroy the Culture They Were Designed to Improve Nobody deletes findings. It fails through a hundred small, individually defensible judgment calls, and the dashboard improves the entire time. Accountability without safeguards eventually becomes blame with better metrics.

The framework: the new surface, the model, and the ritual that keeps it alive

11. AI Made QA More Important, Not Less Why generated output errs at all, why the resulting defect often cannot be regression-tested, and the distinction that decides what escapes: technical errors are wrong as artifacts and machines catch them. Business-logic errors are perfect artifacts that are wrong in context, and nothing automated will ever object.

12. From QA Checklist to Creative Risk Management CQIS, the Creative Quality Impact Score. Five dimensions scored 0 to 2 for potential impact, five escape levels for how far it got, and the rule that the two are never collapsed into one number.

13. A Critical Defect Caught Internally Is a Win Two things happened, and only one is interesting. A Critical finding caught before the client is a near miss, the cheapest data a quality process will ever generate, and the whole system runs on what happens in the ten minutes after somebody says "I found something."

Three Sentences

If the series compresses to anything, it is these.

Assume error.

Red-team the work, not the people.

You do not want a team that makes no mistakes. You want a team that is exceptionally good at catching them.

Back to the Onion

The reason the metaphor holds is not that quality work is deep, or complicated, or misunderstood.

It is that a process built on one excellent layer is a process with a single point of failure, staffed by human beings and, increasingly, by systems that are non-deterministic by construction.

Layers are not bureaucracy. Independent layers are the only known defense against the fact that any given control will eventually be having a bad day.

And Donkey was right about one thing. Everybody does like cake better.

Cake just does not survive being dropped.


Sources for the film reference: Shrek (2001) quotes, IMDb and WikiShrek: Onions.