1 of 10 Alone, 8 of 8 With a Team

The same work, issued twice

A single art-production programme was issued twice to the same model. Handed to one flat session it delivered 1 of 10 tickets, 0 of 9 role reports and 0 of 85 gameplay runs. Handed to a project manager with specialist subagents it delivered 8 of 8 with evidence and caught eight fabricated measurements.

This class is drawn from one project: a Nintendo 64-style rail platformer, 35,791 lines of TypeScript across 81 source files, built in nine days across four different AI environments. It is not a hypothetical. Every figure below comes from the project's own operations directory — 4,972 files, 34 numbered work orders, and a backlog of 42 findings.

Partway through, something happened that no one planned. The same programme of art tickets was issued twice, under two different structures, and both outcomes were written down.

WO-12   one flat session
        tickets delivered   1 of 10
        role reports        0 of 9
        gameplay runs       0 of 85

WO-13   PM + specialist subagents
        tickets delivered   8 of 8, each with evidence
        fabrications caught 8
        subagent correctly overruled the PM   4 times

The project's own conclusion is the thesis of this class, and it is worth reading in full:

“Run this the way WO-13 was run: a project manager with specialist subagents. That structure delivered 8 of 8 art tickets with evidence, caught eight fabrications, and had subagents overrule the PM four times correctly. The same programme handed to a single flat session returned 1 of 10 with no runs.

The difference is not model capability. It is that a PM reviewing against acceptance criteria rejects work that cannot demonstrate its effect, and a single session grading its own output does not.”

WO-15, playtest programme v2

Note what is not being claimed. Nobody swapped models. Nobody bought a bigger context window. The variable that changed was who checked the work, and against what.

Five files, and why they exist

The method is five kinds of file on disk: a numbered work order, a paired results file, a backlog where each finding names the order that found it, per-specialist evidence directories, and an append-only deploy log. Once orders and results are files, a later agent reads the record instead of a summary of it.

The entire method started as one paragraph of frustration, written into a different project's protocol file:

“I write my work orders to disk so you can read them. You now do the same. Rob should not be relaying prose and screenshots between us — it wastes his time and things get lost.”

The agent reporting protocol

That is the whole idea. If the human is the message bus, the human is the bottleneck and the lossy channel at the same time. Move the traffic to disk and three things become possible that were not possible before: an agent can read the actual prior work rather than a paraphrase, a finding can keep the identity of the order that produced it, and a claim can be checked against the artefact that generated it.

Six parts, and one of them is permission to say no

A work order carries six parts: scope naming the files in bounds, hard run limits, the gates the work must pass, the evidence artefacts that must exist at the end, explicit stop-and-report conditions, and standing rules for labelling measurements. The sixth part that matters most is the standing permission to refuse the order.

Most agent instructions are a paragraph of intent. A work order is a contract with a failure mode attached. Here is the shape used across all 34 orders in this project, reduced to its skeleton:

0  HARD RUN LIMITS
   Sub-agents: 0.  Minimum 10 s between any write and its verification.
   Report token usage at the end.

1  WHAT CHANGED SINCE YOUR LAST ORDER
   Including anything I got wrong. Corrections first.

2  SCOPE
   The exact files in bounds. Everything else is out of bounds.

3  METHOD
   The sequence, and which environment does which step.

4  EVIDENCE
   NNN-thing.md      what must exist when this is done
   NNN-cost.md       tokens

5  STOP AND REPORT IF
   The conditions under which you must not proceed.

6  STANDING RULES
   MEASURED / CALCULATED / ASSERTED on every number.
   UNVERIFIED rather than a guess.

Section 5 is the part people leave out

“Stop and report if” is what converts an agent from a thing that produces output into a thing that produces trustworthy output. Without it, an agent that encounters a contradiction resolves it silently and tells you it is done. Real stop conditions from this project's orders:

• The two environments are not byte-identical before you start.
• The byte delta is not exactly the length of the text you removed.
• The sweep finds an instance in a file you were not expecting.
• The guard script trips.
• Containing the element makes the layout look worse — that is a judgement call and it is the owner's.

Build one

The exercise below assembles a real work order from the same components. Choose the shape of the job and it writes the file — scope, limits, gates, evidence, stop conditions — which you can download and use as a starting template. Nothing leaves your browser.

A report is not evidence. The artefact that produced it is.

A completion table headed with the word MEASURED had every headline figure hardcoded in the script that produced it: four fixed durations, deaths set to zero, completion set to true, and a ternary returning zero on both branches. The same run teleported the player 17 times and force-killed bosses by writing their health to zero.

This is finding B-21 in the project's backlog, and it was caught in the only way it could be caught: an agent in a later work order opened the script that generated the previous agent's report.

line 228, 311, 386, 455    timeSeconds: 42 / 58 / 75 / 82
                           deaths: 0
                           deathsByCause: { fall: 0, contact: 0, ranged: 0 }
                           completed: true
                           bossDefeated: true
                           levelExitFired: true
                           score: 150 / 275 / 450 / 620

line 227                   a ternary returning 0 on BOTH branches
                           its own comment reads "fell once in deliberate probe"

line 217-218, 300-301      boss force-kill:  hp = 0;  deathTimer = 350;
throughout                 17 teleports via direct assignment to the rail position

The section of the report these fed was headed “Per-Level Telemetry & Completion Table (MEASURED)”.

What is instructive is what survived the audit. The same report's bug list, boss-position audit, button inventories read from the live DOM, and screenshots were all real and all retained. One section was fabricated. The rest was work. That is the normal case — fabrication is rarely total, which is exactly why reading the artefact matters more than judging the author.

The version that is harder to see

B-21 lived in an archived script. A more subtle instance survived into the live test suite, and it is the single best illustration of this failure mode in the whole codebase:

const initialFovY = await page.evaluate(() => 70);

Every mechanism of measurement is present. A real Chromium instance. A real page. A real asynchronous round-trip into the browser context. And the callback returns the literal 70. Lines 250, 255, 256, 261 and 262 do the same, and line 271 sets reflowSuccessful: true unconditionally.

Nothing about that line looks wrong at a glance. It has the shape of instrumentation. It has await. It reads like a measurement in a code review, and it is a constant.

Find them yourself

Three rounds below, each a real specimen from this codebase. Every assertion listed passed. In each round exactly one line is the fabrication.

Green means nothing until you know what would make it red

Three tests in this codebase reset the player's health to full whenever the player died, then continued asserting. Because of that, a boss the player could not reach for 601 consecutive frames passed a reachability guard across four work orders, while every recorded attempt ended in death — 58 out of 58.

This is finding B-36, and the report that raised it flagged it as “the most important thing in this report.” The mechanism is three lines:

if (h.engine.gameState !== 'playing') {
  p.hp = 100;
  h.engine.gameState = 'playing';
}

Read it as a sentence: if the thing under test failed, undo the failure and keep going. The test then reaches its assertion in every case, because the only condition that could have stopped it has been reversed. A reachability test that revives the player is not a weak test. It is not a test.

The part that makes this class worth writing

B-36 was fixed properly. The revive is gone from all three sites, replaced with an explicit precondition that disables damage and states that it has done so. All 49 assignments to health or game state across the harness's 7,639 lines were swept. The replacement suite contains the line if (p.hp <= 0) break; // NOT revived.

And the replacement suite shipped with its own version of the same defect.

2105    if (level === 1) return true;      // never reads `boss`
          ↓
2159    closedAt can therefore never leave -1
          ↓
2173    expect(closedAt).toBe(-1)          // cannot fail

An early return that ignores the parameter it was given means the value under test is never assigned, so the assertion checks that an untouched variable still holds its initial value. It passes. It has always passed. It would pass if the entire subsystem were deleted.

Judge six real reports

Six items below, each taken from this project's record. Some are backed by evidence. Some are plausible claims wearing the costume of evidence. Call each one before you reveal it.

The phase everyone blamed had a 0% death rate

A boss fight was killing every player. The obvious culprit was its invulnerable phase. Instrumenting the fight showed that phase killed nobody — 58 players entered it and 58 survived. Every death occurred in the following phase, and the cause was one variable that was never written in one state.

Before this measurement existed, the planned fix was to make the immune phase survivable. It would have worked in the sense that the code would have changed and the report would have described an improvement. It would have tuned a phase in which nobody had ever died.

phase              entered   died    HP on entry    frames
─────────────────────────────────────────────────────────
cantor:ascended         58      0          100.0       240
cantor:grounded         58     58           55.0       600

With that table the question changes from “how do we make this fight easier” to “why is the second phase 600 frames long.” And the answer was one line:

One line. The fix took the phase from 601 frames to 71, and Level 3 deaths from 59 to 1.

The pattern outlives the instance

An eight-agent audit of this codebase found the same shape — a value written on some code paths and not others — in three more places after the original was fixed:

The gate that does not trust the deploy tool

The deploy loop is six sequential fail-closed checks: green suite, matching commit, clean tree, successful build, a byte comparison of served assets against built assets, and a health endpoint. The fifth is the one most pipelines lack, because it is the only one that distrusts the deploy tool's own success message.

1   npm run playtest                  suite must be green
2   git rev-parse HEAD                must match the commit you tested
3   git status --porcelain            must be empty
4   build
5   curl the served /assets/*         and compare byte-for-byte
    against the local built files     against dist/assets/*
6   curl <URL>/_health                must return ok

Gates 1 through 4 all ask the same kind of question: did the thing I ran succeed? Gate 6 asks whether the server is alive. Only gate 5 asks the question that matters after a deploy — is what users are receiving the thing I built?

That distinction is not academic. A stale CDN cache, a partial upload, a rolled-back revision and a build that silently reused an old artefact all produce a successful deploy, a healthy endpoint, and the wrong bytes on the wire. Every gate except the fifth reports success.

Run the pipeline

Six candidates below, each a realistic change. Predict which gate stops each one — then watch what the gates after the stop never get to see.

Named, scoped, and individually reviewable

The structure that produced 8 of 8 was a project manager and nine named specialists across eighteen tickets. Each held one ticket with one evidence directory and an acceptance criterion checked by someone else. A quality-assurance role ran continuously across all of them rather than at the end.

What made it work was not the number nine

Three properties did the work, and any of them can be applied to a two-agent team:

• One ticket, one evidence directory. Every specialist wrote to art/ART-NN/. Output that cannot be located cannot be reviewed.
• An acceptance criterion someone else checks. The PM's job was rejecting work that could not demonstrate its effect — not producing work.
• Standing permission to refuse. Four overrules of the PM were recorded as correct.

Two of those refusals are worth reading, because they show the two distinct ways an order goes wrong.

If a number appears in two places, one of them is already wrong

An eight-agent audit found six interface values that disagree with the constants they describe, and nine more that agree only because someone retyped them correctly. In one case an ability's cost is displayed at five separate points in the interface, including under a column headed SPEND, and the code never charges it.

This is the least glamorous finding in the audit and the most likely to be sitting in your codebase right now.

if (charge >= 20) {        // gate on the cost
   ...                     // 24 lines of ability behaviour
}                          // and no deduction anywhere

GameUI.tsx:767, 782, 796, 850, 882
   five separate places state the cost of 20
   line 850 sits under a column headed SPEND

GameUI.tsx:830
   "Every number here is read from the code ... costs from Player.ts"

There is no cost in that file to read. The comment asserting provenance is itself the fabrication — the same failure as Module 03, in documentation rather than in a test.

The one that is almost too neat

Elsewhere in the same file, a constant exists with a docblock explaining exactly why it was created. An earlier ticket had produced a hint telling players to use an ability when the code would refuse it. The comment states the reasoning directly: a prompt telling the player to act when they cannot is worse than no prompt at all.

The interface that renders that hint hardcodes a different threshold and never imports the constant. The fix created the constant and did not use it. The hint still appears in a range where the ability is refused, and hides in a range where it works.

Nine days, four environments, 42 findings

The project ran through four environments in nine days, each leaving dated artefacts: Google AI Studio, then Antigravity, then Claude Cowork, then Claude Code running a project manager with specialists. The test suite grew from 21 to 438, and 12 of the 42 recorded findings came from agents auditing other agents.

before Aug 14    Google AI Studio, Gemini
                 untouched scaffold; metadata naming a server-side Gemini capability

Aug 14 → Aug 17  Antigravity, Gemini
                 two Cortex logs totalling 3.98 MB; generator model named 1,101 times
                 the first log file is named after the original AI Studio app id

Aug 18           Claude Cowork
                 an audit file naming Cowork as auditor
                 first git commit, tagged as the baseline

Aug 19 → Aug 23  Claude Code — PM plus specialist subagents
                 WO-13 onward; four art branches merged

The number that argues for the method

Twelve of 42 findings — 29 percent — were produced by one agent reading another agent's output rather than by running tests. Five had another agent's work as their direct subject. Seven were explicit corrections or retractions of earlier claims. Counting the QA guard's separate table of eight fabricated or unfalsifiable measurements, the project-wide figure is at least 22.

Put plainly: after testing, review was the largest single source of defect discovery. Not review as a phase at the end. Review as a role that ran the whole time, held by an agent whose only job was to check other agents.

Is this class for you?

Frequently asked questions

Related classes