Getting the verification test right is the biggest time saving in any build

Last year a group of experienced developers were given AI coding tools and set loose on real work in their own repositories. Sixteen of them, 246 real issues, randomly assigned to use the tools or not.

They finished 19 percent slower with the tools than without.

Before they started, they predicted they would be 24 percent faster. After they finished, still slower, they believed they had been about 20 percent faster.

That study is small and it is one of the few properly randomized ones we have. What it measures is not that the tools are useless. It is that the time moves somewhere people cannot feel. It moves into finding out what is broken.

I know exactly where it went, because for months it went into me.

I was the test

When I started building this way I had a weak verification plan. So I became the middleman QA tester.

Every deployment, I was the one clicking every link. Opening every page to see whether it looked right. Typing into every form to find out where the answer landed. Checking the uploads, the downloads, the sign-ins. Looking at images that came back wrong and layouts that broke at one screen width and features that had been reported as finished and were not.

I was the bottleneck, and I was also the only one looking.

So I stopped writing the tests after the build and started writing them before it.

Why writing tests afterwards fails

A test written after the build is written to pass the build. It inherits every assumption the code made, including the wrong ones. It cannot tell you the thing is correct, only that it still does what it did on the day you wrote the test.

A test written before the build is a specification. The build is finished when the tests pass, and not before. Nothing is done because it looks done.

This is not a new idea, but the tooling around it changed in the last year. GitHub open sourced Spec Kit in September 2025, which runs a project as specify, plan, break into tasks, then implement. Amazon shipped Kiro in July 2025, which writes requirements in a form borrowed from requirements engineering: WHEN this condition, THE SYSTEM SHALL do that. Both are built on the same observation. A requirement written that way converts directly into a test case. A requirement written as a paragraph does not.

The practical version is simpler than either. Before you build, write down what has to be true, in a form a machine can check.

The six phases

This is the playbook I run now. Every tool named is free and open source.

Phase 1. Before you build

Write what has to be true in a form a machine can check, not "looks right." Name three edge cases for anything that takes input, before the code exists.

Phase 2. The code

Count the files the build produced. A build can print "compiled successfully" and emit nothing at all. I have watched it happen: the success line was a progress message, the build then died, and no pages were written. The only thing that caught it was going to look for the files.

Scan every dependency for known vulnerabilities. osv-scanner does this from the command line and fails your build on a hit.

Then the one people are not thinking about yet. Confirm every new package actually exists. A 2026 study ran nearly 200,000 code samples through five current models and found they invent package names between 4.6 and 6.1 percent of the time. More usefully, 127 invented names were produced identically by all five models, and 53 of those were still available to register after the researchers disclosed them. An attacker only has to register the name once and wait.

Test the corner cases rather than the happy path. Empty, zero, negative, one over the limit, two people doing it at the same time.

Phase 3. The screens and the journeys

Screenshot every page and compare it against the last version you know was good. Playwright does this with one assertion. It catches the shifted button, the clipped heading, the section that collapses at one screen width. Freeze animations, hide the cursor, mask anything that changes on its own like a timestamp, and run the comparison on the same machine setup that produced the baseline. Skip that last part and the check fails for reasons that have nothing to do with your code, and you stop believing it.

Follow every link. Not read them, follow them. lychee crawls a site and reports the dead ones, and it runs as a build step.

For images and video, fetch the file at its own address, compare its size to the file you shipped, and start playback. A page returns a success code whether or not the video inside it loads. The element renders, the poster frame shows, and nothing plays.

Feed every form field the ugly inputs. Emoji, right to left text, a name ten thousand characters long, a leading space, a string that looks like code. The Big List of Naughty Strings is the standard collection and it is one file. Then check what got stored, not what got displayed. Those are different questions.

Run sign in, upload and download from end to end. These three break the most and get tested the least. Playwright handles all three, including drag and drop uploads and catching a download as it happens.

Phase 4. The qualities

Score page speed on every commit with Lighthouse CI, and set it to fail the build rather than warn. A budget that only warns is a budget nobody enforces.

Check the accessibility rules with axe. And know what it is doing: axe reads the page structure, so it can tell you an image has no description and cannot tell you two elements are sitting on top of each other. Overlap and clipping are invisible to it. That is what the screenshots in phase 3 are for.

Phase 5. The review

Have a second agent review the code, and never the one that wrote it.

This is not a stylistic preference. A model reviewing its own output is measurably more generous than an independent one, and the effect is strongest exactly where you least want it, on changes that are wrong. Hiding who wrote the code reduces the bias more reliably than telling the model to be critical.

Tell the reviewer to disagree and to show evidence for every finding. Research published in August 2026 tested a coder, a reviewer and a critic using structured disagreement, and it beat a five agent setup. The same work named the failure it avoids: agents rubber stamping each other and calling it consensus. More reviewers is not the mechanism. Disagreement is.

Then check the tests themselves can fail. Break something on purpose and see whether the suite notices. If it stays green, the suite is decoration. The formal version of this is mutation testing, which plants small bugs and reports how many your tests caught. It exists because coverage lies: a line can be executed by a test that asserts nothing about it, and the coverage report counts it either way.

Phase 6. Release

Confirm the live site returns the new version. Ask it for its build identifier and poll until it changes. Do this on every address the thing answers on, not just the one you usually check. I have seen a check pass against the platform address while the address the client actually uses was still serving the old build.

Bust your own cache before you believe the answer. My own check once read a stale response twice and nearly reported a failed deploy on a build that had shipped correctly.

The part that is not on other checklists

Next to every check, write one line saying what that check cannot catch.

I went looking for prior art on this. The Front End Checklist has 385 items. The security testing guide from OWASP has 129 test procedures. Google's own launch checklist from its site reliability book has around 50. Not one of them says, per item, what a passing result fails to prove.

That column is where the real failures live. Some of mine:

A scan came back clean across 94 files. A complete build produces 190. The result was clean over half the work, and the number that would have exposed it was the one nobody printed.

A search for a brand mark hunted for a specific color in a specific corner of the page. The name it was looking for was sitting in gray text in a footer, outside that corner, and two versions in a row reported zero.

A check for the absence of some text returned zero for all fourteen strings. It returned zero because a program it depended on was not installed on that machine, so the check never ran. Fourteen zeros was the only tell, and only because it was too clean to believe.

The rules that came out of those:

Assert the count before you trust the result. A clean answer over an incomplete set is the most dangerous shape a green can take.

Every check for the absence of something carries a known example that must come back positive. A negative from an instrument you have never seen produce a positive is not evidence.

Name what you actually looked at. Reading the code, running a test, rendering the page and looking at the live site are four different things, and when two disagree, re-run the check before you believe the report.

One number to stop repeating

You will see it claimed that a bug costs ten times more to fix in testing and a hundred times more in production. The table everyone reproduces is credited to an IBM institute, and it has no traceable published source behind it. The citation trail runs into a textbook citing internal training notes, and stops.

The real research is softer and more useful. Boehm and Basili, in 2001, put post release fixes at around a hundred times for large critical systems and around five to one for small ones. The direction is right. The multiplier everyone quotes is folklore.

Say five to one and be correct, or skip the number and describe the failure instead.

What to do on Monday

Pick one deployment you are nervous about.

Before you touch the code, write the list of what must be true when it is done, in a form something other than you can check.

Then go down that list and write one line per check saying what it cannot catch.

That second pass takes about twenty minutes and it is the one that changes what you build.

Automate. Ascend.