August 9, 2026 · Engineering

How builds verify themselves

This article describes the product at publication. See AI Builder and Agent Teams for current capabilities.

0:00 — a build finishes. The agent says it's done, which is a claim about writing code, not about whether the code works. Every user of an AI builder has felt the gap between those two claims at least once: preview opens, third button gets clicked, nothing happens. I've watched a demo room go quiet at exactly that moment. So before any human sees a build, it goes through six minutes-ish of a chain arguing with itself. Here's what that actually looks like, traced through one build we watched go sideways and then get fixed.

0:02 — code review starts. Not the agent that wrote the code rereading its own homework — a separate agent, different prompt, no stake in the build passing. That separation matters more than it sounds like it should. An agent that decided at 2:14pm that a fetch call without error handling was fine will still think so at 2:15pm if you ask it to check its own work. A fresh reviewer told "find what's broken, cite the file" behaves like the grumpy senior engineer you actually want on this duty. On a past build it caught a cart total that silently never updated — `updateTotal` was defined in `Cart.jsx` but never wired to the quantity-change handler, so the function existed and simply never ran. That's the category code review is for: things a compiler shrugs at.

0:04 — security audit. Narrower than it sounds, deliberately — this isn't a pentest, it's a pattern hunt for the handful of mistakes that actually show up in AI-generated code. String-concatenated SQL. Client-side-only validation trusted as if that were the whole story. And the house special: a hardcoded API key, because the agent writing the feature didn't have an environment-variable convention sitting in front of it and reached for the thing that worked. We see that one often enough that it barely counts as a surprise anymore.

0:07 — links and SEO. Unglamorous, and it catches the stuff nobody notices until a customer does: a nav link pointing at /pricing when the page actually generated at /price, a sitemap entry for a page that 404s, a meta description still holding the template placeholder text. None of that breaks the build. All of it quietly tanks the thing most of our users built the site to do — get found, get clicked.

0:09 — accessibility. This is an automated axe-core pass, not a full manual audit, and it's worth being honest about what that trade buys. axe-core catches contrast ratios, missing alt text, unlabeled form inputs, tab-order traps — the mechanical layer, something like 30-40% of what a full WCAG review would flag. It will not catch a screen reader experience that's technically compliant but genuinely confusing to use. We picked automated-only because it runs in seconds and most of what ships through here is marketing sites and small tools, not the kind of application where a partial audit is a real risk to someone.

0:11 — conformance. This layer isn't asking "is this good," it's asking "does this match what was promised." Plan said four pages, build shipped three — conformance is what notices. Plan promised a working contact form, what shipped is a form with no submit action — same layer, same catch. It's the check most directly answerable to the user, because it's measuring against the user's stated intent, not some abstract notion of quality.

0:13 — the in-browser check, and this is where our build actually broke. This layer is the hardest to fake because it doesn't read code, it drives a real browser — clicks, types, waits, checks that the DOM changed the way it should have. The build in question was an idle game, and games get an extra pass here because a game can render pixel-perfect and still be unplayable — the score display can look flawless while completely disconnected from the scoring logic. The verifier played it. Score updated fine. Audio didn't make a sound.

Three rounds, then an escalation

The finding didn't come to us as a bug report — it went straight into a fix pass, and the chain re-verified, up to three rounds inside the build. Round one: the fix touched the mixer initialization, which was already fine, so the audio stayed silent. Round two: a different fix addressed a loading-state edge case that looked adjacent, and — this happens more than you'd expect — introduced a small new problem while not solving the original one. Round three: still silent, and by this point you're usually looking at either something genuinely hard or a false alarm, and this was the hard kind.

So the platform escalated on its own. It queued a follow-up fix run scoped entirely to the surviving finding, working on a clone of the build rather than the build itself — meaning the escalation run could fail without costing us the working version we already had. That run found the actual cause: a mute flag set during an earlier debug pass that had never been flipped back, sitting in a completely different file than the two prior fixes had touched. Cleared the flag, re-verified, passed. Nobody looked at this build until it already worked.

OrderLayerCatches
1Code reviewBroken logic, dead handlers, state bugs
2Security auditInjection surfaces, leaked secrets, unsafe patterns
3Links & SEOBroken links, missing metadata, sitemap/robots.txt correctness
4AccessibilityAutomated axe-core: contrast, labels, keyboard nav
5ConformanceDoes the build contain what the plan promised
6In-browser checkRuns the build for real — clicks, types, watches it respond

What I'd skip next time

A few months before that idle game ran, we tried a softer version of the whole system — verifiers allowed to raise any concern, phrased however. It produced findings like "consider extracting this into a helper" and "this variable name could be clearer," which read as diligence and fixed nothing. Fix passes burned entire rounds polishing prose instead of fixing what was actually broken. We tightened the rule to: name a file, describe a failure, or say nothing. Verifier output dropped by roughly half and almost everything left was actionable. If I were rebuilding this from scratch I'd skip the soft version entirely and go straight to the evidence rule — we didn't need to learn that lesson the expensive way, we just did.

The rule has a real cost, and I won't pretend otherwise: a vague-but-true concern like "this API design will bite someone in six months" now gets dropped on the floor, because a verifier can't pin it to a concrete failure. We've made peace with that trade. A chain that also does architecture review isn't fast enough to run on every single build, and speed is the whole point of running this automatically instead of asking a human to do it.

I'd also skip adding a fourth round, if anyone asks. We tuned the round count against real builds, and the marginal value past round three drops off hard — round one resolves most fixable findings, round two mostly cleans up problems round one introduced, and by round three what's left is either genuinely hard or was never really broken. A fourth round mostly buys you longer wait times for the same outcome.

None of this is free, and none of it is infallible. Six layers plus however many fix rounds it takes adds real time to every build — the difference between finishing in under a minute and finishing in several. We think that's the right trade for anything you're about to put in front of your own customers, but "fast" and "verified" pull in opposite directions, and we picked verified. Verifiers are LLMs too, so they occasionally flag something that isn't actually broken, or miss something that is. The evidence rule and the multi-round loop are hedges against that, not guarantees.

What you get at the end is a record: which layers ran, what they found, what got fixed, and anything left for your own judgment. That record is closer to the real product than the code is — it's the difference between trusting a build because it looks finished and trusting it because something adversarial tried to break it first and failed.

And when something still slips through? Tell the build's chat. The fix becomes a new version alongside the old one, runs the same verification chain, and you can roll back at any time. The loop doesn't assume it's infallible; it assumes it can always run again.
Engineering
ShareXLinkedInFacebookRedditQuoraWhatsAppTelegramEmail
← All posts