Verification Is the Bottleneck Now

Google and Kaggle published a whitepaper this month, The New SDLC With Vibe Coding by Addy Osmani, Shubham Saboo, and Sokratis Kartakis. It is about 50 pages and, despite the title, it is not really a vibe coding paper. It is an argument about verification.

This post is our read on it from the delivery side: what the framework gets right, where it lands differently when the artifact is client infrastructure rather than a weekend prototype, and what we have actually changed in how we work.

Note: The statistics below are the paper’s, current as of mid-2026, not our own measurements. We cite them because they match what we see, not because we gathered them.

The central claim

Generation stopped being the constraint. Verification became it.

The supporting numbers are uncomfortable. Roughly 85% of professional developers now use coding agents regularly and 51% use them daily, with something like 41% of new code AI-generated. But only 48% consistently verify AI-assisted code before committing it, and 38% report that reviewing AI-written logic is harder than reviewing a human’s.

The paper cites a METR study finding developers were 19% slower on some tasks once review time was counted honestly. That result gets dismissed a lot. We think it should be taken seriously, because it describes a specific failure mode we have watched happen: the code arrives faster, and the reviewing capacity to absorb it does not exist, so the work queues up somewhere less visible than the ticket board.

Agent = Model + Harness

The paper’s most useful idea is a decomposition. An agent is a model plus a harness, and the split is roughly 10/90. The model is 10%. The harness (instructions, tools, MCP servers, sandboxes, orchestration, guardrails, observability) is the other 90%.

The conclusion follows immediately: most agent failures, examined honestly, are configuration failures. Not model limitations. The paper cites a team that moved a coding agent from outside the top 30 to the top 5 on a benchmark through harness changes alone.

This matches our experience closely enough that we now treat “the model isn’t good enough” as a claim requiring evidence. Usually the tools are too broad, the instructions are absent, or nothing tells the agent which of its options to reach for. We wrote about a concrete instance of this last month: the Agent Toolkit for AWS ships 40+ skills that agents will simply never load without a seventeen-line rules file. That is the 10/90 split in miniature.

The spectrum, not the binary

The paper frames vibe coding and agentic engineering as ends of a continuum rather than opposing camps:

1. Vibe coding. Casual prompts, minimal review, disposable output.
2. Structured AI-assisted. Some conventions, human review, conventional testing.
3. Agentic engineering. Formal specifications, automated evals, production systems.

What moves you along the spectrum is verification rigor. Nothing else. Not model choice, not tooling spend, not how much of your code is AI-generated.

The distinction between tests and evals is worth internalizing: tests check deterministic behavior; evals assess non-deterministic trajectories. A test tells you the output was right. An eval tells you the reasoning that produced it was sound, which is what you need when the same prompt can take a different path tomorrow.

The paper also borrows Anthropic’s generator-evaluator finding: models skew positive when grading their own output, so use a separate skeptical evaluator. Cheap to fix, and the single change we would recommend to a team that has adopted agents but not eval discipline.

Where the compression is uneven

The paper’s most practically useful section argues that the SDLC does not compress uniformly:

Implementation collapses. Weeks to hours, in the good cases.

Requirements and architecture do not. These are judgment work, bounded by how fast a human can decide something with real consequences.

Testing and verification do not, and now carry more load. More code arriving means more code to verify, with the same people verifying it.

Maintenance is the sleeper win. The paper’s framing, and we agree. Agents can now touch legacy code that was previously too risky to modify, because the cost of a careful, well-scoped change to an unfamiliar codebase dropped enough to make the work tractable. For our clients this is the most under-appreciated shift of the past year: the modernization work everyone deferred is now affordable.

The economics invert too. Vibe coding is cheap up front and expensive operationally, in token burn, maintenance tax, and security cleanup. The paper puts the crossover at agentic engineering costing 3 to 10x less per feature at scale.

What this means when the artifact is infrastructure

Here is where our read diverges a little. The paper is written mostly with application code in mind. When the output is a CloudFormation template or a Terraform module, three things are different:

The blast radius is larger and the feedback is slower. A bad function fails in CI. A bad IAM policy fails quietly, in production, six weeks later, during an audit.

Verification is more automatable, which is good news. Infrastructure has real static analysis: cfn-lint, cfn-guard, policy-as-code, change sets, plan diffs. Application code mostly does not have an equivalent. The verification bottleneck the paper describes is genuinely easier to relieve on the IaC side, and teams should exploit that.

Rollback is not symmetric. You can revert a commit. You cannot always revert a stateful resource replacement. This is the reason we hold the line from our February post: agent recommends, human approves, agent executes.

What we actually changed

Concretely, across our custom application work and client IaC:

• Rules files are a first-class artifact. Checked in, reviewed, versioned like code. Most of the leverage lives here.
• Bounded tools, not broad access. Specific audited operations rather than “AWS access.”
• Validation in the pipeline, not in the reviewer’s head. cfn-lint and policy checks run before a human sees the diff, so review time goes to architecture rather than syntax.
• A separate evaluator. Never the generating agent grading itself.
• Reviewing capacity planned as capacity. If generation triples and review does not move, you have not sped up. You have relocated the queue.

Closing

The paper’s real contribution is naming the constraint correctly. Teams keep investing in generation speed while the actual limit sits downstream in verification, and then wonder why delivery did not improve.

If you are working out what agentic engineering looks like on a team that ships real infrastructure, we’d be glad to help.