InsightPractitioner Truth

The Vibe Coding Crisis: What Actually Fails in Production

June 8, 2026·8 min read·atin-agarwal.com
← Back to Blog

The dangerous thing about vibe coding production failures is not that AI writes bad code. It is that AI writes code that looks right. I learned this the uncomfortable way: I pointed our own AI code quality scanner at its own codebase — a scanner that was itself largely built with AI coding tools — and it returned 406 findings in 35.2 seconds. Grade D. Seventy-five security issues, twelve of them critical. This was code I had described, directed an AI to write, reviewed at human speed, and considered ready for production. The tool built to find these patterns contained the very patterns it was built to find. That recursive loop is the whole crisis in one sentence.

I told the story of that scan when it happened, in 406 Findings in 35.2 Seconds. This piece is the other half — not the headline number, but the mechanism underneath it. Why does AI-generated code fail in production specifically? Why does code review miss it? And why is the answer not "write better prompts" but "build the quality infrastructure that nobody has built yet"? The numbers behind vibe coding are not subtle. GitHub's Octoverse 2025 report found that more than 46 percent of all code in its repositories is now AI-generated — 61 percent for Java developers. Ninety-two percent of US developers use AI coding tools daily. The adoption curve is exponential. The quality-infrastructure curve is flat. The widening gap between them is where production failures live.

AI Code Does Not Fail Randomly

The first thing to understand is that AI-generated code does not fail the way human code fails. Human developers make typos, off-by-one errors, copy-paste mistakes. AI fails in systematic, predictable, categorisable patterns — and that distinction is the entire reason the problem is tractable. If AI code failed randomly, you would need a human to read every line, which would destroy the speed advantage that makes the tools worth using. But it does not fail randomly. From building and operating code-scanning pipelines across real codebases, we identified ten specific anti-patterns, each with a name, a severity, and a root cause in how large language models generate code.

Two of the ten are rated critical, and both are dangerous for the same reason: they are semantic, not structural. The first is the auth logic gap — the AI generates authentication that has the shape of being correct. Middleware is in place. JWT tokens are verified. Role checks exist. But a role check confirms a user has a role without confirming they have the right role for the resource they are touching. A JWT verification checks the signature but not the expiration. The code follows the form of correct authentication without achieving the substance. The second is injection via interpolation — building SQL queries or shell commands by concatenating strings instead of using parameterised queries, because string interpolation produces more readable code, and readability is what the training data rewards.

The other eight are just as systematic. Hallucinated APIs — calls to methods that were never part of any library version, that compile in dynamically typed languages and fail only at runtime in production. Deprecated cryptography — MD5 and SHA-1 reproduced from tutorials, broken since 2004 but still dominant in the training corpus. Permissive defaults, secrets hardcoded in source, missing input validation, insecure deserialization, race conditions, error messages that leak your stack to attackers. Cluster the root causes and four categories emerge: training-data bias, context limitation, happy-path optimisation, and adversarial blindness — the inability to reason about what an attacker would do, because that requires a theory of mind that statistical prediction does not provide.

Why Code Review Does Not Save You

The obvious objection is that code review should catch all of this. It does not, for three reasons that compound. First, the error profile is wrong: reviewers spent three decades training their instincts on human mistakes, and AI mistakes do not trigger those instincts. Second, the surface quality is misleading. AI code follows conventions, is consistently formatted, includes helpful comments, and handles the common cases cleanly. It looks like work from a competent developer who pays attention to style — so the alarms that should fire never do. A study published at IEEE ISSRE 2025, analysing over 500,000 samples, found AI code is structurally simpler and more repetitive than human code. The simplicity is camouflage. It took the most common path through the training data, and the most common path is not the most secure path.

Third, and most brutally, there is a volume problem. A developer using AI tools produces ten, twenty, fifty times more code per day than one writing by hand. Review capacity has not scaled to match. The team that reviewed three pull requests a day now faces thirty — each one cleaner-looking and more dangerous than the last. The review bottleneck is not laziness; it is arithmetic. CodeRabbit's 2025 analysis quantified the paradox precisely: AI-authored pull requests contain 1.7 times more issues overall, 1.75 times more logic and correctness errors, and 1.4 times more critical issues than human-authored ones — yet they slip through review more easily because they read so well. Apiiro tracked the consequence at scale: AI-generated code introducing over 10,000 new security findings per month, a ten-times spike in just six months.

What the Self-Scan Actually Showed

Back to my own code. Broken out by category, the scan was a portrait of exactly these dynamics. Security: 75 findings, grade F — injection patterns, missing validation, permissive defaults. A tool built to flag missing input validation lacked input validation in its own API endpoints. Test coverage: 53 findings, grade D — only three of forty testable files had any tests at all. Eight percent. AI tools generate application code readily and tests far less often, and the gap between "code that works" and "code that is tested" is one of the most dangerous patterns in vibe coding. Architecture: 27 findings. Code smells: 244. And then the most revealing number — 36 cross-category patterns, including eleven instances of untested vulnerable code: security findings sitting in files with no test coverage whatsoever. A vulnerability is dangerous. A vulnerability in untested, architecturally fragile code is a crisis waiting for a trigger.

The finding rate landed at roughly 26 findings per thousand lines of code. Industry benchmarks for human-written code before testing run 15 to 50 defects per thousand lines. So the AI-generated code that I had reviewed and considered production-ready was shipping at the quality level of human code before any testing or review had happened — while wearing a surface finish that convinced me it was better than that. The AI tool did not improve on the pre-testing baseline. It produced at the pre-testing baseline, with a coat of paint that fooled an experienced reviewer. That is the trap, stated as a number. And it matters far beyond one codebase: a solo operator running ten ventures' worth of AI-generated code without scanning is shipping ten ventures' worth of vulnerabilities at machine speed. This is the same wall, from a different direction, that I describe in why most enterprises can't get agents to production — the demo works, then reality arrives.

The Fix Is Infrastructure, Not Discipline

None of this is an argument against AI-generated code. The productivity is real and transformative, and the one-person operating model I build from depends on it. Abandoning AI code is not the answer and nobody serious is proposing it. The answer is AI-generated quality infrastructure. AI writes code fast; AI can check code fast. Because the failures are systematic — the ten anti-patterns, not random noise — they can be detected automatically, at the same machine speed the code was generated at. You need AI to check AI. Human review keeps its place for design intent and business logic; the systematic patterns belong to the scanner.

This is why I treat AI code quality as a dependency-layer play, not a feature. Every agent, every pipeline, every system in the agent economy increasingly runs on AI-generated code. If that code has a systematic quality problem — and the evidence says it does — then the whole economy has one. Quality scanning for AI code is to the agent economy what SSL/TLS was to e-commerce: trust infrastructure without which the market cannot scale. In 1994, online merchants were already selling, but customers were typing credit-card numbers into plaintext forms. The market was functional. It was not trustworthy. SSL did not create e-commerce; it let e-commerce scale. AI code quality serves the same function — and the dependency layer is where value concentrates in every technology era.

I will not pretend the timeline is comfortable. The trajectory of those two curves — adoption rising steeply, quality infrastructure barely above the baseline — produces an inevitable intersection. I have put a date on it: one of my falsifiable predictions for the agent economy is that by 2028, at least three major breaches causing over $100 million in damages each will be publicly attributed to vulnerabilities in AI-generated code, triggering regulatory response. The early tremors are already here — MCP vulnerabilities used to hijack agents and exfiltrate source code, prompt injection through hidden commands in web content. No breach has crossed the threshold yet. The question is whether the quality infrastructure is in place before the damage or after it.

The vibe is running ahead of the verification. That is the crisis in one line. Chapter 7 of The AI Agent Economy lays out the full taxonomy — all ten anti-patterns with their CWE mappings and root causes, the complete self-scan breakdown, and the market case for why quality infrastructure is the next dependency-layer opportunity. The short version is the one the scanner gave me in 35.2 seconds: the code looks right, it ships, and "looks right" is not "is safe." The tools to know the difference exist. Using them is a choice — and right now, most teams are not making it.

vibe-codingai-generated-codecode-qualityproduction-aisecuritydependency-layer

Frequently asked

Why don't vibe coding production failures show up in code review?

Because AI-generated code is structurally correct. It follows naming conventions, is consistently formatted, includes comments, and handles the common cases cleanly. Human code review was trained over three decades to catch human error patterns — typos, off-by-one errors, copy-paste mistakes. AI errors are semantic, not structural: an auth check that verifies a role but not the right role, a JWT verification that checks the signature but not the expiration. The surface signals reviewers use to assess quality all pass inspection. CodeRabbit's 2025 analysis found AI-authored pull requests contain 1.7 times more issues than human-authored ones — yet they are harder to catch precisely because the code looks cleaner.

What are the most dangerous AI code anti-patterns?

The two rated critical are auth logic gaps (the code has the shape of correct authentication — middleware, JWT verification, role checks — without the substance) and injection via string interpolation (SQL queries or shell commands built by concatenating strings instead of using parameterised queries). Both are dangerous because they are semantic rather than structural: a reviewer sees the middleware and the role check and moves on. Other systematic patterns include hallucinated APIs, deprecated cryptography defaulting to MD5 or SHA-1, hardcoded secrets in source, and missing input validation.

Is AI-generated code less secure than human-written code?

Per snippet, yes — and the gap is measurable. A Stanford and MIT joint study analysing over two million code snippets found 14.3 percent of AI-generated snippets contain at least one security vulnerability, against 9.1 percent for human-written code — a 57 percent higher rate. Veracode, testing over a hundred AI models, found 45 percent of AI-generated samples failed security tests. The per-snippet rate becomes a per-codebase crisis at the volumes the industry now produces — Apiiro tracked a ten-times spike in AI-code security findings in just six months.

How do you actually catch these failures before they ship?

You need AI to check AI. Human review remains essential for architectural decisions, design intent, and business-logic validation. But for the systematic vulnerability patterns AI code generation produces at scale, you need automated scanning designed specifically for the error profile of machine-generated code, operating at machine speed. The patterns are predictable and categorisable, which means they can be scanned for automatically — the quality gap is an adoption gap, not a capability gap. The tools exist. The practice of using them does not.

Will better AI models fix the vibe coding crisis?

Not entirely. Better models produce fewer of these patterns — GPT-4 produces fewer than GPT-3.5, and each generation improves. But the fundamental dynamic is structural: language models generate code by predicting the most likely next token, and the training corpus is dominated by tutorials and demonstration code written for clarity, not security. Optimising for likely code rather than secure code is inherent to the approach. The patterns will persist. They need to be scanned for, not wished away.

Get predictions before they're published

Weekly AI Agent Economy insights. Book chapters delivered to subscribers first.

Delivered via Substack. No spam. Unsubscribe anytime.

Or subscribe via LinkedIn:

Subscribe on LinkedIn