Seoul Labs

Photo by Zulfugar Karimov on Unsplash

A Carnegie Mellon University benchmark found that 77% of code generated by an AI agent running on Claude Sonnet 4 passed functional tests. Only a small fraction of that same code passed security tests. And the developers who produced it rated their confidence higher than developers who wrote equivalent code by hand. Every vendor dashboard and satisfaction survey tracks completion rate as the signal that matters, not correctness rate, so the gap between what shipped and what was safe never appears in the metrics teams actually review. This post maps where that confidence gap lives, why senior engineers are not exempt from it, and what a governance model looks like that covers the part of the iceberg the standard frameworks miss.

The surface story in 2026 is adoption. Widely cited figures from multiple industry surveys suggest AI tool usage may exceed 80% across developer populations, the kind of number that ends budget conversations and starts mandate conversations. Developers outside traditional engineering roles are shipping apps. Analyst projections suggest a significant share of new enterprise software will involve vibe coding patterns by 2028, and the ecosystem is building toward that. The less comfortable story, sitting right underneath that adoption curve, is that productivity gains are concentrated in exactly the work that was already cheap to do. The expensive work, the integration layer, the auth flow, the edge case in the payment processor, is getting slower and less safe at the same time. Teams are shipping faster on routine work and slower on the work that pays the bills, and senior engineers cannot tell which is which while it is happening.

This is not a case against AI tooling. Claude Code, Cursor, and the generation of agentic coding tools that matured through 2025 and into 2026 are genuinely useful in specific contexts. The problem is that the feedback loop inside those tools is optimized for completion, not correctness. A green checkmark from an LLM that just finished scaffolding an Express 4.x API does not mean the session headers are set correctly. It means the code ran.

Where the Confidence Gap Actually Lives

The CMU study design is worth understanding because it shapes how you interpret the number. Developers using AI assistance were given tasks, then asked to rate their confidence in the security of their output. Developers without AI assistance did the same. The AI group rated their output higher. The automated security analysis rated it lower. That gap between self-reported confidence and measurable outcome is not a personality quirk. It is a structural property of how these tools present their output.

When Claude Code finishes a function, the interface resolves. The spinner stops. Something that looks complete appears in the editor. The cognitive signal you receive is completion, not validation. Humans are already bad at distinguishing those two states under normal conditions. With an AI tool narrating the process and producing syntactically correct output, the completion signal is louder than usual, so confidence rises. The tool performed. What it performed is a separate question entirely, and the interface design of every major coding assistant on the market today makes that second question feel optional.

The scenario that keeps surfacing in practice: a developer uses an agentic session to build an API endpoint. The endpoint works. The tests pass because the AI also wrote the tests, and the tests were written to match the implementation rather than to probe its boundaries. The code goes to review. The reviewer is also using AI tools, is moving fast, and the code looks idiomatic. It gets merged. Three weeks later a penetration tester or a production incident finds the input validation gap that was never part of the original spec and never tested.

AI generated input handler that passed its own tests
How AI-Assisted Code Ships Without Security Validation: The Feedback Loop

  
How AI-Assisted Code Ships Without Security Validation: The Feedback Loop
  


    

      
1
      

        Developer prompts AI to build API endpoint
        
Agentic session starts, scaffolding begins automatically
      
    

    

      ↓
    

    

      
2
      

        AI writes code AND writes the tests
        
Tests match the implementation, not the security boundary
      
    

    

      ↓
    

    

      
3
      

        Spinner stops, interface resolves: "Done"
        
Completion signal fires. Developer confidence rises. Second question (was it safe?) feels optional.
      
    

    

      ↓
    

    

      
4
      

        Reviewer (also using AI) approves quickly
        
Code looks idiomatic. No red flags surface. PR merged.
      
    

    

      ↓
    

    

      
5
      

        3 weeks later: pen tester finds input validation gap
        
Never in the original spec. Never tested. Ships to production undetected.
      
    

  
Source: Article analysis of vibe coding workflow patterns, 2025 to 2026

def create_user(data: dict):
    db.execute(
        f"INSERT INTO users (email, role) VALUES ('{data['email']}', '{data.get('role', 'viewer')}')"
    )
    return {"status": "created"}

That block is not exotic. It is the kind of output a fast-moving agentic session produces when the prompt does not explicitly constrain the result to use parameterized queries, and the developer reviewing it is pattern matching on shape rather than reading for semantics. The test the AI wrote for it passed the email field, got back a 200, and called it done.

The confidence gap is not a documentation problem or a training problem. It is an interface design problem that the major tooling vendors have not addressed because addressing it would slow the perceived completion rate, which is the metric users report in satisfaction surveys. Until completion rate and correctness rate are measured separately and shown side by side, the gap will keep growing. Every team shipping AI-generated code without a separate security gate is running that experiment on their own production infrastructure.

Why Senior Engineers Are Not Immune

The productivity story for senior engineers is more honest than the story for junior developers, but it has its own distortion. Senior engineers using AI tooling do show real throughput gains on prototyping, on scaffolding, on the parts of a task where domain knowledge matters most and mechanical production matters least. A principal engineer who spends two hours on architecture and twenty minutes on boilerplate instead of ninety minutes on boilerplate is getting something real from the tool.

The problem is calibration drift. After several months of working with an agentic coding assistant, the review patterns that senior engineers developed over years start to atrophy slightly in the areas the tool handles most often. The instinct to read every import statement, to check the version pinning in a requirements file, to notice that the AI reached for an older API pattern because that pattern dominated its training data: those instincts require active exercise to stay sharp. When the tool is handling the mechanical layer, the mechanical layer gets less attention from the human, and that attention deficit accumulates across hundreds of merged pull requests before anyone names it.

A requirements.txt produced by an agentic session in mid 2026


Note the version floor, not a ceiling, on packages with known CVEs in older minor versions
cryptography>=3.4.8
requests>=2.25.0
pyjwt>=2.0.0

That requirements file is not wrong in a way that fails a lint check. It is wrong in a way that requires knowing the CVE history of the cryptography package across minor versions, knowing that a floor constraint without a ceiling is a maintenance liability, and caring enough about that to override what the tool produced. A senior engineer who has been moving fast for six months may still know all of that. The live question is whether the review habit that would catch it is still reflexive or has quietly become optional.

Industry analyst projections warning about dramatic defect increases in vibe-coded software without governance are mostly framed as warnings about developers outside engineering building apps without oversight. That is a real risk. The quieter version of the same risk is the senior engineer whose review depth has shifted because the tool has made fast review feel sufficient. Governance frameworks that only target the non-specialist use case are going to miss that entirely.

AI tooling does not uniformly degrade senior engineering judgment. But it creates conditions where degradation can happen slowly and invisibly, which is exactly the kind of problem that shows up in post-mortems rather than sprint reviews. A governance model that does not account for calibration drift in experienced engineers is covering the visible part of the iceberg and leaving the rest.

What Shipping Faster Actually Means in 2026

The teams that report the clearest productivity gains from AI coding tools share a pattern that is easy to miss: they are shipping faster on the work that was already well understood. The CRUD endpoint. The data transformation script. The test fixture. The migration boilerplate. That is real acceleration and it is not trivial. Reducing friction on the mechanical layer frees up cognitive capacity for design decisions, and that has compounding value over time.

The trap is treating that acceleration as uniform. The codebase does not know which parts of it were written in a fast agentic session and which parts were written slowly and carefully. It just runs, until it does not. The places where it stops running are disproportionately the places where the fast session made an assumption the developer did not notice and the reviewer did not catch. That clustering is the mechanism behind the analyst defect projections, and it is more tractable than the headline implies once you see it clearly.

CI pipeline configuration from a team that added AI review gates


after a production incident traced to unreviewed AI generated auth logic
steps:
 , name: semgrep-security-scan
    run: semgrep --config=p/owasp-top-ten --error .
 , name: ai-code-flag-review
    run: |
      git diff origin/main --name-only | xargs grep -l "# generated" \
        | xargs python scripts/require_manual_sign_off.py
 , name: dependency-audit
    run: pip-audit --requirement requirements.txt --fail-on-vuln

That pipeline reflects a pattern several engineering teams arrived at independently after incidents: gate the AI-generated code differently than the handwritten code. Not because the AI is always wrong, but because the review instincts that catch problems in handwritten code do not automatically transfer when the code looks complete and idiomatic before the human reads it. The gate is not a judgment about the tool. It is a correction for a known human cognitive bias that the tool triggers reliably.

The pattern that has not resolved yet as of mid 2026 is this: the tooling vendors and the governance vendors are building in parallel without a shared model of where the risk actually lives. One side is optimizing completion rate. The other is selling audit trails. Neither is directly addressing the confidence calibration problem that sits between them. The defects are not randomly distributed across the codebase. They cluster in exactly the places where the AI confidently handled something the developer did not fully specify, and the developer trusted the completion signal instead of the specification. Fix the signal, or at minimum stop trusting it without a gate, and the security curve looks different. Until the vendors building the tools and the vendors selling governance agree on that mechanism, the gap between what developers believe they shipped and what the CMU benchmark methodology would score them on is likely to keep widening.