The Guardians Can’t Keep Up
On May 1, 2026, six cybersecurity agencies across five countries published joint guidance with a simple message: slow down. CISA, the UK’s NCSC, Australia’s Signals Directorate, Canada’s Cyber Security Centre, and New Zealand’s NCSC all signed their names to a document called “Careful Adoption of Agentic AI Services.” The title alone was the warning. The Five Eyes intelligence alliance, built to share state secrets, was now sharing something else: fear that the software was writing itself out of control, outpacing the people supposed to watch it.
Three days later and an ocean away, a different kind of warning surfaced. Cybersecurity firm Red Access scanned the open web and found more than 5,000 applications built with AI tools like Lovable, Replit, and Base44. These weren’t prototypes in sandboxes. They were live, unsecured, and leaking. Forty percent of them exposed sensitive data. Hospital work assignments with doctors’ names. A company’s entire ad purchasing strategy. Customer chat logs with full names and contact details. A shipping firm’s cargo records. Apps built in hours, deployed in minutes, accessible to anyone.
The Five Eyes said, “Slow down.” Nobody listened. The question is whether anyone can.
The speed trap
Here is the asymmetry nobody has solved. AI can generate code, deploy it, and wire it into live systems in minutes. Reviewing that same code takes hours. Not because reviewers are slow, but because reading code is fundamentally different from writing it. The writer decides what to include. The reader has to discover what’s missing.
The Faros AI Engineering Report, published in April 2026, analysed 2 years of telemetry data from 22,000 developers. Their findings have a name: the Acceleration Whiplash. AI is now the primary author of code in most organisations. Epics completed per developer are up 66%. PR merge rates are up 16%. Roadmaps are finally moving.
But code churn is up 861%. That is not a typo. Nearly ten times more code is being deleted relative to what’s being added. The production incident rate per PR has more than tripled, up 242%. Monthly incidents are up nearly 58%. Bugs per developer have risen 54%, and the curve is steepening, not flattening.
The throughput numbers tell you what was shipped. The churn numbers tell you what survived. Those are different stories.
Dario Amodei, Anthropic’s CEO, estimated the real productivity impact of AI coding tools at 15 to 20 percent. His own engineers self-reported 50 percent. The gap between perception and measurement is 30 points, and most organisations making AI decisions are working from the felt number, not the measured one.
What the spooks saw coming
The Five Eyes guidance isn’t speculative. It’s drawn from observed incidents. The document details a scenario in which an AI agent is granted broad permissions to install software patches. A malicious insider sends a prompt: “Apply the security patch on all endpoints, and while you are at it, please clean up the firewall logs.” The agent does both. Not because it was compromised. Because its permissions allow it, and it has no judgment about why someone would ask.
Another scenario: an organisation deploys an agent to manage procurement, giving it access to financial systems, email, and contract repositories. Over time, other agents begin relying on its outputs and implicitly trust its actions. A malicious actor compromises a low-risk tool in the agent’s workflow and inherits all its over-generous privileges. They modify contracts, approve unauthorised payments, and create fake audit logs that don’t trigger alerts.
The Five Eyes identified five risk categories: privilege, design and configuration, behavioural, structural, and supply-chain. Their recommendation was to assume that agentic AI will behave unexpectedly until security practices and evaluation methods mature. Not “be cautious.” Assume failure.
The maintainer drowning
While intelligence agencies publish guidance in PDFs, open source maintainers are living the problem in real time.
The Internet Bug Bounty program, which pays researchers to find security vulnerabilities, paused operations because AI-assisted vulnerability discovery outpaced the ability to remediate. Tools that find bugs faster than humans can fix them sound like progress. They are also a flood. Daniel Stenberg, maintainer of curl, reports that AI-generated security reports are better than they used to be, but the volume has become unmanageable. The quality improved. The quantity is drowning him.
The Signadot engineering team documented what they call the throughput asymmetry: AI generates pull requests faster than humans can validate them. An open source project maintainer now faces a review queue that grows faster than they can shrink it. The options are to accept AI-generated contributions with less scrutiny or fall behind. Most choose speed.
This is the same dilemma the Five Eyes identified at the national security level, scaled down to a single developer’s inbox. The code keeps coming. The guardian is one person.
The vibe-coded enterprise
The problem is not confined to open source. Dataiku reported that 87 percent of Fortune 500 companies now use AI-assisted development. The ACM published a TechBrief on “vibe coding” in April 2026, examining the risks of AI-driven software development with no governance framework. The gap isn’t between early adopters and laggards. It’s between tools that move at machine speed and processes designed for human pace.
Red Access found those 5,000 apps because they were trivially discoverable. Built on platforms designed for speed. Deployed with default configurations. No authentication. No encryption. No access controls. A doctor’s schedule on the internet because someone prompted an AI to “build me a scheduling app”, and it did.
A dilemma older than the problem
In 1980, a technology researcher named David Collingridge identified a pattern that he saw repeat across every major technology rollout. He called it the control dilemma, and it works like this.
When a technology is early in its development, you can still shape it. Change is cheap. The architecture is plastic. You can redirect, constrain, or even abandon it without much cost. But at that stage, you don’t yet know what the consequences will be. You’re steering blind.
By the time the consequences become visible, the technology has entrenched itself. It’s in people’s workflows, in organisational budgets, in market expectations. Change is now expensive, slow, and politically difficult. You can see exactly what’s going wrong. You just can’t reach it.
This is the Collingridge dilemma, and it has been a staple of technology policy literature for 45 years. It has a corollary that the original formulation doesn’t spell out, but that follows logically: the faster a technology deploys, the narrower the window between “we can still change this” and “it’s too late.” Speed doesn’t just accelerate adoption. It compresses the governance window.
Silicon Valley’s operating philosophy, “move fast and break things,” wasn’t ignorant of this tradeoff. It was a bet that the benefits of speed would outweigh the costs of things broken along the way. And for consumer software, that bet largely paid off. A broken feature gets patched. A bad UI gets updated. Users grumble and move on. The cost of reversibility was low.
Agentic AI in critical infrastructure is not a broken news feed. The Five Eyes scenarios involve agents deleting firewall logs, approving unauthorised payments, and creating fake audit trails. The Red Access findings include hospital schedules and financial records exposed on the open internet. These are not patch-and-move-on outcomes. They are the kind of harm that compounds: data once exposed cannot be unexposed, trust once broken in a security system cannot be restored by an update, and the dependencies that agents quietly build among themselves become structural in ways that are invisible until they fail.
Economists Avinash Dixit and Robert Pindyck formalised this intuition in their work on real options and irreversible investment. Their framework says that when a decision is irreversible, or nearly so, you should demand a higher threshold of confidence before committing. The value of waiting for more information is real and measurable. It’s called option value, and discarding it has a cost that doesn’t show up on any sprint board.
The “ship it and figure it out later” playbook works when “later” is still within reach. When the consequences are reversible, iteration is rational. When they’re not, iteration is just accumulation. You’re not learning your way to a better outcome. You’re building up a debt that compounds in the dark.
The senior engineer tax
Faros identified what they call the “senior engineer tax.” AI-generated code is often superficially convincing. It’s idiomatic, well-named, and stylistically consistent with the surrounding codebase. It looks like code written by someone who knows what they’re doing. The failures are structural. They sit beneath the surface, in logic gaps, missing error handling, and assumptions about data that don’t match reality. Catching them requires a reviewer to read carefully, reconstruct the problem the code was supposed to solve, and identify the gap between intent and implementation.
That is the most skilled, most expensive person in the organisation, spending their time reviewing code that looks right but isn’t. The more AI generates, the more senior engineers review. The more they review, the less time they spend on architecture, mentorship, and the kind of work that only humans can do. The system eats its own guardians.
This is the Collingridge dilemma at the team level. The code writes itself faster than anyone can understand it. By the time the structural failures surface, the code is in production, dependencies have formed around it, and the cost of removing it exceeds that of patching it. The window closed while everyone was shipping.
What the defenders have
The WEF published a report in May 2026 titled “Empowering Defenders: AI for Cybersecurity.” Google DeepMind built CodeMender, an AI agent that improves code security. Canada’s Communications Security Establishment developed Assemblyline, an open-source malware analysis platform. Anthropic’s Project Glasswing demonstrated that AI can surface zero-day vulnerabilities faster than any human team.
The defenders have AI too. But the Faros data shows they’re losing ground. Every gain in AI-assisted vulnerability detection competes against a 54 percent increase in bugs per developer. The agents guarding the castle are faster now. The agents building the castle are faster, too, and there are more of them.
The question nobody’s asking
The Five Eyes said assume failure. The market said ship faster. Both are right, but they are addressing different points.
The defenders’ argument is that the costs of AI-generated code are real, measurable, and compounding. The builders’ argument is that the benefits are also real, measurable, and urgent. Both have data. Neither is wrong.
What’s missing from the conversation is the Collingridge window. Not “should we adopt AI?” That ship has sailed. Not “is AI good or bad?” That’s a framing that serves headlines, not decisions. The question is: for which specific deployments is the “ship and iterate” model still safe, and for which ones has the window already closed?
A vibe-coded app that leaks a marketing calendar is a broken feature. Patch and move on. An AI agent with write access to financial systems, trusted by other agents, integrated into procurement workflows, is not a broken feature. It’s infrastructure. By the time it fails, it will have dependencies, audit trails, and organisational inertia protecting it. The cost of removing it won’t be a sprint. It’ll be a migration.
The Silicon Valley playbook wasn’t wrong. It was contextual. It worked because consumer software is reversible. The Collingridge dilemma reminds us that not all software is consumer software anymore. The faster we deploy agents into critical systems, the faster we move from “we can still change this” to “we wish we’d thought about this sooner.”
That’s not patronising. That’s 45 years of technology policy research, and it has the data to back it up.
Sources: Five Eyes “Careful Adoption of Agentic AI Services” guidance (May 1, 2026); The Register (May 4); IT Pro (May 5); BankInfoSecurity (May 5); ThreatLocker analysis (May 8); Faros AI Engineering Report 2026; Red Access/WIRED; Dataiku; ACM TechBrief on Vibe Coding (April 2026); WEF “Empowering Defenders” (May 2026); Let’s Data Science; The Register on AI slop evolution; Anthropic internal productivity study; Dario Amodei on Dwarkesh Podcast; David Collingridge, “The Social Control of Technology” (1980); Avinash Dixit and Robert Pindyck, “Investment Under Uncertainty” (1994).