There is a conversation happening in businesses everywhere at the moment.
Why should we pay a developer for this? I built something similar with Claude over the weekend.
And, increasingly, that statement isn’t ridiculous.
Claude, Codex, Replit, Lovable and the growing army of AI coding tools can produce remarkably good software remarkably quickly. Someone with limited programming experience can describe what they want and, a few hours later, have something that looks and behaves like a real application.
It logs users in. It stores data. It sends emails. It talks to an API. It has a nice dashboard.
It works.
And that’s precisely where the problem starts.
Because “it works” and “it is safe to run a business on” are two very different things.
AI has made code cheap
Let’s get one thing out of the way first.
AI is not a gimmick, and professional developers pretending otherwise aren’t doing themselves any favours.
We use AI ourselves.
In the hands of an experienced developer, it is an extraordinary productivity tool. It can write repetitive code, generate tests, explain unfamiliar libraries, refactor existing systems and turn ideas into working prototypes far faster than was possible a few years ago.
The economics of software development are changing.
But something important gets lost in the excitement:
AI has made code cheap. It hasn’t made professional software engineering optional.
Writing code and engineering software were never the same thing. AI has just made the difference visible.
The expensive parts are the things you can’t see
Imagine somebody in your company builds a small customer portal with Claude.
They test it.
They can log in. They can create a customer. They can edit the customer. They can generate a report.
Everything works.
What they probably haven’t tested is whether changing customer/173 to customer/174 in a request gives them somebody else’s data.
They may not know whether permissions are being checked in the browser or enforced on the server.
They may not know whether an API key has ended up in the source code.
They may not know what happens when two people update the same record at the same time.
They may have a backup. Have they ever restored it?
What happens when the database contains 500,000 records rather than 500?
What happens when the AI rewrites a database migration six months from now?
Who notices when a library the application relies on develops a critical vulnerability?
None of these things stop the application working today.
And that is the problem.
Some of the most expensive defects in software are the ones you cannot see while demonstrating that it works.
This isn’t a theoretical concern
Veracode has been testing AI-generated code across more than 100 models since 2023. Its 2026 report found an average security pass rate of 56%. The 2025 figure was 55%.
The interesting part is that over the same period the models became extremely good at producing syntactically correct code. Above 95%, in fact.
In other words, the code looked right. It ran.
Security barely moved.
Researchers examining thousands of applications built on AI development platforms have found the same thing in the wild.
Escape scanned more than 5,600 publicly deployed vibe-coded applications in early 2026 and reported over 2,000 high-impact vulnerabilities, more than 400 exposed secrets and 175 cases of exposed personal data. All in live systems.
Red Access, an Israeli security firm, scanned roughly 380,000 publicly accessible assets built with Lovable, Base44, Replit and Netlify. About 5,000 of them contained sensitive corporate or personal data: internal financials from a bank, UK clinical trial details, unredacted customer service conversations. Many were indexed by Google. Axios and Wired independently verified exposed apps. It is vendor research, so the precise figures deserve some caution, but the platforms did not dispute that the exposed applications existed.
Gartner predicts that prompt-to-app development by citizen developers could increase software defects by 2,500% by 2028, creating what it describes as a software quality and reliability crisis.
And then there are the incidents with names attached.
Lovable, spring 2025. A developer noticed that a site built on Lovable let anyone read other users’ data by tweaking a query. He scanned the platform and found 303 endpoints across 170 projects with no working row-level security on their databases. It became CVE-2025-48757, assigned a CVSS score of 9.3 by MITRE, although NVD lists the CVE as disputed. Exposed data included emails, phone numbers, payment status and API keys. Lovable disputed the CVE on the grounds that customers are responsible for securing their own data. Which is exactly the point. The tool is not responsible for your data. You are.
Replit, July 2025. SaaStr founder Jason Lemkin documented an AI coding agent deleting his production database despite being told, repeatedly, not to make changes without permission. Records relating to 1,206 executives and 1,196 companies were deleted. The agent then generated thousands of fake records and reported that tests had passed. Replit’s CEO called it unacceptable and introduced protections that should have been there from the start, including separating development and production databases.
Moltbook, January 2026. A viral social network for AI agents, built by a founder who said publicly that he hadn’t written a single line of code. Within days, researchers at Wiz found a database key in the site’s public JavaScript that gave anyone read and write access to everything: about 1.5 million API tokens, 35,000 email addresses and thousands of private messages. Nobody needed an exploit. They browsed the site like a normal user and looked at what the browser downloaded.
The lesson isn’t that Lovable or Replit are bad.
It is that a prompt saying “don’t touch production” is not a security control, and a database setting the tool never mentioned is not one either.
A properly engineered system makes certain dangerous actions impossible regardless of what the developer, or the AI, decides to do.
That’s what environment separation, least-privilege access, backups, approval gates and deployment controls are for. The incidents above are different versions of what happens when those engineering controls are missing, misconfigured or never considered.
“But I just saved the company £2,000”
Maybe.
This is where AI-generated software becomes seductive.
Something that might previously have required a developer and a £2,000 invoice can now appear on somebody’s screen after two afternoons with Claude.
So the calculation seems obvious:
Development cost before: £2,000 Development cost now: £0 Saving: £2,000
Except that isn’t the calculation.
You also have to account for the employee’s time, professional review, security testing, deployment, future maintenance, remediation and the operational risk the business has just accepted.
Those costs haven’t disappeared.
They’ve become invisible.
The bill arrives later. When the person who built the application can’t work out why it has stopped working. When another system needs integrating with it. When the business grows past the assumptions buried in the generated code. When somebody finds the security problem.
There is already an industry forming around auditing, repairing and rebuilding AI-generated applications. Freelance platforms and development agencies now openly sell “vibe code audit”, security hardening and production-readiness services.
That should tell us something.
The first version may have been cheap to generate. Making somebody professionally responsible for it is still engineering work.
The real problem isn’t AI. It’s the missing engineer.
Experienced developers use AI too.
The difference is what happens around it.
A 2025 study observing professional developers using coding agents was titled, bluntly, “Professional Software Developers Don’t Vibe, They Control.” The engineers never skipped planning, even when they let the AI draft the plan, and 69% carefully reviewed every change the model made. In METR’s trial of experienced open-source maintainers, three quarters said they read every line of AI-generated code.
They plan, inspect, test, challenge and modify the output.
They also know when the AI is confidently doing something stupid.
That last part matters enormously, and it is where experience shows up in the data.
A February 2026 analysis of AI-assisted pull requests across roughly 1,800 GitHub contributors found that code from low-experience developers using AI created significantly more review work for maintainers, and took longer to be accepted, than code from experienced developers using the same tools. The paper set out to answer one question: can novices with AI substitute for experienced developers with AI? On their data, the answer was clearly no.
A Stanford study by Perry and colleagues found something worse. Participants with an AI assistant wrote more insecure code than those without, while being more confident it was correct. That study predates the current generation of tools, but the pattern it describes is what every incident in this article has in common. The person can’t see the flaw, and the tool sounds sure.
The people who understand these tools best trust them least. Stack Overflow’s 2025 survey found 84% of developers using or planning to use AI tools, yet only around a third trusted their accuracy, down from 43% the year before. Two thirds complained about AI solutions that were almost right.
“Almost right” is an inconvenience when generating CSS.
It is rather more serious when implementing authentication.
The skill of a software engineer has never simply been knowing the syntax of PHP, C#, JavaScript or Python.
It is knowing what questions need to be asked before anybody starts writing code.
Where does this data come from?
Who is allowed to see it?
What happens when this service goes offline?
What happens when the request is submitted twice?
What happens if the API returns something unexpected?
What needs logging?
What needs backing up?
How do we roll this change back?
What happens when this system is still running five years from now?
AI can help answer those questions.
Somebody still needs to know which questions to ask.
Shadow AI is shadow IT on steroids
There is another problem businesses need to think about.
Twenty years ago, IT departments worried about employees installing unauthorised software.
Then came shadow IT: departments buying their own SaaS products without involving IT.
AI takes this much further.
An employee can now create an entire application without anybody responsible for technology knowing it exists.
It may contain customer information.
It may connect to company systems.
It may have credentials for third-party services.
It may quietly become part of an important business process.
And nobody has reviewed it.
IBM’s 2025 Cost of a Data Breach research found that one in five organisations surveyed reported a breach involving shadow AI, meaning AI used outside normal organisational governance. IBM also found that organisations with high levels of shadow AI experienced breach costs averaging $670,000 more than organisations with little or none.
This is no longer an IT department worrying about employees playing with ChatGPT.
It is a governance problem.
So should businesses stop building things with AI?
Absolutely not.
That would be like telling businesses in the 1990s not to use spreadsheets because somebody might build a terrible one.
Use Claude.
Use Codex.
Build prototypes.
Automate repetitive internal tasks.
Experiment with ideas that would previously have been too expensive to explore.
Let technically capable employees become dramatically more productive.
But establish a boundary between something useful that somebody built with AI and software the company is prepared to depend on.
Once an application handles customer data, financial information, payments, authentication, commercially sensitive information or an important business process, somebody technically competent should take responsibility for it.
That doesn’t necessarily mean throwing it away and starting again.
Sometimes the sensible model is:
You build it. We review it.
A professional developer can inspect the architecture and generated code, test the security assumptions, fix vulnerabilities, introduce proper deployment and backup processes, document the system and take responsibility for maintaining it.
That engagement is measured in days. A breach is measured in months. AI may actually make the review model cheaper for everyone.
What businesses are really paying developers for
This is perhaps the biggest misunderstanding AI coding has exposed.
Businesses were never really paying developers for typing.
Typing code was simply the visible part of the job.
You are paying somebody to understand the problem, make architectural decisions, recognise risks, protect your data, test assumptions, integrate systems, manage deployments and leave behind something another competent person can understand three years later.
Most importantly, you are paying for accountability.
When an AI-generated application fails at 9am on Monday morning, Claude doesn’t have a support contract with your company.
When customer information is exposed, Codex isn’t explaining the incident to your clients.
When Lovable’s CVE landed, the platform’s position was that customers are responsible for their own data. When Replit’s agent wiped a database, the remedy was a refund.
And when the ICO asks how personal data came to be publicly accessible, “the AI wrote that bit” is unlikely to be a particularly useful answer.
The business owns the consequences.
The question has changed
For years, the first question businesses asked about software was:
“How much will it cost to build?”
AI is rapidly making that less interesting.
Increasingly, almost anyone can build something.
The questions worth asking now are:
Who knows whether it is safe?
Who understands how it works?
Who will maintain it when the requirements change?
Who will recognise the problems that aren’t visible on the screen?
And who is accountable when it fails?
Use AI.
We certainly do.
But don’t confuse the ability to generate an application with the ability to engineer and operate one.
The code may now be cheap.
The engineering isn’t optional, and the responsibility isn’t either.
Sources
- Veracode, 2025 GenAI Code Security Report and 2026 GenAI Code Security Report
- Forbes, “Vibe Coding Has A Massive Security Problem” (March 2026), citing Escape’s scan of 5,600 apps
- Axios, “AI vibe-coding apps leak sensitive data” (May 2026), and Wired’s parallel investigation, reporting Red Access’s findings
- Gartner, Predicts 2026: AI Potential and Risks Emerge in Software Engineering Technologies
- Superblocks, “Lovable Vulnerability Explained: How 170+ Apps Were Exposed”; NVD entry for CVE-2025-48757
- eWeek, “AI Agent Wipes Production Database, Then Lies About It” (July 2025); Jason Lemkin’s posts on X
- Wiz Research, “Hacking Moltbook: The AI Social Network Any Human Can Control” (February 2026)
- “Professional Software Developers Don’t Vibe, They Control: AI Agent Use for Coding in 2025”, arXiv:2512.14012
- METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” (July 2025)
- “Novice Developers Produce Larger Review Overhead for Project Maintainers while Vibe Coding”, arXiv:2602.23905 (February 2026)
- Perry et al., “Do Users Write More Insecure Code with AI Assistants?”, ACM CCS 2023
- Stack Overflow, 2025 Developer Survey, AI section
- IBM, Cost of a Data Breach Report 2025