2026 Vibe Coding Meta Study

The Cheat Sheet

What to do, what to fear, what the numbers say.

1. The checklist — what to do, and what to demand from a vendor

Require these in the platform

Ask these four separately in the contract — vendors answer one and let you assume the rest

Run these in your own programme

2. What is new — the threats vibe coding introduces

Most of these evade the controls you already run, either because they appear after code review, or because nothing about them looks wrong.

3. The numbers that matter

Drawn from the 34-source corpus. Full method, sample sizes and commercial interests are in Appendix A.

60 Second Overview

Your employees have started building their own software, describing what they want in plain English and letting an AI assemble it. The industry calls this vibe coding, and it took roughly 16 months to travel from hobbyist novelty to something running inside real companies.

They are building the right things, largely because the person with the problem writes a better specification than any analyst sent to interview them. That part of this is working well.

What they are not doing is finishing. The AI builds the screens and then stops, leaving out everything that keeps data private, auditors satisfied and the application alive a year from now.

The flaw is almost always the same one: the application never checks whether the person using it is allowed to see the data. Your employee has no way of catching this, because they can see that their app works but not that it works equally well for a stranger.

They also ship without audit logs, so when something does go wrong there is no record of who saw which record, who changed what, or when.

Finding 1

1. Your Best Requirements Are Walking Around the Building

The employee with the problem writes a better specification than anyone you could hire to write it. This is why banning the practice fails.

Adoption was unusually fast. The Cloud Security Alliance clocked the move from fringe technique to broad enterprise use at roughly 16 months, which is not a speed enterprise practices normally manage.

A vendor survey of 50 executives found that half of enterprises already have non-engineering teams building internal applications. The sample is small, but it points the same way as the shadow-AI figures: Microsoft found 75% of employees using AI tools their employer never approved, and 78% of workplace AI users bringing their own.

What they build

The applications share a profile. They are small, they serve one department, and they automate something a spreadsheet and an email chain used to handle between them. None of them was ever important enough to earn a place on the IT backlog.

• Dashboards that pull live numbers out of an operational system.
• Workflows with a form and an approval step, such as client intake, vendor onboarding or expenses.
• Calculators for quoting, pricing, or configuration.
• Portals and trackers that let clients check status without telephoning anybody.
• Report generators that end a monthly copy-and-paste ritual.
• Glue between two systems that ought to talk to each other and do not.

Why the specification is better

The person with the problem knows the exceptions, and knows which step genuinely wastes an afternoon and which one only looks wasteful. Write all that down for somebody else and most of it evaporates; route it through an analyst who then briefs a developer and more goes still.

When the employee builds it themselves the specification never has to travel, which removes the largest single cause of failure in internal software: the wrong thing, built beautifully.

Vendor case studies claim ten semi-technical builders shipping eight production applications in 30 days with no developer involved, and a process falling from five days to twelve hours. Treat that precision sceptically, since Section 11 explains what such figures do and do not measure, but the direction of travel is not in dispute.

Why a ban does not work

A prohibition produces one of three outcomes. Staff ignore it; or they move to personal accounts and personal email, at which point you lose visibility altogether; or an executive overrules you, because the value is real and somebody senior has noticed. None of the three is what you wanted, and since the tool is a browser tab and a credit card, blocking it was never really available to you.

Yet these apps often end up in the same place — company data sitting in an application nobody in IT has ever looked at, which is both a leak waiting to happen and an open door into whatever it connects to.

Finding 2

2. Almost Every App Has the Same Hole

Four teams, four methods, four samples — same failure.

Independent researchers using different techniques, with different commercial motives and no coordination between them, arrived at the same conclusion. In a field 18 months old that almost never happens, which is why this is the strongest result in the corpus.

Four independent studies, one conclusion

It is always the same four failures

The defect rate is less interesting than the pattern behind it. Nobody is finding exotic or clever vulnerabilities; they keep finding the same small handful.

1. 1.The database says yes when it should say no. In the Symbiotic sample, 172 applications let any stranger delete records, 172 let anyone modify them, and 39 exposed entire tables to be read. The cause in each case was a public key with no access policy behind it.
2. 2.Passwords and keys left in the open. Credentials sit in code that the browser downloads, which is how one count reached 1,275,105 leaked AI service keys in 2025, up 81% in a year.
3. 3.The lock is on the wrong side of the door. The application checks your password inside your own browser, where anyone can remove the check.
4. 4.No lock at all. Internal dashboards and test systems published straight to the open internet.

An autonomous penetration test reported on 22 July 2026 found the same thing: 434 exploitable problems across three applications, in scans of 30 minutes each. The biggest groups were authorisation failures (88) and access-boundary failures (54), and of the 23 critical findings, 11 were hardcoded secrets and 6 were remote code execution through a debug mode somebody left switched on.

Why this failure and not some other

The mechanism behind all this matters rather more than any of the percentages above.

Your employee can verify that the application works, because they open it, do the task, and watch the task get done. What they cannot verify is that it refuses to work for somebody without permission, since access control has no visible symptom at all. A safe application and a dangerous one look identical from the builder's chair.

Finding 3

3. The Bill Did Not Vanish.

Writing code got cheap. Checking and fixing it is still expensive.

The largest study in the corpus examined 302,600 verified AI-written commits across 6,299 public repositories and five different AI assistants, running static analysis before and after every change and then following each problem forward through later versions.

It found 484,366 distinct problems, with more than 15% of commits from every single assistant introducing at least one. But one number matters considerably more than the rest.

That is a measured liability rather than an opinion, and it means nearly one problem in four becomes permanent furniture.

Where the work went

The Stack Overflow survey of 49,009 developers shows exactly where the effort relocated to.

• 66% say the tool produces "solutions that were almost right, but not quite".
• 45% say debugging AI-written code takes longer than debugging code a human wrote.
• 40% say it handles genuinely complex work badly.
• 75% go and find a human when they do not trust an AI answer.
• 62% want a human involved in anything touching security or ethics.

"Almost right, but not quite" is the most expensive category of wrong in software. Obviously wrong costs nothing because you throw it away, whereas almost right sails through review and then fails in front of a customer six weeks later.

Fast and broken at the same time

The Cloud Security Alliance states the trade directly: developers using AI ship code three to four times faster, and introduce vulnerabilities at ten times the rate. Both figures are true at once, and together they describe a straightforward problem, which is that output accelerated and checking did not.

A brand new way to get attacked

One consequence of all this has no real precedent. Roughly 20% of AI-generated code refers to software packages that do not exist, so attackers now register those invented names and fill them with malware, and the next person to run the code installs it.

The industry calls this slopsquatting. No traditional review process catches it, because nothing about it looks wrong.

Finding 4

4. Nobody Can Feel Their Own Productivity

They were 19% slower. They were certain they had been 20% faster. This is the most important number in this document.

METR ran the only randomised controlled trial in the field and published it in July 2025.

The method was strict. Sixteen experienced developers worked on 246 real issues in their own repositories, projects carrying more than 22,000 stars each, with every issue randomly assigned either to allow AI tools or to forbid them. Sessions were recorded, tasks averaged two hours, and the developers were paid $150 an hour.

They took 19% longer with AI. Beforehand they had expected a 24% speed-up, and afterwards, having personally lived through the slowdown, they estimated they had been 20% faster.

What this does and does not prove

The authors are careful about this and we will be too. They make no claim that the result holds for all developers or all software work, sixteen people is a small sample, and the tasks were complex changes to large existing codebases, which is a different activity from building something new.

Strip away those caveats and one thing survives: a person using these tools cannot tell what the tools are doing to them.

What that means for governance

Nearly every productivity claim in this market is self-reported. A vendor survey proudly reports that 72% of enterprises measured a return on their AI investment, which is a self-report, and METR tells you what a self-report is worth.

You therefore cannot govern any of this with satisfaction surveys or enthusiastic team updates. You have to count things instead: defect rate, time to fix, applications in production, and applications with nobody's name on them.

Finding 5

5. The Sentence Your Employee Types Is the Asset

The research examines whether the app leaks IP to frontier labs who may empower your competitors with your process improvements and ideas.

Your employee types a sentence in plain English, and that sentence describes how your company prices a job, triages a claim, or decides which client to take on. The description is the valuable part and the code is merely its output, which is why a competitor who obtained the sentence would learn far more than one who obtained the software.

On most public platforms that sentence goes straight to a model vendor, and whether it becomes training data is settled by a contract term your employee never read and your company never negotiated.

What 34 sources cover — and what they do not

"Sovereign AI" is four different questions

Vendors routinely answer one of them and let you assume the rest, so ask them separately.

1. 1.Residency. Which country, and whose hardware, does the data sit on?
2. 2.Ownership. Who owns the model weights, and can you keep operating if the vendor withdraws the model?
3. 3.Control. Who can switch it off, and who can read the logs?
4. 4.Training rights. Do your instructions and your data improve a model that your competitors then use?

A private-cloud arrangement answers the first question and part of the third, while saying nothing whatever about the second or the fourth. This is not a technicality. In August 2026 a major cloud provider and a platform vendor announced exactly such an arrangement, with the reassurance that data never leaves, and that statement is entirely accurate about residency without being a statement about training rights at all.

Why the researchers missed this

The explanation is structural rather than sinister. The security researchers who produced the best evidence in this review sell scanning products, and while a scanner can find an exposed database in seconds, no scanner has ever read a contract.

Finding 6

6. Everyone Names the Disease. Nobody Writes the Prescription.

A standards body has now said this out loud

The Cloud Security Alliance examined the major AI security frameworks and reported that none contains dedicated guidance for the citizen developer, naming the NIST AI Risk Management Framework, the OWASP Top 10 for LLM applications, CSA MAESTRO, and the CSA AI Controls Matrix. That is a formal statement, from a neutral body, that the rulebook for this does not yet exist.

Why advice to the builder can never work

Most published guidance tells the builder to do something: review the generated code, check the access policy, strip out the secrets, add a test. It carries a fatal flaw, which is that it requires precisely the skill the builder does not have. If your operations manager could audit an access policy she would be a developer, and she would not need the tool in the first place.

There is a second and more arithmetic reason. Nobody reviews 400 applications built by non-engineers, and review will never scale to that number.

7. The Twelve Things AI Will Not Do

The AI builds the visible part, meaning the screens, the forms, the basic logic and the report at the end. Everything below is what it leaves behind, and each item names the finding that proves it is necessary.

The last mile, itemised

Invented packages are the smaller half of item 9. In November 2025 the Shai-Hulud 2.0 campaign compromised roughly 800 legitimate npm packages, ran during installation, and harvested 775 GitHub tokens along with 373 AWS, 300 GCP and 115 Azure credentials — the dependency that should worry you is not the one that does not exist, but the trusted one that does.

Let's look at items 4, 5 and 6 again

They are knowledge problems wearing the costume of programming problems, and item 6 has no technical solution anywhere. Somebody in your building knows which customer table is the real one, why the discount field holds text, and what status code 7 meant back in 2019, and that knowledge sits in one or two heads without ever having been written down.

Which leads to what we think is the single most useful observation in this document.

Employee-built software does not create data debt. It exposes data debt that spreadsheets have been hiding for a decade.

The moment an application connects two systems, those two systems have to agree with each other, and they almost never do.

8. The Rise of the Forward Deployed Engineer

The 10x Engineer doesn't code 10x faster, they convert disparate employee meetings into architected data flows.

Section 7 describes twelve things that have to happen after the AI stops. Somebody has to do them, and over the past eighteen months a particular kind of somebody has become the standard answer across the industry.

The forward deployed engineer, or FDE, is an engineer who works inside the customer's organisation rather than at a vendor's office. Palantir invented the pattern around 2005 and for years employed more embedded engineers than conventional product engineers. The model stayed a Palantir peculiarity for the best part of two decades, and then AI arrived and everybody copied it at once.

Palantir still leads on volume, followed by OpenAI, Anthropic, Google, Databricks and Scale AI, with the steepest growth rates appearing at vertical AI companies. Mid-senior roles cluster between $300,000 and $550,000 in total compensation, and staff and principal roles at the frontier labs run past $1.2 million.

Why the model came back

The reason is not a talent shortage. It is a failure rate. Roughly 95% of enterprise AI pilots produce no measurable business impact, and the cause is almost never the quality of the model: it is everything around the model, in the customer's own systems, with the customer's own data.

One forward deployed engineer described the work in a sentence that could serve as the caption for Section 7 of this document.

The model is usually the cleanest part. The hard part is finding the workflow nobody documented, and the data source people actually trust.

That is items 4, 5 and 6 of our specification, described by somebody who does the work rather than somebody who studies it. The undocumented workflow and the trusted data source are not technical objects at all. They are organisational knowledge, and they cannot be extracted by a model, a consultant working from a written brief, or a platform, however well governed.

The four jobs inside the job

What makes the role effective is that it combines four functions that companies normally split across four different people, none of whom talks to the others.

1. 1.Organising the team and its knowledge. The FDE sits with the people who have the problem and writes down what has never been written down: which system holds the true record, why a field contains text instead of a number, what an undocumented status code means, and which of three customer tables anybody actually trusts. Item 6 of Section 7 has no technical solution, and this is the only reliable way it gets solved.
2. 2.Coding the gaps. When the AI stops at 80%, somebody has to build the connector into the ERP that has no modern interface, or the scraping robot pointed at a supplier portal, or the reconciliation logic between two systems that disagree. That is engineering work, and it needs an engineer.
3. 3.Hardening what the employee built. Access control at the data layer, identity, secrets, audit logging, error handling for automation that runs unattended, and a way back to the last version that worked. Section 2 of this document explains why the person who built the application cannot do this, and why the failure is invisible to them.
4. 4.Working with IT and compliance. This is the function nobody else performs and the one most often left out of the job description. The FDE translates in both directions, turning a business user's intention into something a security team can approve, and turning a compliance requirement into something the platform enforces by default. Without it, governance stays a document and the applications keep shipping around it.

Why hourly beats headcount

The economics matter as much as the skills, and they are the reason the model suits mid-market companies specifically.

The last mile is episodic rather than continuous. It arrives in bursts, when an application is ready to graduate from a demonstration into production, and then it goes quiet until the next one. Hiring a senior engineer converts that spiky demand into a permanent salary, a manager, a retention problem and a backfill risk, and at the compensation levels above it is a competitive hiring fight that most mid-market companies will lose.

Buying the same expertise by the hour matches the cost to the shape of the demand. It scales up when three applications are ready at once, scales down to nothing when none are, and ends when the work ends rather than becoming a line item that outlives the project.

Four ways to get the last mile done

The biggest risk in the FDE model

The FDE model has a real failure mode, which is dependency: the engineer accumulates the organisational knowledge from function one, and if that knowledge leaves when they do, you have rented understanding of your own business rather than acquiring it.

Make the written artefacts a deliverable in their own right, including the data map, the decisions about which system is authoritative, and the documented meaning of the fields nobody could previously explain. If an engagement ends and the only record of what was learned is in somebody else's head, the engagement failed regardless of how well the software works.

9. What to Do on Monday

First, stop treating every app the same

A dashboard showing public data does not need what a client portal needs, so sort them into three tiers and spend your effort on the top one.

Three tiers, three levels of effort

Then next 11 moves

1. 1.Find out what already exists. You cannot govern an unknown quantity, so check expense reports for platform subscriptions and identity logs for connections to build platforms.
2. 2.Declare an amnesty. Ask people to register what they have already built, with no penalty whatsoever. Punishment produces concealment, and concealment is the entire problem you are trying to solve, which is why this single step decides whether the programme succeeds.
3. 3.Make the approved path the easy path. People take the route of least resistance, so if your sanctioned platform is slower to start than a public one they will use the public one and you will have achieved nothing.
4. 4.Put identity in the platform. Every application inherits company sign-on, and no application manages its own passwords.
5. 5.Put permissions in the connector. The connection to a system of record carries the user's own permissions, so nobody can build an application that reads data they themselves cannot read.
6. 6.Ban secrets from application code. The platform holds the keys and the builder never sees one.
7. 7.Log everything by default. Audit records should appear without anybody configuring anything.
8. 8.Gate the launch, not the build. Let people build freely, and apply the control the moment an application touches real data or gains a second user.
9. 9.Name an owner for every application. An application with no owner gets switched off after notice, which is how you avoid accumulating a graveyard of software nobody dares delete. You may want to have only one user in the organization pre-configure connectors.
10. 10.Count things rather than surveying them. Track applications in production, defect rate, time to fix, and applications with no owner. Section 4 explains why a satisfaction survey is worthless here.

10. A Weekend Hobby Went Into Production

Andrej Karpathy coined the term in February 2025 with a precise definition: you tell an AI what you want, accept what it gives back, and never read the code. You can, he wrote, "forget that the code even exists". He was equally precise about what it was for, and what it was for was throwaway weekend projects.

The correction that drew the line

Simon Willison drew the useful line on 19 March 2025, pointing out that using AI to help write software is not the same activity as vibe coding, and that the boundary between them is the code review. A developer who reads what the model produced is a professional using a tool, and one who does not is vibe coding. Willison's own rule is that he will not commit code he could not explain to somebody else.

What working programmers actually think

The 2025 Stack Overflow survey collected 49,009 responses from 160 countries, and the split it found is a sharp one.

Only 3.1% report high trust in what the tools hand them, and among experienced developers that figure falls to 2.5%. They use these tools constantly and trust them very little, which is roughly what happens when somebody actually reads the output.

Why this is the whole problem

The word travelled from weekend projects into production systems without the practice changing to match. A method designed explicitly for work that does not matter is now producing applications that hold client data, and Sections 2 and 3 of this document are a consequence of that single mismatch.

Finding 7

11. Both Sides Are Right. But They're Timing Different Races.

The two bodies of literature look as though they are calling each other liars, when in fact they are standing at different points on the track holding different stopwatches.

The same app, measured twice

Both sets of numbers can describe the same application. A team can absolutely ship eight applications in thirty days, and those eight applications can absolutely be full of holes, since nothing in the evidence prevents it and it is in fact the expected outcome.

The study nobody has run

Not one source follows a set of employee-built applications from first version through security review, correction, deployment and twelve months of ownership, and then reports the total. Until somebody does, no enterprise can calculate a real return on any of this, because you can only price the first stage, which is conveniently the cheap one.

Three questions that make a speed claim useful

1. 1.What state was the application in when the clock stopped? Demonstrated to a room, or in production with real users and real data?
2. 2.Who finished it, and how long did that take? This is the number that never appears in the case study.
3. 3.What condition are those applications in now, and who maintains them? Ask specifically about the ones built eighteen months ago.

The vendors agree with us

A platform vendor whose entire business is selling a tool that lets non-engineers build software surveyed 50 executives and vice presidents at companies of 500 to 10,000+ employees, and its own research found the following.

• 70% disagree that this practice will replace core engineering.
• 72% plan to employ the same number of engineers next year, or more.
• Top three worries: security (25.6%), compliance (19.2%), and the model simply being wrong (17.9%).

A company with every commercial reason to claim the opposite is reporting that expert engineers are still required. The sample is small enough that the decimal places should be treated loosely, but the direction is what counts, and it matches every independent source in this review.

Finding 8

12. The Studies Are Watching the Wrong People

Almost all of this research examines professional programmers. Yet almost none of your builders are professional programmers.

Read the method sections back to back and the pattern jumps out immediately.

Who each study actually watched

Every study that watches people is watching professionals, and every study that examines applications has no idea who built them. Your own population sits outside both: an employee who is not a developer, building on company data, inside a company with compliance obligations. Nobody has studied them.

This cuts both ways, and honest reading requires saying so

• The upside may be understated. METR watched expert developers doing hard work in large codebases, whereas an operations manager building her first small application has no comparable skill to lose. The gain for her could be far larger than anything in these studies, and nobody has measured it.
• The risk may also be understated. Every scan looked at applications visible on the open internet, so anything sitting behind a company login appears in no sample anywhere. Those are the ones holding the more sensitive data.

What we can say safely

One inference is airtight: if trained programmers who read every line still produce these defect rates, people who never read the code will not do better. That is the weak form of the argument, and it is enough, because the direction is certain even though the magnitude remains unmeasured.

13. What This Evidence Cannot Tell You

1. 1.The population is wrong. No rigorous study watches the employee who is not a developer, which is the most important hole in the field and one somebody should fill.
2. 2.Only visible applications have been examined. Every scan looked at the open internet, so the applications behind your firewall, which hold more sensitive data, have never been studied by anybody.
3. 3.The field is young and largely unreviewed. Much of the strongest work here is a preprint or a vendor report, and the subject is about 18 months old.
4. 4.Much of the evidence comes from people selling a cure. That does not make it wrong, though independent replication would make it stronger. Appendix A records every publisher's commercial interest so you can weigh it yourself.

The four studies this industry needs

• A cohort study following employee-built applications from first version through twelve months of ownership, reporting the total cost.
• A scan behind the firewall, with the consent of the companies concerned.
• A controlled trial with non-programmers, to finally measure the effect on the population that actually uses these tools.
• A control framework for the citizen builder from a standards body, since the Cloud Security Alliance has confirmed that none exists.

14. Who Wrote This, and What We Sell

JetBridge published this document, and JetBridge operates AirCode, a platform for employee-built applications. We have a commercial interest in these findings, since Finding 6 and Section 7 describe a need our product exists to meet, and those two sections are worth rereading with that in mind.

There is one further connection. A principal of JetBridge is a researcher at METR, the organisation behind the randomised controlled trial in Section 4. We had no part in that study, the disclosure appears in full alongside the finding itself, and our argument does not rest on it.

What AirCode is

• A build platform with the Section 9 controls, so identity, permissions, secrets, audit logging and inventory all happen without the employee doing anything.
• A sovereignty position that answers all four questions in Section 5. We do not train models on customer prompts or usage, and enterprise customers can run the harness and an open model on their own hardware.
• Forward deployed engineers who complete the twelve items in Section 7.

Section 7 shows that the last mile requires an expert human, and most platforms in this category hand you a governed platform and then leave that work to your engineering team. Findings 8 and 6 explain why this fails in a mid-market company, which is that you do not have spare senior engineers and neither does anybody else. The ones you have are committed to the revenue roadmap, and the last mile quietly taxes it.

We supply the engineers alongside the platform, which turns the last mile into a variable cost rather than permanent headcount.

Potential Trade Offs Between Sovereignty and Efficiency

AirCode does not train on customer prompts or usage, whereas platforms that do get steadily more efficient, because they learn from their customers and spend fewer tokens on the same task. We never get that benefit, so our platform uses more tokens than a platform that learns from you, and we accept it.

Slightly increased inefficiency may be the receipt for the privacy.

When you should not buy this

• You have a large engineering team with spare capacity. Then you do not need our engineers, so buy a platform and use your own people.
• You have no compliance obligation and no proprietary process worth protecting. A public platform will serve you perfectly well and cost you less.
• You want a demonstration rather than a production system. None of this applies to you, so go and build something on a weekend, exactly as Karpathy intended.

15. How We Read the Field

34 sources digested: What happens when a non-programmer ships software?

We looked at everything published between February 2025 and August 2026 that either stated a method or reported original data:

The corpus at a glance

What we recorded about every source

We noted seven things about each source:

• Date. This field moves in months rather than years, so anything much older than a year needs checking before you lean on it.
• Method and sample size. Sixteen developers and forty-nine thousand developers support very different claims.
• Unit of analysis. A person, a commit, or a deployed application. These are not interchangeable, and mixing them is how bad statistics get made.
• Population. Professional developers, or ordinary employees? Section 12 explains why this turns out to matter more than anything else on the list.
• Access. Free, gated, or paywalled.
• Commercial interest. What does the publisher sell? Profiting from a finding does not make the finding wrong, but you should be able to see the incentive.