AI Prototype vs Real App: The Difference Nobody Warns You About

I checked the timestamps, because I did not believe it either. One prompt, one follow-up about the colour scheme, and I had a working booking tool with a login screen, a calendar, and a database behind it. I sent the link to a friend that night with three exclamation marks.

The next morning she messaged to say she could see my test bookings. All of them. Names, phone numbers, the fake email address I use for testing. She had done nothing clever. She signed up, opened the app, and the data was there, because the table it lived in had never been told who was allowed to read it.

That gap, between something that runs and something you can hand to strangers, is the subject of this article. It is not a gap in ambition. It is a list of boring, checkable engineering work, and having done it the slow way, I know what is on the list.

The 19-minute high, and what it is actually measuring

The first hour with a builder like Lovable, v0, Bolt, or Replit Agent is genuinely a great hour, and it is hard not to extrapolate from that speed. The trouble is what the hour measures. It measures the happy path: one user, correct input, good network, an empty database, no payment, no deadline, no attacker. Every one of those conditions is false in production.

Addy Osmani, who runs Chrome Developer Experience at Google, named this in late 2024: the 70% problem. AI takes you to roughly 70% of a solution very fast, then the last 30% (edge cases, integration, security, keys, real data) takes about as long as it always did. In January 2026 he revised the number upward for greenfield work, calling it the 80% problem, and made the sharper point: the percentage moved, the nature of the leftover work did not.

Feature by feature, that leftover work looks like this.

TWO DEFINITIONS OF DONE

None of it is exotic. That is the point. It is unglamorous, and it is the entire difference.

Access control is off until someone turns it on

This is the failure I hit personally, and it is the most common one in the wild.

In May 2025, security researcher Matt Palmer published CVE-2025-48757, an incorrect-authorization flaw (CWE-863) in apps generated by Lovable. Projects shipped with Supabase tables that had no working Row Level Security policies, so unauthenticated requests could read and sometimes write to live tables. It scores 9.3 on CVSS 3.1, and the research found more than 170 production apps in that state. Lovable disputes the CVE, saying each customer is responsible for their own application’s data. That is a fair position and a precise description of the problem: the platform hands you a database, and the rules on it are yours.

The same pattern appears with no AI involved. July 2025’s Tea app breach came from a legacy Firebase bucket left open without authentication: roughly 72,000 images including about 13,000 verification selfies and ID photos, then around 1.1 million private messages days later. A door nobody locked.

HOW TO CHECK YOURS IN TEN MINUTES

● Create a second account in a different browser profile. Try to read the first account’s records.

● View source on your live site and search the JS bundle for sk-, sk_live, service_role, SECRET.

● Hit your API endpoints with the auth header removed. See what comes back.

● Confirm every table has a policy, not just RLS switched on.

The code compiles, which tells you almost nothing about whether it is safe

Veracode’s 2025 GenAI Code Security Report tested more than 100 large language models across 80 coding tasks in Java, JavaScript, Python, and C#. AI-generated code introduced an OWASP Top 10 vulnerability in 45% of cases. Java was worst at 72%. Cross-site scripting failed 86% of the time.

The follow-up matters more than the headline. In Veracode’s Spring 2026 update, syntax pass rates had climbed past 95% while security pass rates stayed flat at roughly 55%, effectively unchanged since 2023. Newer models write code that runs more reliably. They do not write code that is safer, so waiting six months will not fix this and neither will switching models.

Assume generated code is plausible and security-neutral at best. It optimises for what you asked, and nobody asks for input sanitisation.

A prototype has no memory of being wrong

The most instructive incident of the last two years is Replit’s, in July 2025. SaaStr founder Jason Lemkin was nine days into a 12-day public experiment when Replit’s agent ran destructive commands against a live production database during an explicit code freeze, deleting records covering more than 1,200 executives and around 1,190 companies, then misreported what had happened.

Replit CEO Amjad Masad called it unacceptable and shipped three fixes: automatic separation of development and production databases, better rollback, and a planning-only mode. That list names what a prototype environment lacks by default.

The lesson is not that the agent misbehaved. It is that the code freeze existed only as a sentence in a chat window. Nothing in the execution path enforced it. A rule that lives in a prompt is a request. A rule that lives in permissions is a rule.

GUARDRAILS A PROTOTYPE LACKS

"Almost right" is a category of bug that prototypes hide

Stack Overflow’s 2025 Developer Survey, with more than 49,000 respondents, named the thing every builder eventually feels. The top frustration, cited by 66%, was AI solutions that are almost right but not quite. Second, at 45%, was that debugging AI-generated code takes more time. Trust ran the other way: 46% distrust the accuracy of AI output against 33% who trust it, and only 3% report high trust, falling to 2.6% among experienced developers.

Almost-right code passes a demo perfectly, then fails on the fifth Tuesday of the month, or for one timezone, or when a name contains an apostrophe. In a prototype you never find out, because a prototype is only ever used by someone who knows what to type.

CHEAP, UNFASHIONABLE, EFFECTIVE

1. A written list of the ten things your app must never get wrong.

2. A test for each one, run automatically before every deploy.

3. For model-powered features, fixed example inputs with expected outputs, rechecked on every prompt change. Prompts are code, and editing one without a regression check is editing production by feel.

4. Error tracking from day one.

Prototypes have no unit economics

A demo runs a handful of model calls a day and the cost vanishes into a free tier. A product runs them per user, per session, sometimes per keystroke, and the bill arrives monthly.

Gartner’s June 2025 forecast expects more than 40% of agentic AI projects to be cancelled by the end of 2027. Read the three named causes closely: escalating costs, unclear business value, inadequate risk controls. Model capability is not on the list, and no better model fixes any of them.

MIT’s Project NANDA reported in July 2025 that only about 5% of custom enterprise AI tools reached production. That figure has been argued over since, but it points where everything else here points: the demo is not the hard part.

ANSWER BEFORE LAUNCH, WITH REAL NUMBERS

●  What does one active user cost per month in model calls, storage, and egress?

●  What is the ceiling if a post goes viral, and what breaks first?

●  Where is the rate limit, and what does a user see when they hit it?

●  Which features degrade gracefully when the model provider has an outage?

The speed question, answered honestly

This is where most articles pick a side. The research supports neither cleanly, and one study’s history explains why.

In July 2025, METR ran a randomised controlled trial: 16 experienced open-source developers, 246 real tasks, repositories they had worked in for an average of five years. They predicted AI would make them 24% faster. It made them 19% slower, and afterwards they still believed they had been 20% faster. The perception gap was larger than the effect.

Then METR ran it again: 57 developers, 143 repositories, more than 800 tasks, from August 2025. Returning developers now showed an 18% speedup, new recruits 4%. Publishing in February 2026, METR added an unusually candid warning that this is weak evidence, because developers who love AI increasingly refuse to join a study that might forbid it, and 30% to 50% admitted withholding tasks they did not want to do unassisted. Developers are probably faster now than in early 2025, and nobody can say by how much.

Google’s DORA 2025 report, drawing on nearly 5,000 technology professionals, adds the finding that matters most here. Adoption reached 90%, more than 80% believed AI improved their productivity, and adoption correlated with both higher delivery throughput and higher delivery instability. More shipped, more broken.

DORA’s framing is that AI amplifies whatever practices you already have, which is also the real answer to "prototype or product". If your discipline is a chat window and hope, that is what gets multiplied.

When a prototype is genuinely enough

Not everything needs to be a product, and deciding deliberately saves months.

WHAT TO DO WITH WHAT YOU HAVE

Rebuilding from scratch is usually the wrong instinct. In my case the UI, the routing, and about two thirds of the business logic survived untouched. What got replaced was the data layer, the auth flow, and anything touching money.

The crossing: a practical order of operations

Do these in order. Earlier items prevent damage the later ones cannot undo.

1.  Lock the data. Row-level rules on every table, tested from a second account. One afternoon, and the best one you will spend.

2.  Move every secret server-side. Any key in the browser bundle is public. Rotate what is already exposed.

3.  Split your environments. Production gets its own database and credentials.

4.  Turn on backups, then restore one. An untested backup is a belief.

5.  Add error tracking and uptime alerts, so users are not your monitoring.

6.  Write the ten tests for things that must never break.

7.  Add abuse controls. Rate limits per IP and per account, file size and type limits, a spend cap at your model provider.

8.  Run one external scan with the OWASP Top 10 open beside it.

9.  Do the paperwork: privacy policy, terms, a deletion path, a decision on where data lives.

10. Give someone the pager. Production without an owner is a prototype with customers.

The verdict

I still start everything in a builder, and I do not expect that to change. Getting a working shape of an idea in front of a real person the same day it occurs to you is a real change in how software gets made, and anyone saying otherwise has not sat with these tools for a week.

What changed is the story I tell myself at minute 19. I no longer think I have an app. I have an answer to one question: is this idea worth eleven weeks. That reframing has saved me more time than the generation ever did, because two of the four prototypes I built this year did not survive the question, and I found out in an afternoon instead of a quarter.

The slightly deflating truth is that the hard parts of software did not change. Access rules, error paths, money, backups, and someone accountable when it breaks at 2am were the hard parts in 2015 and they are the hard parts now. What changed is that you reach them in an afternoon instead of a month, so you meet them earlier, with less scar tissue, and often without knowing their names.

So build the prototype today, and enjoy the 19 minutes. Then open a second browser, sign up as a stranger, and try to read your own data. What you find in the next five minutes tells you which of the two things you have.