From 250+ Prototypes to Product: What I Learned

TL;DR

Most AI projects do not fail on the technology. They fail on the step from prototype to product. I have been building with AI agents daily since late 2024, by now 250+ prototypes and 10,000+ logged development sessions. The same patterns keep showing up. This article writes them down, including the part nobody likes to say: most prototypes should never become products.

The problem: why so many AI projects stall at the prototype

The sequence is almost always the same. A team builds a prototype. The demo impresses. Stakeholders get excited, budget gets approved. Then the actual work starts, and from there it gets slow. Costs go up, quality wobbles, and eventually the whole thing dissolves into rework.

I am deliberately not quoting a market study with a neat percentage here. I have no solid survey of my own, and passing estimates off as evidence is exactly what I find tiring in this field. What I do have is my own track record across many projects. That is enough for this one pattern.

The reason behind the pattern is unspectacular: a prototype and a product are two different things. A prototype has to prove an idea. A product has to run every day, with real data, real users, and the edge cases nobody thought up.

Underestimating that gap is the most expensive mistake in AI projects. If you are still at the start and want to know where AI actually pays off at all, the realistic getting-started guide gives you a structured overview.

What 250+ prototypes showed me

I do not build prototypes out of academic interest. Each one is supposed to answer a concrete question: does this approach hold? Can the problem be solved with AI at all? Is the data good enough? Three lessons came up so often that I consider them solid.

Technology is rarely the problem

That sounds wrong at first, but it holds. The models are there, they are good, and they get better every month. When a prototype does not turn into a product, the reason is almost always somewhere else.

Unclear problem definition. "We want to use AI" is not a goal. "We want invoice turnaround down from three days to four hours" is one.
Missing integration. A model that works in isolation inside a notebook is worthless as long as it does not sit inside a real workflow.
No plan for operations. A prototype needs no maintenance. A product needs monitoring, error handling, and someone who understands it when things tip over at night.
Organizational friction. The best solution fails when the people who are supposed to work with it first hear about it at rollout.

So I never start with the technology, I start with the problem. What exactly should be solved? Who is going to use it? And what happens when the AI gets it wrong?

Data decides success or failure

If I could take just one sentence away from 250+ prototypes, it would be this one: the quality of your data sets the ceiling of your AI solution.

No model compensates for bad data. And "bad" rarely means obviously broken. Usually it means inconsistent formats, invoices arriving as PDF, as a scan, or as a photo taken on a desk. Addresses in ten spellings. Documents without metadata, processable but error-prone.

An example from my own work: in BuchhaltGenie, my AI-assisted accounting tool, text recognition on receipts was the central building block. The first recognition rate was usable but not production-ready. What made the difference was not the model but the preprocessing: normalize documents up front, post-process results in a structured way. The result was a 71% improvement in OCR accuracy. Same model, better data.

The leap from prototype to product

Here is the honest part: very few of my prototypes should become products. That is not failure, that is the point. A prototype answers cheaply and quickly whether an idea holds.

In a prototype the question is "does this work in principle?". In a product the questions are different:

Reliability. Not 90% of cases correct, but a rate you can live with, plus a clear strategy for the rest.
Scalability. What runs with 100 documents has to run with 10,000.
Maintainability. Who looks after this when the data changes or a model gets deprecated?
Failure behaviour. A prototype may crash. A product has to fail cleanly and say what went wrong.
Law and compliance. GDPR and the EU AI Act are not a side stage, they are part of the architecture. I completed the Article 4 AI-literacy training with certificate, which does not replace legal advice but helps to ask the right questions early.

That leap costs a multiple of the prototype effort. If you do not plan for it, you are planning to fail.

10,000+ sessions: what is behind the number

A number I mention often: more than 10,000 logged development sessions. That is not a brag, it is a counting method. A session is a focused block of work: solve one problem, build one feature, corner one bug, make one architecture decision.

What comes out of that volume is pattern recognition. You cannot read your way to it, you have to work your way there.

1. Iteration beats perfection.

The first draft is almost never the best one, but it is the precondition for the second. In BuchhaltGenie the codebase started with more than 600 TypeScript errors. Session by session the number came down to under 50. Not in one big sweep, but in many small controlled steps.

2. Patterns repeat.

After enough projects you know the same recurring traps: the same integration problems, the same data quality, the same handovers that get stuck. That speeds up every following project, because you spot what you are heading into earlier.

3. Documentation is not a luxury.

Within one session, memory is enough. Across thousands of sessions it is not. Every decision, every discarded approach belongs on record. Not for posterity, but for the next session.

When a prototype should not become a product

This is the section that matters most to me, because it argues against my own order book. There are warning signs where I advise against going on.

The prototype only runs under lab conditions. If it only works with hand-picked test data, what you have proven is that you have good test data. Ask what happens with real, unprepared inputs. If results drop noticeably there, that is a serious signal.

Cost per request does not carry. Every model call costs something. Do the math on your monthly volume before you scale. It is one line in a spreadsheet and it sometimes saves you half a year.

Nobody owns it after go-live. Who watches the quality? Who reacts when it drops? If nobody can answer that, the product goes stale within a few months.

The benefit does not justify the effort. Sometimes the manual process is simply cheaper. That is not embarrassing, it is a good decision. Not every problem needs AI just because it would be technically possible.

Stopping a prototype is not a burned investment. You got an answer, it just happens to be "no, not like this". That saves you the far higher cost of a failed product.

From prototype to product: a realistic roadmap

If the prototype phase gives a clear yes, you need a structured transition. The following sequence has held up across my projects. The durations are ranges from my own practice, not commitments: depending on the state of your data, they shift a lot.

Phase 1: Validation (1 to 2 weeks)

Before money goes into product development, validation has to hold:

Does the prototype run on real data, not just on test data?
Does cost per request carry at realistic volume?
Which regulatory requirements apply, for example GDPR and the EU AI Act?
Is the team ready to use and maintain the solution?

Plus the hard question: at what hit rate is this useful at all? Write that down in one sentence with a number in it, otherwise you will be arguing about feelings later.

Phase 2: Architecture and operability (2 to 4 weeks)

Prototype code does not get tidied up, it gets rethought:

Error handling for the edge cases you know, and defined behaviour for the ones you do not
Monitoring and logging for production
Scalable infrastructure
Access control and data retention
Automated tests as a gate, not as decoration

Phase 3: Iterative development (4 to 8 weeks)

You build in short cycles, not in one monolithic push:

Regular releases with real user feedback
Continuous measurement of AI quality against the number from phase 1
Adjusting prompts and parameters based on real results
Documentation for operations and maintenance

Phase 4: Handover and operations (ongoing)

The best AI product is worthless once it goes stale after handover:

Training for the internal teams
Runbooks for the frequent problems
Escalation paths that somebody actually knows
Regular quality reviews in the calendar

What this costs depends on scope. I name a fixed price after the scoping conversation and not before, because a number without an agreed scope is worthless for both sides.

Conclusion

Out of 250+ prototypes and 10,000+ sessions, five points remain:

Start with the problem, not with the technology. The question is not "where can we use AI", it is "which problem do we want gone".
Invest in your data. Data quality is unglamorous and still decides success or failure.
Plan the step to product from the start. As its own phase with its own effort, not as an afterthought.
Have the nerve to stop a prototype. A timely no saves more than a late one.
Iterate. Small steps, fast feedback. That holds for AI just as it does for any other software.

If you are deciding whether a prototype of yours should go into production, write to me via the contact section. I will tell you honestly when I am the wrong person for it.