The astonishing growth of AI Code generation engines
| Loveable | $10m ARR in 60 days with 15 people (listen to Lenny’s podcast with the founder, Anton Osika) |
| Bolt | $40m ARR in 5 months with less than 20 people (listen to Lenny's podcast with the founder, Eric Simons) |
My definition of an AI code generation engine
They use generative AI to take some requirements and to squirt out fully functional code – right into your lap. People have been using ChatGPT to help them correct their code or to hook up APIs. Lovable and Bolt punch through to a new ceiling, IMHO.
lovable - What I did
In 20 minutes, I typed in a specification for a wizard consisting of 6 webpages to achieve an objective. It was about three quarter of a page long with lots of headings and bullet points. I cut and paste this into Loveable’s super simple prompt.
The spec was reasonably clear to someone with my industry, but it definitely needed context.
What happened next
There was some processing. And then, a stream of code appeared: first the JavaScript, then the style sheets and finally the HTML of each of the six pages.
When the dust had settled, there was a preview button. I can’t say this button worked first time. I had asked for the results to be delivered as a zip file (not knowing that there was preview functionality). It tried to render the zip file in a browser and failed. My bad.
After a bit of tinkering with the spec, I bashed in my v2 instructions. And BANG – a functional webpage, nicely styled, with sensible prompts, populated with sample data.
Whoa! Ready to publish? Do you want to purchase a domain? Are you ready to launch your new business??
Totally lovable – hence the name.
Bolt.app
Bolt was even more impressive. Here's a better review than I can do: Bolt AI: Build, Fix, and Deploy Full Stack Apps Faster Than Ever!
I had my specifications in PowerPoint because I had previously built some sample layouts to explain my wants to a developer. Consuming PowerPoint was a fail. So I converted a table in PowerPoint with sample data into text that said, “The table should have the following fields: …..”. (Another learning point.)
You can specify the programming language and / or tech stack that you’d like to generate. I created a supabase account – an online database – and hooked it up to my Bolt app. I was also obliged to purchase 20M tokens per month for $20.
What happened next
Database instructions appeared. Would I like to put these into supabase? Yes, please. Boom – tables substantiated.
Code generated next. Preview. BOOM – an application appears. When I interacted with the app, I could data appearing in supabase in another browser window. Corks - this is for real! I could edit data in the tables, create new rows, all of which was immediately visible in the app. Wonderous!
All for $10, possibly even less if I hadn't spent so much time messing around with PowerPoint fails!
My conclusions so far and some crystal balling
This tech is perfect for green field projects: super rapid prototyping of concepts and ideas. Next up will be enhancing existing tech stacks with improvements. This will require deep integration with github and the implementation of a corporate knowledge library (design considerations, technical mandates (use this tech or we have a license for this technology) or tech standards and red lines) or industry-speak or corporate euphemisms.
What WILL happen next to Product Management, Engineering and Testing
Well, very interesting indeed. Developers are still the all-powerful supremos in any tech company – they were the highest-paid and the rarest resource. Their pedestal has been shortened somewhat. They are still essential: it is plainly obvious that the output of these tools won’t work perfectly: prior integrations exist already. Security & maintainability are critical.
BUT profound understanding of the business requirements and deliverables have always been important. This code-gen technology makes that even more important if, in 5 minutes, you can build your own deliverable. Product Management will change – for me it already has.
New Role: Product AI Gen Engineer
Cast forward, I think there will be a new role generated in the product development organisation: I haven’t got a good name yet, but Product AI Gen Engineer, perhaps?
These people are a combination of Product Manager, Business Analyst, Prompt Engineer and Tester. Let’s break down the job description.
Business Analyst
An excellent understanding of the requirements of the functionality and what the delivered results should look like. Clear unambiguous specifications have never been in greater demand. From my own experience with Lovable, then the ambiguity or a critical typo (even in my case, formatting of bullet points!) really made a difference to the net result.
Prompt Engineer
This is a new skill that dovetails the business requirements with experience of using a particular AI tool: knowing how to surgically correct instructions without messing up everything else. It does come back to unambiguity.
Do you remember when Internet Search became available: some people became experts at entering search terms to find the right information: it was a balance of specifying the topic of interest with distinctive keywords which would return the result set in which the answer to your question could be found. It will be the same with these AI code generating skills.
Acceptance Tester
There will be a very tight cycle: requirements, delivery, review, respecify. Functional Acceptance testing will be a skill that needs to be glued onto to the end of Product Manager skillset. AI generated functionality, from I can see, delivers lots of common sense: there’s field level checking, but it is the functional flow through multiple steps where the AI is likely to brittle – and often as a result of ambiguity in the specifications.
Conformity / Continuity Validations: Given the sensitivity of the delivered results to the specifications, there will be a renewed emphasis on making sure that, towards the end of the development cycle when minor tweaks are being asked for, that it is only the tweaks that are changed and not unanticipated knock-on effects. Code modification analysis between each build becomes really important. Bolt has this, but this, in itself, requires AI to interpret. Regression testing is critical.
AI can be used for these two tasks (code generation and testing), but I assume that you will want to use two different AIs to do these two tasks, so that you don’t get over-fitting between developer and tester.
No comments:
Post a Comment