Before AI for the last 8 or so years now first at a startup then working in consulting mostly with companies new to AWS or they wanted a new implementation, it’s been:

1. Gather requirements

2. Do the design

3. Present the design and get approval and make sure I didn’t miss anything

4. Do the infrastructure as code to create the architecture and the deployment pipeline

5. Design the schema and write the code

6. Take it through UAT and often go back to #4 or #5

7. Move it into production

8. Monitoring and maintenance.

#4 and #5 can be done easily with AI for most run of the mill enterprise SaaS implementations especially if you have the luxury of starting from the ground up “post AI”. This is something you could farm off to mid level ticket takers before AI.

But there's not one tool there that triggered a major boost in output or number of apps / libraries / products created - unless I missed something.

Sure, total output has increased, especially since the early 2010's thanks to both Github becoming the social network of software development, and (arguably) Node / JS becoming one of the most popular languages/runtimes out there attracting a lot of developers to publish a lot of tools. But that's not down to productivity or output boosting developments.

https://safebots.ai

But it requires A LOT of work to make sure it is actually safe for people and organizations. And no, an .md file saying “PLEASE DONT PWN ME, KTHX” isn’t it at all. “Alignment” is only part of the equation.

If you’re not afraid to dive into rabbitholes, here is how it works: http://community.safebots.ai/t/layer-4-browser-extensions-pe...

Yup. And for most purposes, that's enough. An app does not have to be productized and shipped to general audience to be useful. In fact, if your goal is to solve some specific problem for yourself, your friends/family, community or your team, then the "last step" you mention - the one that "takes majority of time and effort" - is entirely unnecessary, irrelevant, and a waste of time.

The productivity boost is there, but it's not measured because people are looking for the wrong thing. Products on the market are not solutions to problems, they're tools to make money. The two are correlated, because of bunch of obvious reasons (people need money, solving a problem costs money, people are happy to pay for solutions, etc.), but they're still distinct. AI is dropping the costs of "solving the problem" part, much more than that of "making a product", so it's not useful to use the lack of the latter as evidence of lack of the former.

Having worked extensively with vibe-coded software, the main problem for me is that I have tuned-off from the ai-code, and I dont see any skin-in-the-game for me. This is dangerous because it becomes increasingly harder to root-cause and debug problems because that muscle is atrophying. use-it or lose-it applies to cognitive skills (coding/debugging). Now, I lean negatively to ai-code because, while it seduces us with fast progress in the first 80%, the end outcome is questionable in terms of quality. Finally, ai-coding encourages a prompt-and-test or trial-and-error approach to software engineering which is frustrating and those with experience would prefer to get it right by design.

Just doesn't have the same ring to it.

This is true, and I bet there are thousands of people who are in this stage right now - having gotten there far faster than they would have without Claude Code - which makes me predict that the point made in the article will not age well. I think it’s just a matter of a bit more time before the deluge starts, something on the order of six more months.

-- we had a terrible time building something so now we're only going to buy things

-- we had a terrible time buying something so now we're only going to build things

-- repeat...

Either way you can have a brilliant success and either way you fail abjectly, usually you succeed at most but not all of the goals and it is late and over budget.

If you build you take the risks of building something that doesn't exist and may never exist.

If you buy you have to pay for a lot of structure that pushes risks around in space and time. The vendor people needs marketing people not to figure out what you need, but what customers need in the abstract. Sales people are needed to help you match up your perception of what you need with the reality of the product. All those folks are expensive, not just because of their salaries but because a pretty good chunk of a salesperson's time is burned up on sales that don't go through, sales that take 10x as long they really should because there are too many people in the room, etc.

When I was envisioning an enterprise product in the early 2010s for instance I got all hung up on the deployment model -- we figured some customers would insist on everything being on-premise, some would want to host in their own AWS/Azure/GCP and others would be happy if we did it all for them. We found the phrase "hybrid cloud" would cause their eyes to glaze over and maybe they were right because in five years this became a synonym for Kubernetes. Building our demos we just built things that were easy for us to deploy and the same would be true for anything people build in house.

To some extent I think AI does push the line towards build.

1. Debugging my own code or obvious behavior with other libraries.

2. Debugging pain-in-the-ass behavior with other libraries.

My patience with the latter is significantly less now, and so is perhaps my skill in debugging them. Libraries that change their apis for no apparent reason, libraries which use nonstandard parameter names, libraries which aren’t working as advertised.

I've started tons of scratch my own itch projects. There's adoption, UX, onboarding costs even if you're the only audience.

TLDR: i don't even use my own projects. I churn.

IMHO (this may not apply to you!) a lot of people launch a "competitor" of a product which seems to be a clone of the product without improving something that the other product misses/is very bad at.

I do think there would be value in sharing your setup at some point if you get around to it, I think a lot of builders are in the same boat and we're all trying to figure out what the right interface for this is (or at least right for us personally).

0. It runs way too fast and far ahead. You need to slow it down, force planning only and explicitly present a multi-step (i.e. numbered plan) and say "we'll do #1 first, then do the rest in future steps".

take-away: This is likely solved with experience and changing how I work - or maybe caring less? The problem is the model can produce much faster than you can consume, but it runs down dead ends that destroy YOUR context. I think if you were running a bunch of autonomous agents this would be less noticeable, but impact 1-3 negatively and get very expensive.

1. lots of "just plain wrong" details. You catch this developing or testing because it doesn't work, or you know from experience it's wrong just by looking at it. Or you've already corrected it and need to point out the previous context.

take-away: If you were vibe coding you'd solve all these eventually. Addressing #0 with "MORE AI" would probably help (i.e. AI to play/validate, etc).

2. Serious runtime issues that are not necessarily bugs. Examples: it made a lot of client-side API endpoints public that didn't even need to exist, or at least needed to be scoped to the current auth. It missed basic filtering and SQL clauses that constrained data. It hardcoded important data (but not necessarily secrets) like ports, etc. It made assumptions that worked fine in development but could be big issues in public.

take-away: AI starts to build traps here. Vibe coders are in big trouble because everything works but that's not really the end goal. Problems could range from 3am downtime call-outs to getting your infrastructure owned or data breaches. More serious: experienced devs who go all-in on autonomous coding might be three months from their last manual code review and be in the same position as a vibe coder. You'd need a week or more to onboard and figure out what was going on, and fix it, which is probably too late.

3. It made (at least) one huge architectural mistake (this is a pretty simple project so I'm not sure there's space for more). I saw it coming but kept going in the spirit of my experiment.

take-away: TBD. I'm going to try and use AI to refactor this, but it is non trivial. It could take as long as the initial app did to fix. If you followed the current pro-AI narrative you'd only notice it when your app started to intermittently fail - or you got you cloud provider's bill.

https://addyosmani.com/blog/comprehension-debt/

To a point, but I think this overstates it by quite a bit. At the moment I'm weighing some tradeoffs around this myself. I'm currently making an app for a niche interest of mine. I have a few acquaintances who would find it useful as well but I'm not sure if I want to take that on. If I keep the project for personal use I can make a lot of simplifying decisions like just running it on my own machine and using the CLI for certain steps.

To deploy this to for non-tech users I need to figure out a whole deployment approach, make the UI more polished, and worry more about bugs and uptime. It sucks to get invested in some software that then constantly starts breaking or crashing. GenAI will help with this somewhat, but certainly won't drop the extra coding time cost down to zero.

But based on the hype (100x productivity!), there should be a deluge of high quality mobile apps, Saas offerings, etc. There is a huge profit incentive to create quality software at a low price.

Yet, the majority of new apps and services that I see are all AI ecosystem stuff. Wrappers around LLMs, or tools to use LLMs to create software. But I’m not really seeing the output of this process (net new software).

Emacs with Hyperbole[0]?

[0]: https://www.gnu.org/software/hyperbole/

Do you never open a code editor?

They can only be interested in one thing, self-advancement. No other explanation works! If they were interested in self-improvement, they might try reading or writing something themselves! Wouldn't it show if they had?

I recognize that models are getting better, but consider: if you already don't understand how programming or LLMs work, and you use LLMs precisely to avoid knowing how to do things, or how they work (the "CEO" mode), each incremental improvement will impress you more than it impresses others. There's no AI exception to Dunning-Kruger.

I recognize that "this" is a difficult thing to pin down in real time. But in the end we know it when we see it, and it has the fascinating and useful quality of not really being explainable by anything else.

Unless and until the culture gets to a place where no one would risk embarrassing themselves by doing something like this, we're stuck with it.

You can get a throw away app in 5 mins, before I wouldn't even bother.

Before that, it single-shot an app for me where I can copy-paste a table (or a subsection of it) from Excel and print it out perfectly aligned on label sticker paper; it does instantly what used to take me an hour each time, when I had to fight Microsoft Word (mail merge) and my Canon printer's settings to get the text properly aligned on labels, and not cut off because something along the way decided to scale content or add margins or such.

Neither of these tools is immediately usable for others. They're not meant to, and that's fine.

I vibe'd a basic ticketing system in just under an hour that does what we need. So not 20 mins, but more like 45-60.

For 20 minutes of time, I had a simple TTS/STT app that allows me to have a voice conversation with my AI assistant.

The amount of useless slop in the app store doesn't matter. There are no new and useful apps made with AI - apps that contribute to productivity of the economy as whole. The trade and fiscal deficits are both high and growing as is corporate indebtedness - these are the true measures for economic failure and they all agree on it.

AI is a debt and energy guzzling endeavor which sucks the capital juice out of the economy in return for meager benefits.

I can't think of a reason for the present unjustified AI rush and hype other than war, but any success towards that goal is a total loss for the economy and environment - that's the relation between economics and deadly destruction in a connected world, reality is the proof.

Really the one thing that conclusively has changed is that the 'ask it on stackoverflow' has become 'ask it an LLM'. Around 95% of the stackoverflow questions can be answered by an LLM with access to the documentation, not sure what will happen to the other 5%. I don't think stackoverflow will survive a 20-fold reduction in size, if only because their stance on not allowing repeat questions means that exponential growth was the main thing preventing them from becoming stale.

In that they obviously have no real utility, sure. There hasn't been a paradigm shift, they still suck at programming, and anyone trying to tell you otherwise almost certainly has something to sell you.

The premise is that AI has already fundamentally changed the nature of software engineering. Not some specific, personal use case, but that everything has changed and that if you're not embracing these tools, you'll perish. In light of this, I don't think your rebuttal works. We should be seeing evidence of meaningful AI contributions all over the place.

Detractors of AI are often accused of moving the goalposts, but I think your comment is guilty of the same. Before Claude Code, we had Cursor, Github Copilot, and more. Each of these was purportedly revolutionizing software engineering.

Further, the core claim for AI coding is that it lets you ship code 10x or 100x faster. So why do we need to wait years to see the result? Shouldn't there be an explosion in every type of software imaginable?

There's a very real problem of low effort AI slop, but throwing out the baby with the bathwater is not the solution.

That said, I do kind of wonder if the old model of open source just isn't very good in the AI era. Maybe when AI gets a lot better, but for now it does take real human effort to review and test. If contributors were reviewing and testing like they should be doing, it wouldn't be an issue, but far too many people just run AI and don't even look at it before sending the PR. It's not the maintainers job to do all the review and test of a low-effort push. That's not fair to them, and even discarding that it's a terrible model for software that you share with anyone else.

I have a list of ideas a mile long that gets longer every day, and LLMs help me burn through that list significantly faster.

However, the older I get, the more distraught I get that most people I meet "IRL" are simply not sitting on a list of problems they simply lack time to solve. I have... a lot of emotions around this, but it seems to be the norm.

If someone doesn't see or experience problems and intuitively start working out how they would fix them if they only had time, the notion that they could pair program effectively ideas that they didn't previously have with an LLM is absurd.

There are many small, different, and one-time tasks that don’t fit full blown apps. Which I would characterize an AI building a novel app as building a house out of random bits of lumber. It will work but will have no cohesive process and sounds like a nightmare.

Would it be harder? Sure. And perhaps the difficulty adds an additional cost of passion being a necessary condition to embark on the innovation. Passion leads to really good stuff.

My personal fear is we get landfill sites of junk software produced. To some extent it should be costly to convert an idea to a concept - the cost being thinking carefully so what you put out there is somewhat legible.

I'd rather not pay monthly for something (like water) that I'm turning on and off and may not even need for weeks. But paying per-liter is currently more expensive so that's what we currently do.

I think the future is going to be local models running on powerful GPUs that you have on-prem or in your homelab, so you don't need your wallet perpetually tethered to a company just to turn the hose on for a few minutes.

Right.

I don't think you even need cynicism or whatever you felt you were having impolite thoughts about:

I'd expect the top mature libraries to be the most resistant to AI tool use for various reasons. They already have established processes, they don't accept drive-by PR spam, the developers working on them might be the least likely to be early adopters, and -- perhaps most importantly -- the todo list of those projects might need the most human comms, like directional planning rather than the sort of yolo feature impl you can do in a one-man greenfield.

All to further bury signals you might find elsewhere in broader ecosystems.

I get that people are upset that making a cool six figures off of stitching together React components is maybe not a viable long-term career path anymore. For those of us on the user side, the value is tremendous. I’m starting to replace what were paid enterprise software and plug-ins and tailoring them to my own taste. Our subject matter experts are translating their knowledge and work flows, which usually aren’t that complicated, into working products on their own. They didn’t have to spend six months or a year negotiating an agreement to build the software or have to beg our existing software vendors, who could not possibly care less, for the functionality to be added to software we are, for some reason, expected to pay for every single year, despite the absence of any operating cost to justify this practice.

Maybe I should have said "obvious to me," but I guess I just struggle to see how a serious crack at using modern opus in claude code doesn't make it obvious at this point.

I'd really recommend trying the "spike out a self-contained minimal version of this rearchitecture/migration and troubleshoot it iteratively until it works, then make a report on findings" use-case for anyone that hasn't had luck with them thus far and is serious about trying to reach conclusions based on direct experience.

Yep, also a huge factor. Why publish something you built with an AI assistant if you know it's going to be immediately dunked on not because the quality may be questionable, but because someone sees an em-dash, or an AI coauthor, and immediately goes on a warpath? Heck I commented[0] on the attitude just a few hours ago. I find it really irritating.

[0] https://github.com/duriantaco/fyn/issues/4#issuecomment-4117...

What's sauce for the goose is sauce for the gander. If you make that argument that 'I don't believe in kinks or discontinuities in code release due to AI, because so many AI coding systems have come out incrementally since 2020', then OP does provide strong evidence for an AI acceleration - the smooth exponential!

This is flat-earther level. It's like an environmentalist saying that nothing made with fossil fuels contributes to productivity. But they don't say that because they know it's not true.

There are so many valid gripes to have with LLMs, pick literally any of them. The idea that a single line of generated code can't possibly be productivity net positive is nonsensical. And if one line can, then so can many lines.

Having your code snatched and its copyright disregarded, to the benefit of some rando LLM vendor. People can just press "pause" and wait until they see whether they fuel something that brings joy to the world. (Which it might in the end. Or not.)

    curl 'https://cdn.statcdn.com/Statistic/1020000/1020964-blank-754.png' \
      -H 'Origin: https://www.statista.com' \
      -H 'Referer: https://www.statista.com/' \
      --output chart.png

Relevant! If the maximalist interpretation of AI capabilities were close to real, and if people tend to point their new super powers at their biggest pain points.. wouldn't it be a big blow for all things advertising / attention economy? "Create a driver or wrapper app that skips all ads on Youtube/Spotify" or "Make a browser plugin that de-emphasizes and unlinks all attention-grabbing references to pay-walled content".

If we're supposed to be in awe of how AI can do anything, and we notice repeatedly that nope, it isn't really empowering users yet, then we may need to reconsider the premise.

Seems like the rest of the whole AI business, the only things going to the top are the AI tools themselves but not the things they are supposed to built.

1. To benefit myself with features/projects

2. To benefit others with my work

1 by itself would mean no bothering with PR, modifications, etc. It's way easier to hoard your changes than to go through the effort getting them merged upstream. 2 by itself isn't enough motivation to spend the effort getting up to speed on the codebase, testing, etc. Together though, it's powerful motivation for me.

I have to remind myself that both things are a net positive with AI training on my stuff. It's certainly not all pros (there's a lot of cons with AI too), but on the whole I think we're headed for a good destination, assuming open models continue to progress. If it ends up with winner-takes-all Anthropic or OpenAI, then that changes my calculus and will probably really piss me off. Luckily I've gotten positive value back from those companies, even considering having to pay for it.

Ok, so do you have a counterexample?

The moat used to be effort, nobody wants to rewrite this from scratch (especially when it's free). What's left is actually understanding why the thing works the way it does. Not sure that's enough to sustain open source long-term? I guess we all have to get used to it?

https://apps.apple.com/us/app/kaien/id6759458971

Indeed, I've got a few applications I've built or contributed too that are (A)?GPL, and for those I do worry about this AI washing technique. For libraries that are MIT or permissive anyway, I don't really care. (I default to *GPL for applications, MIT/Apache/etc for libraries)

The AI wouldn't have been able to do it by itself, but I wouldn't have been arsed to do it alone either.

If that would be true then all these AIs are useless. Who needs them to built something that already exists?

Ah my favorite, entirely made up quote.

Apocraphyly attributed to the U.S. Patent Office Commissioner in 1899.