20-40% sounds about right for me, today. Maybe 40-60% on a good day. But a lot of the reason it's not higher comes from harness gaps and org processes that haven't caught up.

All of that will get fixed with time.

OpenAI's spending commitment is in the ~1T range for the next 5 years, and Anthropic is ~300B.

If they continue to show strong growth, they likely need to be at 100-300B in revenue/yr to support their yearly payments + financing, not 1T.

Thats why most here shouldn’t engage in the discussion - they parrot on about benefits without identifying and articulating the costs and moreover how it affects the firms financial position.

- The publicly available information about how inference costs compare to training costs is conflicted. EEs involved in datacenters talk about power usage spikes during training runs as if they were a major factor in the designs, but academic papers discussing cost-optimal scaling confidently treat inference-time compute as a major factor.

- On the side of the balance indicating that training is more compute-intensive after amortization than inference is that Chinese providers, constrained primarily by access to compute, have nearly unlimited token availability at a lower price than US providers (inference), but poorer model capabilities (training). That would make sense only if US providers are inflating inference costs by 20-30x due to amortized training costs that overseas providers were not able to take on (there are other factors too).

- If training >> inference, they're in a prisoner's dilemma that far exceeds the ordinary zero-marginals model of competition between firms (due to its huge discrete stepwise nature). On the other hand, if inference>>training, the high-level analysis popularized by certain thought leaders, that it's like a utility, would be true. You'd tend to count this as a vote for inference>>training, but the CEOs saying it at least have a huge incentive to agree because the alternative, the prisoner's dilemma, would stop investment very fast.

- The only voice in the story that I just told you to have anything to do with fact (as opposed to high-level analysis and ivory tower armchair management of a secretive business) were the rumors from facilities engineers. That shows you the state of our understanding...

- If we don't even know the ratio between amortized capital expenses and operational costs, outside investor analysis is impossible. It doesn't matter how finely they divide the accounting buckets for office ferns and indoor ferns if the single biggest part of their business is obscured for trade secret reasons.

Our estimated spend for AIaaS would exceed that cost in less than a year.

In a few years, there will be hardware capable of running frontier models good enough for most things at accessible prices for even tiny companies.

"As cable TV and Pay Per View came out, there were studies done about how many movies people would watch if given unlimited access to films. The results were bandied about as proof that we should build out all this infrastructure to support this line of business. When the data was further analyzed by statisticians etc, it turned out that people claimed they were going to watch films 10-12 hours a day, every day of the week. Impossible."

I feel like we are in a similar boat here where some people are assuming:

- EVERYONE is going to be using max tokens

- tokens will NEVER get cheaper due to improvements in hardware, software, design, market forces etc etc

That's the game. There's a view you could take of this that this is just a growing of the pie: with those cost dynamics a lot more "small businesses" get a vast amount of leverage, so the overall economy grows without replacing the knowledge workers. I'm not sure I trust the MBA class to have that view.

What are you basing this on? For reference, Anthropic raised ~$70 billion in total and OpenAI ~$190 billion. Why do they need to make 20-40x that?

And that's just one inflection point. We've had several and there are many more on the horizon. So while I could be convinced that ROI is maybe not even positive today despite the ridiculous enterprise spend, it's perfectly rational to pave the way today for what's coming over the next few months let alone years down the line.

I think it was clearly useful for months to people who had tried it and taken the time to understand it, but now that knowledge has spread to the point where wallet holders are convinced it's not just passing fad or hype so now pmf can be "claimed".

I agree it's weird to say "those people have pmf" though, usually it's something you define for yourself

"I’ve called November 2025 the November inflection point because that was when GPT-5.1 and Opus 4.5, combined with their respective coding agent harnesses, got good—good enough that we’ve spent the last six months adapting to agent systems that can reliably get useful work done."

Most of the money right now is in coding. Openai and Anthropic just have to be 6 months ahead of SOTA open source models and they'll capture most of the enterprise and dev market

Same. It's a nightmare from a Porter's Five Forces perspective.

There will be a ton of businesses competing in this space, and there will be something of a moat due to how capital intensive the business can be, but there will still basically be infinite competitors.

Great for consumers.

I agree with the common trope that open models lag behind by about a year, but something magical happened just around a year ago when the state of the art models became extremely useful. By this reasoning we're about to see open models perform well, but I'm afraid there is more to it than just waiting for another revolution around the sun.

Note, my application is coding assistance. Open models can be great for other purposes.

We all have our own observations and mine don’t significantly diverge. But that’s bottom up. At this point shouldn’t we be seeing it top down?

If we are beyond potential and into significant productivity gains, why isn’t that showing up for the customers?

Why didn’t delta airlines get significantly more operationally efficient in the last 3 months due to the introduction of better software?

This is a genuine question, I am seeing a disconnect.

It's not exactly news, is what I'm saying. And even with the PMF they found, the product is still only a commodity i.e `tokens`, which is what every other provider on the planet is also providing.

All their other products boil down to "harnesses", which does not look viable as a product in the sense of PMF - you cannot sell it, you cannot lock it to your own subscription, API, etc. so you can't use it to generate revenue any more than the free harnesses do.

PMF has a specific meaning, and "code harness" or "coding model" does not satisfy the commonly accepted meaning. Maybe Mythos (or similar) will.

How enormous? 1 trillion dollars, 2, 10 trillion enormous?

And yet we surely need this data for the IPO? Or are they relying on rule changes on the indexes to force ETFs to buy shares?

It's a given that the SOTA models need to raise their prices. It's also a given that they can't. The more they raise the more customers will move to their competition.

So what happens next? Well I think it will suck horribly if you can't move off of SOTA sooner or later, because the Big Two are going to lose customers, and therefore have to raise prices on the locked in customers even more than these projections suggest.

Beyond that if you're looking to start a business, figure out how to use cheap models in new scenarios. Build software which does that and license it. This is kind of contrary to the idea that you shouldn't over optimize for deficiencies in the models that will likely go away in the next generation - for instance a lot of problems were solved when context windows got way bigger. So it's a thin line to walk but I think it's there because a lot of orgs are using Claude today for pretty basic tasks.

The dev who's addicted to SOTA models honestly is going to have to settle for less or get totally screwed. Most applications within business from what I see aside from complex research do not require SOTA. They summarize, they classify, they transform, and doing that accurately has been cheap for a while.

I realized it long ago: one needs output to make meaning. Input can only be the cherry on a cake in one's life. That, actually, makes FIRE or Fat FIRE not so sustainable unless one has other hobbies.

I feel like the reverse assumption is being made, that the current model looks like IBM doubling down on Mainframes soon to become cheap enough to deploy everywhere, when the real action is that the costs coming down represents cheaper hardware or more efficient software, and that a big chunk of "cheaper" AI will be eaten by smaller products deployed by individuals. Whatever the Personal Computer of AI looks like is going to be more disruptive than just an API endpoint you can fling tokens at.

We already see this with things like chrome auto installing an LLM.

You cant tell me with complete certainty that theres a moat here for the people spending 1 trillion + on this infra.

>When the data was further analyzed by statisticians etc, it turned out that people claimed they were going to watch films 10-12 hours a day, every day of the week. Impossible.

I also think this applies to people suggesting that companies will sack workers for AI, when the costs of replacing everything someone does in a day is more expensive in terms of tokens (likely even at a reduced price) than just hiring a bloke.

A lot of these LLM demand scaling scenarios make broad "up and to the right" assumptions about things which in practice have finite limits. Only some percentage of knowledge work benefits from acceleration, optimization or other improvements, and even then the amount of economic gain is capped.

anthropic already hunts down OpenClaw users for using too much on their plan.

I'll give different example: When LED lights started to be more popular, the power usage didn't drop by the amount of power saved

>- tokens will NEVER get cheaper due to improvements in hardware, software, design, market forces etc etc

Well, first, improvements in computing stalled or even rolled back just purely because price of everything compute shot up cos of AI and that will NOT be fixed for a while and ESPECIALLY if AI usage will continue to increase

Second, the token per model might go down in time but better models have more expensive tokens, so we quickly get into spot when:

* price increase in token might not be worth marginal improvement next, better model brings

* more and more models are passing "good enough for the task" threshold so for less and less companies there is any economic sense to pay for the "best" instead of paying deepseek or some other company to run "previous gen" models

“There’s more capital than good ideas to fund” has been a complaint from the likes of A16z & other VCs for a long time now. It’s why we ended up with stuff like NFTs getting funded.

i am pretty sure these services know what it truly costs them to serve you tokens, maybe not in realtime but at least periodically.

however, what they charge us is a constant exercise in price discovery. i agree with this sentiment in the sense that we don't have a stable sense of the cost. all of these comparisons are good for the moment, or at most the near future.

i believe that even the "all you can eat" approach with the max plans, regardless of their crazy pricing, is not sustainable only with the power users. if most of us gets this kind of value through our plans, surely it does not incentivise the service providers to continue pushing it. maybe they can regardless just to gain market share, but not forever.

* At some point model capability reaches diminishing returns. Then inference >> training in the future but training >> inference now. It’s not a prisoner’s dilemma but a land grab to solidify market position and be one of the 2-3 firms left standing as dominant in the space. The model companies aren’t super sticky yet but they’re working on it.

* even if training remains >> inference, it’s possible to have multiple price points like they do today. If you need the most capable model you’ll be paying exponentially more per token to supplement the training cost even though the serving cost is marginal because most people will be satisfied with cheaper / less capable models for most tasks.

I buy that inference is a dropping line item while training is a growing one. There’s all sorts of things on the horizon that’ll be order of magnitudes improvements, from startups burning models into ASICs to get order of magnitudes more performance to alternate architectures like diffusion transformers that have orders of magnitude structural optimizations. It’s inevitable that it’ll come down even further from where we are. It’s possible model training also will go down but I’ve not seen any compelling research suggesting major “easy” reductions here.

Yes I know there's no evidence and this is lazy reasoning. But there's probably a bit of truth to this line of thought.

https://www.gpunex.com/blog/ai-inference-economics-2026/

Training involves multiple passes over the entire training dataset, ideally in large batches where you can perform inference on as many samples as possible simultaneously and then perform backpropagation to adjust the model weights (which is about as expensive as inference).

Let's consider the size of the dataset we're dealing with here. The dataset likely consists of practically every piece of digitized text they can get their hands on (including that extracted from audio and video). We know Google has digitized a large portion of the books in existence as part of their "search book contents" feature and we have no reason to believe they're not using it alongside their cache of 90+% of the internet to train their models. We're talking about 100s of millions of books each with an average of 100,000s of tokens. The internet has 10s to 100s of billions of pages on it with who knows how many tokens on average. This is a huge dataset that we've got to go through hundreds of times.

Second, let's consider the effect of batching and how it sets requirements for our hardware. We know that larger batch sizes converge faster, are more stable, and produce better models. So if you want a good model you need large batch sizes. This means that you need machines several orders of magnitude more powerful than you use for inference. From what I heard Google uses clusters of 100s of the their TPUs all located in a single rack for training. These clusters are organized in a customized computing architecture to maximize memory locality between cores (really critical for efficient back-propagation). Further, you can't use reduced precision weights for training like you can for inference, so there are no shortcuts.

Finally, the initial training stage is followed by reinforcement learning stages - this is key development in how AI models have improved in the past year. This may mean going through a curated set of traces (either synthetic or captured from users) and adjusting the weights based on experienced outcome.

Overall there's so many orders of magnitude more work and more hardware requirements for training that I find it improbable that inference dominates. The number of "inference" steps in training is freaking ridiculous and includes such factors as the "number of words ever written".

Maybe investors will realise that "the only winning move is not to play".

And so we are left with (as was) frontier models getting more and more out of date as whoever their post bankruptcy custodians are tries to eek pennies on the dollar for inference on their decaying property. Perhaps along with local and/or highly specialized models still feeding on the after-glow of the huge amount of training that was (and is no longer) done.

The next AI winter is going to be deep, savage, and long.

The Gemini Flash is very good at searches. Just about any low end model can toss out a poem. All the higher end models (open source and otherwise) seem to be able to churn out code that passes tests. The smaller, "less capable" ones are much faster at it, which means in the hands of a skilled practitioner are the best choice for that task. But they rapidly fall apart where there isn't a hard source of truth (like a good test suite) to grind against. Because of that you have to use a bigger model for bug finding. In that task the open source models tend to fail on larger code bases, where something like Opus still shines. I gather Mythos is an absolute monster, and unparalleled, and unavailable. I'm sure one of the reasons for that is it's so expensive to run.

Or to put it another way - you don't use a 100 tonne crane to pick up the shopping. And ... the smaller models will happily run on in-house hardware. You may not do it today because of the current DRAM price and integrated NPUs have just started shipping, but in 5 years time models will be running on your phone.

If open source models are ~3-6 months behind SOTA, and ~opus4.6 capabilities are good-enough for product market fit, do the frontier labs have half a decade to catch up on their prior burn?

AI cost ballooning faster than companies can afford is becoming a very common topic in my circles right now. The era of "I'll pay infinitely more for marginal gains" is over from what I can tell.

Just think how much further that $100K would have gone if the hardware market wasn't so screwed-up.

Anecdote: I priced-out adding 1TB of RAM to a four node cluster a couple months ago. The cluster was purchased in fall of 2024 w/ 4 nodes, each with 256GB RAM. The nodes cost just over $14K apiece back in 2024 (entire box, not just the RAM).

Dell wanted >$90K a couple months ago to add 256GB to each node.

AFAIK you would get about ~5 concurrent users, with a max context window of ~128K tokens on the larger models.

This wouldn't be good enough for coding -- are you guys thinking of using it for something else?

The decadal move to all-cloud-all-the-time killed off in-house hardware teams while the C-suite chased their OpEx dreams.

It would be interesting if we come full circle on this.

And what happened? How many hours per day/week are people spending watching now?

I suspect that AI will fail to pan out to the same extent for the same reason why outsourcing hasn't fully panned out (even though every company tries it after getting big enough).

The problems that will come up will be and always have been ongoing maintenance. AI is great at writing new code without a brain behind it, but once you get to the point where you need to refactor code, you start really needing someone with coding experience to guide the AI or veto it's mistakes.

I don't think that's really fixable even with a lot better AI. It's not something that ultimately comes out of the likes of github data.

I'm not saying that AI isn't going to make things better, btw, I just don't think we'll see a 20x improvement. Probably more like 1.5 or 2x.

I would argue that that's been the case for quite some time before AI. As an example, what innovative amazing world-changing products have Google or Meta launched in the past decade with their very high numbers of very talented and highly-compensated engineers? The issue with most big tech companies are leadership, strategy, and product direction. I'm not saying that they don't make any profits, just that they probably aren't "building [the right thing]".

AI for product development and management would be far more impactful than automating rote coding tasks / building React UIs that mirror API structures IMO.

It sounds like the economy would largely reduce to the small minority class of independently wealthy people.

What makes you think the people who used to build (or would have built) software will switch into the industry of "knowing that the thing was the right thing to build", as opposed to something cooler like surgery, city planning or experimental physics? The roles within a tech company are not the only jobs in the world.

I spent $200. If I had been paying API pricing it would have been $2,180.16. The article is about how enterprise customers get charged API pricing, which means if I had been employed by one of those companies I would have cost them $2,180.16.

What am I missing?

Yes, value is hard to calculate, but luckily market pricing mechanisms exist exactly for this purpose. There isn't a better number to use than what people are willing to pay for them.

So he's saying that on an enterprise plan, he'd be spending $2,180.16. He's not paying that much, but enterprises are.

- it’s a swindle because ROI of tokens for coding models is not positive? As in it doesn’t bring enough value to charge like the $100/mo?

- enterprise customers are too dumb to see this

- IPO to max out the CEO profits for what is ultimately blockchain vaporware

Am I getting that right? Or am I putting words in your mouth?

> it gives tons of people in the world the impression, they can now build their own startup, game, infra etc without the need to learn it themselves.

I can’t speak for peoples beliefs and motivations, but this seems to be strawmanning, no? AI is a powerful tool to force multiply people. You can’t just prompt “build me an enterprise SaaS app worth $1B” or “build me GTA6 and don’t hallucinate” but is that your impression of what’s happening? Dario and Sam are saying “if you buy our coding agent subscription you can build a game with zero skill and one shot and then be rich”?

If you don’t find value in AI agents I can see reasons why that could easily be true today. Also if it just gives you the heebie-jeebies. But to say it’s a swindle on par with the blockchain I think that contradicts an enormous amount of signals and also the actual dialogue (not just headline sound bytes) around what these systems are capable of today and what we expect them to do say at the end of the year.

> Would we use 10x fewer people to build out infra on the public cloud or the entire so-called Big Data platforms?

No, cannot solve core problems, makes a mess at scale

You are right about the incremental work. But most of the work is historically incremental imo, only few positions are R&D.

I currently pay ~$150/m in tokens to cover my 1099 work. If I had to pay $1000 for those same tokens each month, I would still be massively positive on my margins. I bill customers based upon # of completed features & bugs, not total time spent at the computer. Tokens would have to more than 10x in cost before I would start to have a problem on my end.

So one possible future is that frontier-level training becomes so expensive and the use cases so sparse that it simply isn’t viable to keep going bigger.

Google seems to pretty regularly post about how their TPU and algorithm advancements have been decreasing energy costs for both inference and training.

That seems like a large number, until you realize that OpenAI claims to have almost a billion weekly users. And OpenRouter shows many models at over a trillion tokens per week.

So in pure token terms, I'd say it is in fact extremely plausible that inference dominates, at least for the popular models.

Unless to the grandparent commenter’s point they’re using it to obscure their large prisoner’s dilemma (training) cost?

I skimmed the article, but couldn’t spot any details on their estimates. They mention 70b+ params as being large in several places. But we’ve had several 100b+ param models that trail Sonnet.

A given model is trained once but applied N times. A large enough N will dominate training, no matter how complex and costly it was.

But how long is a model useful for? How often will labs need to train new models? Time will tell.

But what if your competitors sell their knowledge to AI companies?

Then you're still screwed.

Roughly equivalent to 4x H200's for less than half the price.

Vaguely around 60k tokens per second...

My friends in day care tell me the kids hate "movie day" because movies are all the get at home and they are sick of them - they want to play all day. (but I'm not sure if this is representative of anything other than the types of people who put their kids in that particular daycare)

Second to this are countless other areas that have a major impact on the companies bottom line that are entirely engineering driven, especially at google given they are a cloud provider and have meaningfully grown the workspace business and launched waymo in this time.

I would agree but it's really minimized the building. More and more time is being spent on pre-coding work.

To follow on from that comment, if the growth in breadth of capacity of AI leads to a decrease in the risk of running a smaller business, which I don't think is an unreasonable prediction, then it's not inevitable people do lose their jobs. Employers get smaller, higher-leverage, and more plentiful.

And part of my reasoning for this is: the only system capable of actually fixing bugs in vibe-created code is an LLM. If we humans couldn't write it without assistance, we certainly won't be able to debug it without assistance. So there's a real stickiness here.

We're signing pacts with demons - we have to, if we want to outcompete the other warlocks - and those pacts are written in the very size of our codebases.

And this is why many companies go out of business. You always want the best bang for your buck, sometimes this is the "best model" and sometimes it is not.

Not sure about other domains though.

What are your directives?

Also, inference costs are bound to go way down with more optimized architectures. GPUs are fundamentally not great at inference. No platform where the weights are streamed from a large pool of memory is. If the models ever quiet down, there will be massive step changes in cost/token, energy/token and tokens/second, as models are etched into silicon ala https://chatjimmy.ai/

We are still chasing the best because the best is moving rapidly, but it’s a simple thought experiment to work out what the cost to serve an 8B model from 2 years ago is in a world of 2T models.

Note: parameter counts are illustrative. Concretely, qwen3.6 27B delivers opus 4.5 capability at 1/27th the cost on openrouter. Single chip llama3 8b performance can exceed 17k tokens/sec.

Speaking to your point, inference being dramatically less costly than training would not be seen as a delta from the norm. The model of providing inference for anything near the operational costs (like a utility would), would the delta from the norm if it were true.

[1] https://x.com/emostaque/status/1563870674111832066

Why are they getting out of date? Is it because we have new content from the internet that the older models did not have? Or are we simply trying to increase the size of the training data? In other words not more up-todate in terms of time the content was created vs. wanting to use bigger training-input-sets?

They know they do not and that’s why they’re all trying to IPO right now, so they can pass the bag to consumer investors

Your argument rests on the "for marginal gains" part but it's really not clear that the gains are marginal in the foreseeable future.

RAM is expensive, but not THAT expensive. I just bought 128Gb for about $5k for our build cluster (it's not even for AI, sigh). Even if you need larger-sized DIMM sticks, it's still going to be in the vicinity of ~15k tops.

Surely we could just put better stuff on the radio, and accomplish most of the same goals for a far lower price?

My mental model for that is that outsourcing fails where the work is being done organisationally far from the knowledge needed to do it. We know that's true of teams inside organisations, there's been a lot of research on how distance in the organisational tree negatively impacts productivity. Outsourcing is a pathological worst-case of that.

The promise (promise! We're not there yet!) of AI is that I can have a cross-functional team on my laptop. Organisational distance is zero. Where previously the outsourced team has to wait for the time zones to roll round so I can answer their blocking question when I get to my email STRICTLY AFTER I have had my coffee, now it's a prompt in a chat window with a button I can click to make a choice in 5 seconds. Delay is gone, cost of delay is gone.

> The problems that will come up will be and always have been ongoing maintenance. AI is great at writing new code without a brain behind it, but once you get to the point where you need to refactor code, you start really needing someone with coding experience to guide the AI or veto it's mistakes.

Oh, absolutely. That's a minefield. Today. It will be, right up until it isn't. There are ways to set up agents and projects right now that make a dramatic difference to how this part of the picture plays out, but those will sink into the harnesses as time goes on.

But also the big problem with maintenance and outsourced teams tends to be the commercial structure around the contract. You get a Build team, who Build the Thing and then: no more features for you, anything you want to add past the original spec costs extra. They hand over to the Run And Maintain team, who get to fix all the bugs that the Build team left but without the knowledge gained from building the thing, but are scaled and located to be absolutely as cheap as the supplier can get away with so probably don't have the skill, inclination, motivation, or permission to take on any restructuring to make the bug fixing easier and they're on the wrong end of the globe so there's a 24-hour latency on any queries. It's a terrible way to set teams up, but it looks good on paper.

Again, that's peculiar to outsourcing and completely goes away if I have the same team that built the thing own the thing long-term. That's true if it's humans or AI!

> I don't think that's really fixable even with a lot better AI. It's not something that ultimately comes out of the likes of github data.

No, it's a harness problem. You need to start from a maintainable point and keep standards in place. It'll take work to get the harnesses there and it's not ubiquitous. You might also need better models, but I've already personally seen big differences in outcomes between projects that took certain steps and others that didn't; it's nothing revolutionary, mostly stuff that works for humans also works for AIs but you need to know to ask for it.

> I'm not saying that AI isn't going to make things better, btw, I just don't think we'll see a 20x improvement. Probably more like 1.5 or 2x.

I think people radically underestimate the cost of delay. I don't know if 20x is realistic for the AI itself, but I think it's not impossible once the inefficiencies of having to go to other humans is factored in.

The determinant of success was only whether the task needed American-tier labor or could make do with sub-American quality labor.

I don't think there is any shortage of great ideas at these companies, they are just extremely bloated. And I don't think its something like indecision or bad PMs, it's "we have a finite amount of time and resources so we need to be conservative but also not too conservative"

If you have AI systems that can simply build out POCs in days, backtest on real data, show reliable results and numbers, you get a suite of product options you were never able to get before. If you have coding agents that can speed up implementation, you can build more stuff and choose the things that stick.

It changes the cost/benefit calculus of the entire business. I think you are exactly right in that: PMs/leadership are by their nature orchestration machines. Other roles are as well, but I think PM's are at a particular advantage here in that it will be quite awhile I would expect before core product decisions and creativity can be delegated to an AI, but not quite awhile until virtually everything that they're blocked on (legal approvals, POCs, wire frames, etc etc etc) will become less and less of a blocker

Yeah, if this stuff actually worked that well already, OpenAI et al. would just run AI CEOs and engineers. Why get some other company to pay you at all when you can automate every other company out of existence and take all the money they make?

The fact of the matter is that while the tech has some uses, it sure as hell isn't a full scale replacement and you almost always actually have to massage the input into LLMs to get anything decent back out in practice. Some CEOs and managers can learn to do this, of course, and some already are... but that quickly turns into a second full time job. A "programmer" is still needed. The job might change from mostly hand-writing C++/JS/Python to prompt engineering + some manual coding to fix all the stupid fuck-ups that the bots can't solve themselves, but you still need someone to actually prompt the bot.

When that changes, it won't just be engineers losing work; there will be no reason to even have a human CEO any more.

Kubernetes is at 11 years ago, and is huge enough to be included there. The Google Pixel was just under 10 years ago. So... not nothing haha

If they can crack that latter review/spec-check/assurance step, checking that what was built was what was demanded of the problem such that we don't have humans in the loop at that step either, then the bottleneck moves again. Then I think it moves to requirements capture and to product development, but that might depend on the industry.

The problem is they get killed by some other executive who is afraid of their department looking bad by comparison.

I think this is fairly illustrative of the challenges in AI becoming as impactful as the Internet. The bottleneck is not making things. There are plenty of people who are really good at making things and can easily be 10x or 100x as productive as the average corporate worker. YCombinator was founded on that premise - small teams of founders and early employees could be orders of magnitudes more productive than the 1000s of corporate employees at their competitors.

The bottleneck is on bringing your product to market. If your innovative new product is built within a corporate environment, it'll get killed unless the executive you work under can get a promotion out of it, and you'll be denied all sorts of help with approvals, launch process, PR, marketing, branding, etc. If it's a startup, they'll try to shut you out with exclusive distribution deals, legal threats, lobbying efforts to change the legal environment, PR campaigns, FUD, etc.

The Internet was revolutionary because it let millions of people bring products to market without asking permission. Instead of having to bid for retail shelf space among dozens of entrenched competitors that all had sweetheart deals with the retailer, you could just put up a website and sell it to anyone across the globe. Instead of following hundreds of regulations that governed existing commerce, you could just launch something and sort it out later. AI doesn't really have that property - if anything, it makes things more centralized, with more gatekeepers, and so seems more likely to destroy economic value than add to it.

You'll find that most internal "innovation" teams are just lip service. In most cases, the "mothership" will be incapable of reproducing true innovation -- from a statistical perspective, culture perspective (mega corps are anti-scrappy; internal politics), and motivation perspective (startups aren't 9-to-5). It's much easier to have big M&A budgets, a VC arm, and some handwavvy internal innovation group.

Every now and again, you'll get real innovations (Waymo, transistors, GUIs), but even those have a spotty track record of commercialization when created internally.

It takes a skilled knowledge worker to use these things.

Somewhat oversimplifying; writing software and building apps was a bottleneck - now it is not. What is the next bottleneck that LLMs can solve? Is there one? And is there enough publicly available data to solve it repeatably at scale? Or did we just automate stack overflow searches and now we’re stuck again?

Or is the endgame of this innovation cycle the complete removal of interaction with machines through code? Will we simply interact with machine coworkers purely through natural language? Can an LLM make PowerPoint slides and run a meeting? So far not seeing much progress on that.

I'm not sure if this runs counter to your point or not, but: I don't see any future where LLMs aren't a core part of Software Engineering. The horse is out of the barn. There is no going back.

people -> programmers, I haven’t met a non-developer who reports getting more time out of current AI platforms than they put in. If anything I’ve anecdotally heard the opposite, introducing AI at work creates so much slop (output) it takes more time to process it all without a tangible bump in overall productivity

52 on AI misuse: https://simonwillison.net/tags/ai-misuse/

149 on the unsolved challenge of prompt injection: https://simonwillison.net/tags/prompt-injection/

40 on slop: https://simonwillison.net/tags/slop/

If you want an "LLM evangelism blog that rarely, if ever, has any critical analysis that isn’t pro-industry" there are plenty out there. I'm not one of them.

I highly doubt I'll ever use Claude again.

I think you are wrong about Claude being any significant level better

Once the model gets good enough, the returns on bigger models diminishes quickly. I don't want to spend 10x the money and wait 5x the time to get answers that are equivalent.

This is transparently false, because the best "model" is still competent human developers. They're just more expensive. If you're willing to use current LLMs at all, it means you're willing to sacrifice quality for a better price, and your disagreement with the comment you were replying to is entirely about what the optimum tradeoff is.

Currently, the difference is substantial, but what happens if capabilities saturate?

At work I mostly use Claude Code and a bit of Codex; personal projects are OpenCode and honestly I prefer it.

For me, it feels like widely available open models have recently crossed that same canyon. Are they as good as e.g. late-model Claude Opus? I don't think so. But they have absolutely gotten past the point where they are beneficial. This means that, for me, they are about six months behind.

Like how snapchat kind of fell off because the feature could just be a subset of instagram

It seems like it would just become a commodity like EC2

In latest experiment I used opus for implementation plan then used cursor composer 2.5 for execution.

I must say that combo is really good. Main drawback of claude code is that is super slow. So when paired with composer that is super fast it flies.

Wouldn't they be bragging about it to investors? It feels like something that would matter a lot to them, and at least OpenAI kinda feels desperate to find them.

There's also the small question about whether a drop in inference cost would actually change anything about profitability, when training seems to get exponentially more expensive.

They just have to be useful enough that companies don't need the best.

They are.

And 5% worse model for 10% of the price of the bleeding edge will be worth it for majority of people

1- SpaceX + Tesla + xAI merger / IPO while Musk was vocal against IPO for about a decade

2- Warren Buffett cash at record highs

Someone got to be exit liquidity

I haven't had problems w/ Dell support and 3rd party memory, personally, but given the machines' application I understood the concern.

If we still were in the ZIRP era, busting the bubble would certainly kill off the world's economy for good simply due to its size.

There's not that much depth in a lot of 'everyday' writing. For many tasks that means that you don't need to be hyperintelligent - reading a recipe or a shopping list, reading a newspaper article, etc.

Then there’s NTS, BBC… Ypu can listen to them from online service, but at least in Europe there’s amazing national FM broadcastimg services.

TV is just bad radio with flickerimg lights.

Obviously that includes whatever needs to be done to hoover in data from their marks and Meta also does the same thing without fail and both are really good at it. But outside their remit not so much.

I don't.

> What makes you think the people who used to build (or would have built) software will switch into the industry of "knowing that the thing was the right thing to build", as opposed to something cooler like surgery, city planning or experimental physics?

Because it's probably already part of the job. It's a change of emphasis, not a change of career. Your boss can already ask you to do it. If you're producing code, you're probably also reviewing code, checking it matches the acceptance criteria, testing it, sanity checking that it was the right code to have been written, today.

Investor confidence. They have a bit of a need for cash (also an interesting part of the profitability discussion of course).

> Also, inference costs are bound to go way down with more optimized architectures

I agree. Jimmy is incredible, I wonder what non-toy use cases they have. Surely they’ll come out with updated chips soon.

That said, I was apparently a bit over-excited for Groq and Cerebras. I thought they’d quickly dethrone Nvidia for inference, but not so far. Even the GPT spark trial isn’t seeming to go far.

You have to keep in mind that about 99% of their announcements are targeted towards investors (their most important revenue source..), so they're not going to be afraid to mention metrics that make the business look better.

Is this still happening? Opus 4.5 was six months ago, can you get its capabilities for 1/10 cost now? Are we on track to get the same for 4.6 in a couple months?

Training is also done over batches, which increase memory requirements by several orders of magnitude. This is why training needs costly compute.

One of the ways out of this unfortunate situation is to use something like Stochastic Average Gradient Descent [1]. Examples there are mostly concerned with regularized logistic regression, which makes problem more or less convex. Neural networks are inherently non-convex. Still, maybe some ideas from there can be utilized in the context of neural networks, like use of estimated Lipshitz constant to derive curvature and appropriate learning step.

  [1] https://www.cs.ubc.ca/~schmidtm/Courses/540-W19/L12.pdf

That's doing a lot of work here.

The future I see isn't most companies buying hundreds of thousands in hardware to run models, it's them adding a line item to their AWS bill. Inference costs on the larger hosted open source models are dramatically lower than the frontier labs API pricing.

The goalpost we've been bludgeoned with over and over again is that, in particular, Everything Changed in November 2025. That GPT 5.2 and Claude 4.5 were the inflection point. That is actually 6 months ago. And DeepSeek 4 is already there.

> run locally

You can't run DeepSeek locally on consumer hardware[1], but you can on enterprise hardware, and enterprise spend is the subject of this conversation -- and even if you aren't self-hosting, it doesn't matter, because you can just get your inference from one of the the many companies serving DeepSeek, who trivially undercut the pricing of OpenAI/Anthropic because they didn't have to spend hundreds of billions on training frontier from scratch but instead only invest in supporting inference, which is already profitable.

[1] Since this misconception comes up all the time, I'll go ahead and pre-empt it: no, training a 32b parameter model on outputs from DeepSeek and running that locally is not "running DeepSeek", despite the hundreds of stupid articles and Youtube videos making that idiotic claim that they're running it on a 5090.

We're 3.5 years into this current AI wave, and a lot of the valuations have been predicated on what you're arguing here -- that essentially should one of the labs make an order-of-magnitude improvement or hit escape velocity on recursive self-improvement they'd become the most powerful economic chokepoint in history.

The reality has been that given access to compute + capital all of the labs can stay pretty competitive with each other. Someone does a bit better on coding, someone else does a bit better on tool calling, and then they swap after each spending another $100bn.

The market looks like a commodity market where the commodity is intelligence, not a winner-take-all market with massive margins. Plenty of people get rich in oil and airlines, but they notably don't tend to be the innovators long term, they tend to be the operators. Obviously if the machines become sentient tomorrow, turn on their masters, and hit world-dominating intelligence, that assessment changes, but after several years of that narrative while objective reality looks quite different I think the more sober voices are starting to gain a foothold.

My single spark has me running Qwen 3.6 27B and antirez’s specially quantised DeepSeek v4 Flash (which is shockingly impressive)

None of them had the Pirates game.

I was thinking how the transistor radio was a far superior experience for this use case. Just tune to the channel broadcasting the game.

I think the comment put forward that as an incorrect assumption that was made prior to the cable build-out.

That part of dev work, the requirements gathering, attention to details, clarifying requirements, is something AI also struggles with. A lot of companies basically waste time and money on outsourced devs because without a clear path forward they effectively will sit and do nothing, waiting for a prompt.

You still want someone whose ass is on the line if they get it wrong.

I'll also add this: within a large organization, you often need to interact with many different codebases owned by many different teams. Agents have made it much easier to wrangle by having the ability to deploy one to scope out your web of dependencies to learn about what would be needed for feature X, and how that integration can happen.

We've been doing far more away team work simply because it makes things move faster. It's easier to convince a team to sign off/review something than it is to get them to commit to the planning and eventual work.

It genuinely is helping things move faster inside large organizations. Or at least, it is for us, particularly since we're getting organizational prioritization to actually build the scaffolding to make those agents more effective at search.

The human race isn’t ready for that world IMHO. The only reason there is a middle class is because people have leverage in the form of their labor. When that becomes worthless … the people who own stuff and make their living from doing so won’t hesitate to get rid of everyone else - whom are now worthless to them.

> YCombinator was founded on that premise - small teams of founders and early employees could be orders of magnitudes more productive than the 1000s of corporate employees at their competitors.

I think this is still true, but the theory is:

1. You don't need YC-type funding to do YC-type business any more; 2. You don't need to scale the business past those small teams any more, you just buy more tokens.

For clarity YC still obviously has a place as an incubator, mentoring, and networking function. I just think that what was previously the inevitable conclusion that you have to hire all the people the second you hit PMF to keep up with scaling the business as you scale sales is no longer inevitable. If you didn't want to go that way before AI, you were a "lifestyle business" and not worth investing in. As more and more knowledge functions get capably implemented by AI, it's the preferred position: humans are vastly more expensive than tokens, so you want them doing the stuff the AI still can't do.

I don't think this necessarily translates to mass unemployment. I think it translates to masses of smaller businesses that are radically more efficient because the handoffs between business functions are tool calls, not emails to someone who doesn't want to help.

> The Internet was revolutionary because it let millions of people bring products to market without asking permission.

Think about it this way: if I am a small business owner but I think it makes sense to do something that previously only a team in a corporate environment could do but is now within the reach of AI, not only can I do it now, but I also don't have to ask anyone for permission! Who wins between the corporation and the small business in that scenario?

> AI doesn't really have that property - if anything, it makes things more centralized, with more gatekeepers, and so seems more likely to destroy economic value than add to it.

I think this will turn out to be backwards. I can see a version of this where the number of things you can do without needing to turn to a gatekeeper for help increases to the extent that the balance completely inverts.

The vast majority of businesses are small, and AI can give them tools which previously required corporate scale to make sense, without the inefficient hand-offs between busy, political humans. Which is also something that the internet did! Getting an advert in front of a national market pre-internet was Hard but sometimes you had to do it because your target market was "all Canadians who buy toothpaste" or whatever and that meant saturation-bombing the physical environment with physical billboard ads, posters, flyers, and so on. So you only did it if you were P&G-scale. Now you, personally, can do it, trivially, for better or worse.

They do not care unless these companies can get a bailout.

UBI only exists for companies that are too big to fail. Case in point, 2008 and SVB when there was too much money on the line.

One of the AI companies attempted to guarantee themselves a way for the government to bail them out if they were close to defaulting on the debt from the data center build out.

Coding is an interesting case as [1] the pace of progress has been absurd and [2] it's hard to put an upper bound on required capability. However hard to put a bound on and will are different, it's quite possible that the average engineer will cease to see the benefit of rapid progress - or that their employer will be satisfied with lower tier models.

How smart of a model do you need to build a high quality CRUD app for internal users? Or build a scalable web service?

https://openrouter.ai/moonshotai/kimi-k2.6

The march of cost efficiency moves on.

That's the future Amazon sees too. We just had a week long session with the AWS team and they pushed that to us multiple times.

Claude code was a lot of people's introduction to using coding agents that could do a lot more than copy-pasting from a chatbot or autocomplete.

Running software in the cloud gives you certain reliability and scaling advantages that would be very hard to replicate locally. Running some code agents in the cloud vs local hardware, if the local hardware gets "good enough," breaks the other way - offline usage, alone, would be hugely valuable to many people and companies.

It'd be very interesting to see where various players would decide to make a call "local is good enough" though. Buying the hardware isn't a small bet, if it's not something that ends up as part of your standard computer.

I'd qualify that by writing that you can't run it with ordinary, real-time speed and throughput. If all you care about is slow and high-latency inference, there's no reason why that shouldn't be feasible even on the cheapest miniPC around, as long as it can literally store the model weights and keep around the (rather small) context.

With a nice UI on top, for the desktop app too: [2]

[1]: https://developers.openai.com/codex/config-advanced#custom-m...

[2]: https://docs.ollama.com/integrations/codex-app

How I find your argument is that one distinguished engineer from US could do the same with the use of AI.

I worked with both and I know great and bad engineers from both sides. Only thing is that US has a bigger pool of great engineers.

1000x yes: you have touched on what I think is the single biggest factor here, that is the humongous value of POCs. they are gnarly to build without agents, and so we used to have to get everyone on board so we didn't get screwed in performance reviews, which was monumental task because that means convincing very busy PMs who have a lot on their plate and dont want to take risks on things they don't understand, and now it's like "can we scale this out" and you have a very nicely formatted proposal and POC. It de-risks things very quickly

See https://taalas.com/products/

Edit: updated link

Training is inference + backwards pass (~2x inference cost) + activations (vram overhead) + optimizer (vram overhead) + gradients (vram overhead).

The days of requiring a data center to run anything resembling opus 4.6 are already counted. (But the industry will fight hard to get people to keep paying the Claude tax.)

Maybe not DeepSeek v4 Pro, but I've run DeepSeek v4 Flash on my 128GB MacBook Pro using antirez's carefully quantized https://github.com/antirez/ds4 and it's impressive.

Opus 4.6 quality for local inference would be revolutionary.

I'd pay a premium for even just a model that's 20% better, no ASI required, and I think a lot of people would. I wouldn't call that marginal, if it means I'm getting frustrated on 20% fewer tasks.

A recurring pattern that I've seen in myself and others is to at first be very impressed by a new model's coding capabilities, and then desensitize quickly and start being frustrated by the shortcomings.

The larger point I'm making is I think models are rapidly becoming commoditized. There is probably a small market long term that's willing to pay 10x for 10% marginal gains, but the majority of the buyers in the market will be economic and we're likely to have a lot of folks willing to spend 1/10 the cost for 90% of the performance, and plenty of companies that haven't raised hundreds of billions-trillions who can provide that.

A lot of the frontier labs valuations has been based on an assumption that 1-2 companies would get break-away intelligence that basically made them economic chokepoints indefinitely into the future. The reality that's becoming increasingly clear is that model quality is a pretty linear function of (cash burned - ability to copy other's homework) and the economics are starting to look a lot more like airlines than online advertising.

And worse, these are the tasks that help the junior people eventually grow into the skilled knowledge workers required to operate models, so there's a pipeline problem too.

Not completely, but compared to the middle ages we 50x'd their output. Which is a great illustration what it means to make a job 50 times more productive. We went from 80-90% of the population being required to barely make enough food for everyone to survive, to 4% of the population producing such an abundance that consuming too much food has become a systemic health issue

A $50k - 100k rig could do it and an entire company would be able to use it a full speed.

Kimi is close for example regarding SWE bench for code. For reasoning there are open models that surpass opus by quite a margin already.

I bet this will ironically be couched in "safety" reasons or regulation to get anti-AI folks on board, even if it favors the large incumbents.

The point I'm making is that I think we're rapidly hitting levels where corporate buyers aren't willing to pay multiple-times-more for marginal gains, and I expect that to become more the case over time, not less. You, and a small % of other power users in the market might tolerate a $400/month pro-supreme-plan for access to Mythos or whatever, but I don't think that's going to scale up in quite the same ways we've seen so far.

Even a year ago paying multiples times more for a 50% gain was very sensible for a lot of workflows. But if we're getting to "good enough" for things like coding, justifying to your CTO/CFO why the org should go from spending $1m/year to $5m/year for a 10% higher hit-rate on one-shot prompts from the engineers is a much tougher sell.

What you call harnesses I call… bullshit?

The economics of airlines are such that they generally earn a return on capital less than cost of capital.

I think this is exactly where we are heading and OAI-Anthropic are the concordes.

(Yes, all the good developers from Oklahoma move out, but the same is true of Ireland)

Basically every big tech has large offices and employ a lot of people there.

The limitation is that Ireland is a relatively small country, and most Irish developers are already employed (which is why Ireland end up being one of the main destinations for tech workers being hired from abroad).

Nobody is doubting that there are some people who watch films 10–12 hours a day, every day of the week.

People got a lot done before Opus 4.6. In 6 months, would you be dissatisfied by Opus-4.6-level open-weight models, just because Opus 4.8 will be out?

And last, but not least, you need only one hidden layer kept in RAM for inference, but you need all of them (61 for Deepseek models) kept in RAM for computing gradient for one sample.

The people who are claiming Opus level capability does not have sufficiently complex problems to see the difference.