Exactly. I haven't reached the "let 1000 agents bloom" mode yet, so currently I'm spending real headspace managing agents doing work, and that work is all important, so why "settle" for sub-frontier models for that work? Maybe I'll get there for non-coding work.

> separating the trivial and non-trivial tasks is a famously hard problem (if at all decidable).

Famously, this is also a problem for human coders in sprint planning.

Highlighting https://news.ycombinator.com/item?id=49113236 in response.

HN could be run as a BBS on 70's hardware. Instead of using a CPU with ~10 thousand transistors, you're likely using one with ~10 billion to do basically the same thing, and you don't think twice about it.

Cosmically apt username given the substance of this comment.

They have no (other) equivalent to nano, so it makes sense that it’s much cheaper now. It may have been better, but it was also hell of a lot more expensive.

Luna is comparable to GPT 5.4 from 4 months ago on many benchmarks. I know many who have said during that time, myself included, that if that's the model they had to use for the rest of their lives, they'd be fine.

GPT 5.4 is/was a very capable model.

I don't see why it should be all that difficult. All you have to do is first find a library that implements a decent solution to the halting problem and you're off to the races.

I get frustrated with a poor quality model leaving my codebase littered with wrong comments, which then later trip up smarter models.

If you have agents and users, you can run evals and see how far the models go. Luna is not greatest in tool calls, but if you define your problem well and the tools well, it is comparable to Gemini 4 Flash with much lower price tag.

Translation, moderation, classification, guardrails, etc..

I was curious so I photoshopped DSV4 Flash into the graph:

https://files.catbox.moe/csxl32.png

(2 cents to run AA index, score 40)

Looks like OpenAI broke the pareto frontier on the trust-me-bro benchmarks!

(One has to wonder if they used any of the neat tricks from the DSV4 paper :)

These are free due to some different type of reasons like Nvidida sponsoring free tokens or the model companies.

My company checks the models and pays for Opus through AWS.

You still send the WHOLE context of whatever you want to do to a random endpoint on the internet. If you want to write a good email, you give that context your email address, names, the reason for it etc.

Big companies don't randomly use some random api endpoint to do so.

Anthropics quarerly revenue is still growing very fast. I don't think we have seen even the real potenzial of it yet at all.

Not only are still a lot of countries missing which do not even use anthropic or any other frontier model yet but also all the agentic based solutions enterprise companies are currently building on mass (at least in my industry)

I thought the chinese models were cheaper per token, but about the same or more expensive on tasks because they used more tokens for reasoning. Cutting even further, seems like a really big leap.

So now presumably it'd at least be roughly equal cost, or maybe a little less?

Sol as main agent, Luna for coding?

I think there's a ton of room for efficiency improvements in how the models are built and run, and I think OpenAI has both prioritized that work and figured out a lot of the tactics (and borrowed some from the Chinese models like DeepSeek and Kimi, which have published a lot of their research and tactics for running big models fast on minimal hardware).

I think there's also a new generation of hardware in the past year or so tuned specifically for LLM workloads, where it was almost an accident that GPUs worked to run LLMs before. So, while there's still this ridiculous shortage of hardware, what is being delivered is much faster and cheaper to run for these specific workloads.

I wasn't expecting it to happen from the US vendors, though, as they've spent so much capital to get to where they are they need to make huge margins on inference to pay it all back. I expected the Chinese models who're running much leaner operations to be the "frontier" on costs (and they have been). But, I'm glad to see OpenAI joining the "cheap and cheerful" models party. There's a lot of work in that area of capability. Probably most work people are doing falls into that area of capability.

5-10% over months would still be quite crazy.

But yeah I do'nt want to know what Kimi 3 is pushing buttons inside Anthropic, OpenAI and Google.

Besides any floor: For every year the tokens get faster and cheaper, we will see new things like properly working AI factories which mimic expert teams. A lot more parallism as well.

I pretty strongly disagree about comparing this to Kimi and GLM, 5.2 was a big price hike for Chinese models, and Kimi K3 was a big price hike to that. K3 was within spitting distance of OpenAI pricing (more expensive than short context Terra, less than long context). And that's after months of OpenAI/Anthropic prices going up.

Now we have an American lab drastically cutting a price, feels like this is the opposite of that trend.

Its just a matter of time at this point.These companies are working day and night to capture the market.

depends on what you are doing. if you are doing verifiable tasks like fixing bugs then any model would do as long as you write the right verification.

Is it though? I saw a noticeable difference between Sol (xhigh) and Luna (max). Sol appears to understand better my prompts, you need to be more specific/clear with Luna.

The type of job matters. If it is too complex then yeah, you should be using a larger model.

Link to definitive price info?

Agentic layer.

Your support bot.

Your research long running bot.

Your SEO Optimizer bot.

Your incident analyser bot.

Your personal assistent bot.

Experienced similar between 5.4-mini vs 5.6-luna in our own pipelines but after spending some time on prompt optimization and testing out various reasoning effort levels 5.6-luna was well worth it. Did you just replace model selection while keeping everything else in place or spend some time on evaling with newer prompts etc?

Sounds reasonable. Is there a good benchmark on which make this decision?

DeepSeek Flash is a much worse model. Even DeepSeek Pro is much worse.

how many times do you have to be metaphorically hit in the head with a brick before you realize inference margins at api pricing were 80%+

It all comes back to electricity cost. China has cheaper electricity so as long as China keeps pace there is no way for American companies to undercut them. Each boolean operation in China is cheaper than the one in America.

> China: Household rates average around $0.08 / kWh (¥0.53/kWh).

vs

> US: Household rates average around $0.16 / kWh, though regional variation is massive—ranging from ~$0.10/kWh in low-cost states (like Washington or Louisiana) to $0.30–$0.45+/kWh in high-cost areas like California or Hawaii.

It's marketing hyperbole, but Luna is more intelligent per dollar than deepseek-v4 pro. Cost means nothing without the associated capability

They got rid of the 5hr quota, it's just weekly quotas now.

When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon and we will see another 10X improvement in price/performance frontier.

The dynamic GPU clusters will be used for the 5% of tasks, and pushing out the frontier. Also there will be a set of knowledge tasks that are not done today (because they are too difficult for most knowledge workers), that will start being done in the future.

To be fair we don't really know in terms of prices what's real and what's just investor subsidised attempts at market capture at this point. It could well be OpenAI's attempt to drown Anthropic while they've got the halo product if they feel they've got deeper pockets.

Hard to believe numbers. I don't mean that as a critique, but literally I am so impressed. Even if the model is a few percent lower for performance but is 80+% cheaper than competitors and is a US company hosted on US based hyperscaler clouds this is kind of a no brainer. Hard for most businesses to justify otherwise.

it's 80% less cost, not 80% in efficiency gains, could be that Luna was overpriced to begin with, we don't have much info on the models themselves.

Assuming the efficiency gains are real, I feel like something has to give, maybe worse quality due to aggressive quantization/kv cache compression?

Vera Rubin will be hitting racks very soon, and this is purported to have a 10x improvement in token throughput per megawatt. Of course, old chips don't get replaced with new chips overnight, but I don't think we're anywhere near the floor yet.

This is gonna put Sonnet 5 in a really awkward spot.

> Sol vs Luna

> it doesn't feel like night-and-day.

I see what you did there. :)

is kimi that cheap? it's a very expensive model

~2 years ago gemini2.5 helped write better kernes for itself and (only) reached 1% efficiency gains. Today we're at 20%.

Some numbers: https://www.wheresyoured.at/exclusive-openai-financials/

If those numbers are accurate, I don't think 20% is a really, really big deal. It's like saying "we're digging our grave 20% slower." Ok, but they're still digging!

Or, different analogy, if I'm going broke because I lost my job due to executive AI psychosis, cancelling my netflix subscription doesn't really change the math of not being able to afford rent. It doesn't even really slow it. The amount of money that OpenAI is spending is so absurd that a minor cost saving is like, uh, some progress, but they'd need to do it a lot more to move the needle

imagine writing that on your resume

> reduced inference cost by 20 percent saving company x billion dollars per month

How do you run 'deep research'?

Very interesting. Can you share more about your hypothesis/research pipeline? I have been using Sol for those types of task because I figured you'd need more reasoning for getting good ideas, but maybe quantity > quality at a certain point?

Do you have a sense of which tasks benefit from more agents and which don't?

OpenAI's APIs are extremely reliable for sure. I don't even remember when the last incident or downtime was.

Hasn't OpenRouter had Luna and Terra on 50% off sale since they launched? I wonder what will happen to that.

The official doc says, Luna = Previous Nano models, kind of. Is it really good at coding?

Luna is meant to compete with Haiku. What tasks are you seeing it equal Opus on?

I don't know, non of the chinese models I use are served from China. And they are still cheap.

I was metaphorically hit in the head with the brick of already provisioned computing with a lot less demand than anticipated by multiple MoUs.

They're supposed to bring 5h today.

Game play bots (monsters, commanders maybe) would be really cool. But it needs long term support and probably local AI instead.

So, does that mean they're selling tokens at cost? Or that they realized they were doing 180% more work than necessary and they're passing the savings on because "your margin is my opportunity" and this is a dog-eat-dog fight, but 80% margins are nice, no one needs multiple hundreds?

I mean to be fair this price reduction does not prove anything

If somebody already thought the price was subsidized then this price change would just mean it's even more subsidized, so why would that change their mind?

I wouldn't be surprised if they still had some margins since cheaper models are much harder to nail the accurate sizes off, and you still pay 2x for 1M context window.

But if this is even at 400B size it's insanity those inference prices, maybe 10-20% margins, if it's higher I would like to know is it their own chips or maybe they have accurately sized the model to fit on exactly a B300?

Could be a lot of magical things we can only speculate, but from here there likely isn't another 60-70% margin, like I have heard people claim, I would definitely be willing to bet on that.

Could still be a healthy 10-30% margin. Especially with Terra.

Been using OpenAI models since ada/babbage/curie/davinci and at least from my own experience, their APIs feel the same.

If you use Codex it's different, the harness has a lot to do with it and there's definitely been changes including recently.

High-performing open weight models being released recently, and your customers looking into working with multiple providers as a result, are a great reason to drop prices on your non-frontier offerings.

Although I'm sure there are some efficiency gains, the technology is too new and labs are scrambling to release too quickly to think that the low-hanging optimization fruit has been picked already.

Something can be overpriced and still lose money.

I use sonnet as a smart grep and haiku never and that’s only when I have to use Anthropic at all

In 2 years from now we will be at 400%. https://xkcd.com/605/ Also, it is called kernels (you have nitpick in your username)

Canary tests on my data showed I needed xhigh to get good results, but they are good.

This is annoying. Similar with "context poisoning". LLMs can be highly productive... until they're "poisoned" with contradicting inputs / guidance.

don't expect it to be opus, but luna does coding just fine for its size/price

Good points. Though GPT 5.5 Instant does beat out 5.6 Luna non-thinking 34 to 27 on Artificial Analysis Intelligence Index.[1][2] This could mean it is optimised for non-reasoning perfomance. So even if beaten on cost and performance, it could still be on the pareto frontier for speed.

It probably costs them a substantial amount of money everyday to not replace Instant with Luna, and the people who want larger models will upgrade regardless of whether they get Instant or Luna on the free tier. It is unlikely the amount of people upgrading just to have latest models would be enough to offset the costs.

[1]https://artificialanalysis.ai/models/gpt-5-6-luna-non-reason... [2]https://artificialanalysis.ai/models/gpt-5-5-instant-05-26

I just gave you the ROI on a retail blackwell card, the math checks out, particularly on overpriced Haiku and Sonnet, what do you mean by "you're just fighting economies of scale if you're going trying to beat a datacenter on cost" ?

Local LLM argument was always ideology first and never ever about economics.

Dinky stuff I want to be fast like “update the PR with this other JIRA ticket”

Summarization of datasets, categorization

This is exactly the model that DeepSeek V4 Flash followed, and it's been insanely successful as a result, even though it's not frontier.

Good observation! Kudos

You'd be surprised at what Luna can do, especially on xhigh or max. It's capable of working overnight, usually productively, just like Sol.

Haiku 4.5, on the other hand, is comparable to performance to Gemma4 31B (with working tool call formatting) in my experience, and Gemma4 strongly wins on vision and multimodal.

Smaller models are great if you are doing targeted changes in existing codebases. Don’t expect to use it for creating complex architecture from scratch or do major refactors. The larger the context, the greater the drop off will be.

This doesn’t quite count as “API”, but OpenAI’s roll out of OAuth device code authentication was poor, to say the least.

According to the link I mentioned above it's roughly as good as GPT-5.4. Haven't tried it in practice yet.

I bet it must be better in some contexts and worse in others.

Wasn't it down last weekend?

According to the Artificial Analysis benchmark graph in the article, Luna can now outperform Sonnet 5 and Opus 5 low at ~4-10x less cost

No we kept prompts as is, just swapped model. The prompt is already quite optimized for the task. How would updating it possibly make a more intelligent model spend less tokens than a less intelligent model? Care to elaborate?

This doesn't seem correct.

Estimated final electricity price for large industrial customers in energy-intensive industries:

USA 50 USD/MWh

China 68 USD/MWh

https://www.iea.org/reports/electricity-2026/prices

I maintain this meta-benchmark leaderboard: https://metabench.organisons.com/

With this new price change, Terra does look pretty Pareto’ed by Luna.

On agentic coding, pairing Sol Medium for architecting with Luna High for coding does kinda make sense. But beware that architecting can be very read-heavy, and Sol is a bit read-pricey compared to Terra.

is it? i don't know how we measure these things, but here's one measurement that says v4 pro is better than luna: https://artificialanalysis.ai/models/comparisons/gpt-5-6-lun...

presumably it's a much bigger model

Prices have consistently gone down 90% every 18 months like clockwork for about 5 years for the same level of quality. There is no end in sight for at least another generation.

There is ZERO reason to believe models 1/10th the size of frontier are completely capped on intelligence and impossible to get smarter.

They have consistently compressed the intelligence of larger models.

You'll see it first on the small end, when they stop being able to compress intelligence, you know that will slowly bubble up and up the chain to larger and larger models.

There's no evidence we've reached that at the bottom.

DeepSeek v4 Pro & MiMo v2.5 Pro (Opus 4.6 quality models for code) are insanely cheap for agent-driven work due to their super low cached-input prices ($0.0036/mtok) [0]. For Luna, the cached-input price drop isn't disclosed in TFA, but the pricing page puts it at $0.02/mtok, & that's 5x more expensive.

[0] I am constantly surprised how much work pay-as-you-go with DeepSeek / MiMo will get done. I've barely crossed $2 each in a month of use (~200m tokens).

AMD MI400 series is already shipping to customers (basically everybody) and it is crazy fast (8x to 10x faster than the previous gen and beats published Vera numbers in FP8, loses in FP4) and 432 GB per chip. 72 chip unified rack architecture (Helios) already shipping and projected to also beat Vera in NVL72.

MI500 series is supposedly already taping out and they're claiming massive increases (we'll find out end of 2027 prob).

> When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon

Google is already working on a similar idea but more "flexible".

Luna is comparable to Haiku, not Sonnet.

Let's suppose each models was subsidized at 70%, so that we only pay 30% of the cost. They would loose much more money per token on the more powerful models. It's in their interest to encourage the use of the less expensive models. Let's say they increase Luna subsidies at 90%. They would still "save" relative to the use of the more expensive models.

It's cheaper currently on many of the inference providers.

Personally, I'm having surprisingly good results with DeepSeek 4 Pro at home, which is very good value for money: it's not as good as Claude / GPT 5.6 (I have Co-pilot license at work), but it's still really useful for code reviews, validating thoughts, and especially designing / writing unit tests for new (and old before refactoring) functionality.

And it's very cheap per task. (Flash is even cheaper, but I've had issues with that on more complex tasks where it starts forgetting things and arguing with itself "but wait, let me read the function again").

Contributed to. Can't be some IC who made a few nice PRs

You can use my script p-noteq-np.sh too if that helps.

Luna is good as an end user model for simple tasks like classification, but not as a coding model. Also do you mean Gemini 3.6 Flash? 4 doesn't exist, and Gemma 4 exists but doesn't have a Flash option.

But they still have to cover compute cost, and they already committed to more than $650B in infra expenses for 2035

Use a harness like OMP that lets you choose which model does which things. My main model is GLM 5.2, it handles planning and anything I dont have covered by other models. Tasks from todos and in sub agents are done by deepseek, I have different models for the git work like add/commit/push (that goes through cheap Minimax M3), and so on...

This way the expensive/strong model only handles the architecture and orchestration tasks. The cheaper models handle everything else and the strong one knows how to tell them what to do in enough detail to get good work out of them.

I remember bugs always end up being more expensive then the first time I implemented the feature… so for me always sol always max… I don’t want to pay for a bug later

Where are you going to apply to with that resume that’s a step up from your current job though?

We can guess based on the decisions of other inference providers who serve these models.

Burning the weights into silicon would be many orders of magnitude increase, not just 10x. It's kind of crazy that this hockey stick the AI hype bros talk about seems more and more every day like it might be real

Enterprises implemented spending caps and inference providers are lowering prices. Seems they are jockeying for market share.

It's a feature offered in ChatGPT and other platforms, though probably gated behind paid subscriptions.

Anything related to reading and interpreting the environment seems to always benefit from the addition of more agents to the search party, assuming you have some rational way to synthesize their results.

Taking actions that mutate the environment is a different story. I think this is where you run into diminishing returns very quickly. You generally want one strong agent to act given the results of all the searching that was done. If the plan is clear, you don't need a genius model to execute it.

Sonnet and Haiku were already in an awkward spot, likely by design.

Anthropic's big marketing push this year has been entirely focused on getting people to use Opus via a Claude Code subscription, to the point that Sonnet is almost viewed as the poor man's alternative, and from what I've seen, almost nobody uses it.

Actually, here's an interesting project for all the vibe coders looking for their next front page post: scrape a ton of commits from GitHub with Co-Authored-By: Claude and figure out what the percentage split between Opus/Fable/Sonnet is. I'm willing to bet it's less than 10% Sonnet.

Per Artificial Analysis:

- Haiku: 30 points

- Luna Medium/High/Xhigh/Max: 38/46/49/51 points

That's a massive difference:

- 30 points is Gemma 4 31B territory

- 50 points is GLM-5.2 (744B) territory.

In this case, and I don't mean this critically, I guess it would technically be, "Instructed model to find efficiencies... reducing inference cost by 20% saving company x billion dollars per month."

I have no doubt that further work was required to enable this, but it's still very cool to be possible to say that.

It's still 50% off apparently. Listed as $0.10 for input (original price was $1.0 without this reduction or sale)

If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers.

Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost. This is a bad comparison.

[1] https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...

In a data center that is power constrained but not space constrained they could build out new racks and flip the power from the old racks. Wonder if this will lead to moderately used server GPUs on the secondary market someday.

Here is a rough approximation of the pipeline I use:

Phase 1 - Run X copies of Luna in parallel over the user's prompt. The purpose is to generate a diverse set of hypotheses.

Phase 2 - Run Y copies of Terra in parallel to investigate the hypothesis results, with each receiving them in a randomized order.

Phase 3 - Run 1 copy of Sol over investigation reports.

The goal is to ensure that the agent covers more initial starting points before presenting a final conclusion. If you only run a single copy of Sol and it hooks onto something wrong, it might not recover.

Deep research is basically a LLM with web search, and a "work really hard" goal-orientated prompt, and some output formatting suggestions.

Depends on the reasoning effort, see https://deepswe.datacurve.ai (add Luna via model selection drop down, if it's not shown by default)

luna is way better than haiku 4.5

5.6 class models have a brand new caching architecture. Make sure you were using the prompt cache breakpoints and cache keys correctly - it's much different than < 5.5 and makes a huge difference for many production workloads.

DS V4 flash has been updated today!

https://api-docs.deepseek.com/updates/

The update seems to say that it's a re-post train of exactly the same model.

Slower/more verbose in my experience too. However, still self-hostable which is a major plus!

Kimi K3 is fairly cheap per token but thinks like a madman with poor self esteem.

I bet they’ll announce new fundraising soon. Someone gave them a top up to try and corner the market.

Here is an example of a guide from OpenAI on how you should prompt 5.6 differently than their previous models.

https://developers.openai.com/api/docs/guides/latest-model#p...

Thanks, I was looking for something like that

Most of the time when upgrading models we have needed to change prompts to get the same performance (let alone better performance). Usually, your prompt is overfit to the specific model doing the specific task. For example often your previous prompt is overspecifying and creating contradictions that a dumber model would just gloss over whereas a smarter model will try even harder to follow.

Agree on 5.4. And it has 4x more quota than 5.6.

ya im partial to oracle.ts xD

Luna on what reasoning level. Try Max

Why not. Jeff Dean and John Carmack exist. Both are L10 SWEs.

Yeah, been running too many evals in the past week I start to mix the versions up. Probably should sleep...

We run an agent company and outside coding the new Gemini 3.6 Flash and GPT 5.6 Luna are very interesting. Luna can do a bit of research and create reports. Gemini is great for computer use.

For programming it's all Kimi K3 now.

>figure out what the percentage split between Opus/Fable/Sonnet is.

This may be misleading, since I suspect many are using a blend through sub-agents. I tend to bias for Fable to orchestrate and Opus for implementation via sub-agents.

I've found that Claude nearly always claims to be Opus in the commit message, regardless of the actual model making the commit.

The other place, but for more money

When I'm paying for it, Sonnet. When work is paying, Opus 5, then Fable if Opus gets confused.

Partnerships then consolidation comes next...

Its funny because you can write a halting problem oracle by calling out to an LLM and have it return yes / no / not sure and get it to work reliably for almost all real code, like that is an entirely practical thing to do in 2026.

All we need now is some sort of program to evaluate halting problem oracles...

Those finanicals show OpenAI makes good money on inference.

all day yesterday, Taiwan time Sol was having significant issues.

“Overloaded”, maybe half of requests or more wouldn’t go through

Not my experience at all. I've been driving Luna xhigh in CoPilot CLI since it was released as my daily driver. Good reasoning, no problems with tool calls.

lots of places, actually. not everyone wants to be attached to the Silicon Valley culture, and that line alone will guarantee practically any workplace. that person is going to find out what work-life balance is :)

Why assume they are not already making up the compute costs for smaller models?

Used to be part of the $20/mo plan but it's not anymore (not sure if they removed it completely). However GPT-5.6 is pretty good at researching if you prompt it right, I've regularly had it spend 5+ minutes researching topic with lots of web searches.

What does that mean though? Like some kind of a ROM memory ?

Yep it will be ASICs and DSPs all over again. Orders of magnitude changes.

Do you mean if other providers will cut their prices in turn?

https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second.

https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.

And don't underestimate how much money Google, Microsoft, Amazon and Meta still have to spend on this tech.

Blocking Fable for sure made it very politicl a lot sooner than i expected it to happen.

and because China already has massive problems of getting access, they are pushing it on hardware too like what Huawai did without EUV.

It seems China is already able to do DUV a lot sooner than others expected.

Not an expert on this but wouldn’t this be possible with something similar to an FPGA?

I definitely think you want the genius model to synthesize everything that rolls up to them.

Absolutely. Although DeepSeek started announcing "Peak valley" pricing which started making me nervous. I have spent $50 usd in July on deepseek and for that much spend I got SO MUCH mileage.

I feel perfectly content in using pay as you go pricing with deepseek. On the other hand, although Anthropic's models used to be my bread and butter for personal work, they are simply too expensive to reach for these days.

In my real world use Luna is as useful to me as Sonnet. And it gets stuff done faster and follows my instructions more closely.

Anthropic basically downgraded all their tiers when they released Mythos, no nah, Sonnet 5 is the successor to Haiku 4.x

I toggle back and forth between deepseek v 4 flash/pro on FireWorks.ai using OpenCode. Easy to toggle, I default to flash.

That’s comparing Luna on medium thinking with Deepseek on max. If you switch Luna to xhigh in that comparison it’s considerably more capable while still also cheaper.

Yikes, you can't really expect prompts to just be model agnostic

You are comparing Luna medium to Deepseek Max, when you switch to Luna high it's cheaper per task than DS and has a better intelligence score, which goes higher yet on xhigh and max reasoning levels.

https://artificialanalysis.ai/models/comparisons/gpt-5-6-lun...

I checked out your script but it looks like it's just a wrapper for ! ( p-eq-np.sh )

I've done a little bit in this space to try and see what works, and maybe it is different per language/framework but found the use case seems best for largely mechanical code changes, eg small direction, lots of simple changes. This sounds good until you realize that IDEs generally have built in functionality to do these tasks waaay faster, cheaper and more accurately.

Where I'm at too, though max overthinks like crazy. xhigh is my sweetspot.

I think the fundamental difference between our assumptions is you believe prompts to be optimized for tasks rather than model-task pairs. The only elaboration I can give you is empirical observations and model providers own guidance (as someone has already linked here). I'm pretty sure you probably have specific parts of your prompts that came about due to specific failure modes observed in your evals of running the task against first model. These vary across models in my experience, and it's always worth redoing this calibration process.

Opus 5 is not strong enough as the top-of-stack model, and feels idiotic after a week or two of heavy Fable usage, to the point where I'm paying for Usage Credits to keep using Fable rather than having to slum it with Opus.

I think they meant that GPT-5.6-Sol can write that on its resume.

Does the model get the credit for its promo packet then? :^)

> This is a bad comparison.

How so? First, kernel writing (or ML engineering more broadly) is a highly specialised task. Not everyone can do it. It shows that models are getting better and better at (easily verifiable) hard tasks. And you can "hire" that expertise much easier than you can hire the equivalent meatbags. And more importantly you can "fire" them as soon as the task is done. And then hire them 3 months later, when the new model drops. And so on.

Second, 20% gains in inference today gives better end results (i.e. lower overall cost) than 1% in training 2 years ago. Today's models are improving mostly via RL. And RL is highly dependant on fast inference (you want many rollouts for each training scenario). Same for dataset filtering, environment generation, distillation, etc.

I don't think they overengineered a DC like this.

Besides Nvidia Hardware is still sold out and super expensive. Not a single Nvidia consumer GPU got cheaper at all, Nvidia DGX Spark got more expensive too.

It will be swooped of the market the second it hits the market.

And self forking fan out.

Thanks for the link!

I use Luna a lot (over 1T tokens since it came out) and I'd rank Luna (high/xhigh) on par with Sonnet 5, without hesitation.

This has already been updated with the new prices?

Explain.

Totally untrue. Luna and Sonnet 5 are very comparable: https://artificialanalysis.ai/#intelligence

Luna is an extremely strong model.

> Let's suppose each models was subsidized at 70%, so that we only pay 30% of the cost.

why on earth would you suppose that?

DeepSeek V4 Pro is ridiculously priced, especially when you take into account caching. According to the DeepSeek usage panel, 50M tokens have cost me $1.38. It's not the smartest and does like to overthink, but if you have well defined problems it's good for coding. Well... except all your data going to China. I just use it for personal projects.

I tried out deepseek v4 pro via a couple providers from openrouter, and it's always getting 429s. Are you running it on your own hardware?

where are you seeing cheap Kimi? pricing I've seen is the same across the board (presumably due to licensing terms) and is in the Terra range.

I'm curious, how hard/expensive it is to burn a really large model into silicon, and why aren't we doing this already?

Or, when we will start doing this, who's going to be able to do that in scale?

I'm seeing the TAALAS example, but it's only an 8B model, suggesting some real limitations parameter wise. And for 2.5kW?

How do you run this phases and parallelization on each?

Via just ... "prompting it"?

Or do you use any tool in the middle to ensure this agent architecture?

Just curious if there is any workflow-like tool in the middle that is helping.

Sure, but those places don’t need such lofty resumes to begin with.

Using Sol or Fable for implementing is like having your Principal Staff Engineer with 30 year tenure routinely write CRUD functions for a REST API.

You use the big models to plan. Not just the overall plan, but which files need to be edited etc. Then they give that to the lower end model. So Luna or Sonnet, which are perfectly capable of following instructions and still creative enough to not get stuck.

They placed it in the Plugins submenu.

Stacked ROM can, in theory, be a lot denser than anything that depends on a capacitor and refresh cycle.

I don't think it would be that difficult to manufacture compared to other process tech. HBM is really hard to do compared to other memory types.

Yes. For example, third-party inference providers serve DeepSeek V4 Flash just as cheaply as DeepSeek themselves, if not even more so. This is very strong evidence that the low price of the model is not subsidized.

See Cerebras and Groq as well.

> It seems China is already able to do DUV a lot sooner than others expected.

That's the media and in particular US KOLs of all sorts driving the wrong impression of China and other places. China and many other places for example have fast public transport that the US doesn't and can't even imagine today. They're not behind.

China's DUV still isn't that production grade (mass produce-able) so don't get that hyped up the wrong way (in a different direction).

The whole China-is-behind with tech and in particular semi wasn't that they can't. The truth is they spent decades in internal politics and corruption. That all got solved with the bans, so thank the bans! Jensen even said the bans were bad.

Weights can be baked into silicone or programmed into hardware ala FPGA, but the context will always be dynamic.

High speed SRAM is where the $$$ is

This is why in your brain you have trillion threads processing and summarizing sensor data (immutable functions), but a SINGLE thread of “execution” which we call the conscious soul.

Enough for such a massive reduction? If yes that’s really impressive

My understanding is that for FPGA the issue is either it eats all your gates on internal memory if you interleave, or it takes forever to load everything between the SRAM on the board and the actual FPGA component over a bus, last time I looked into it.

Absolutely true although at some point it's not just raw numbers but also the kernels that run matmuls and there seems (from outsider perspective) to have been more optimization in the cuda kernels

In fact there's very good reason to think intelligence has much more room to be compressible - the human brain, for one.

Not true at all. Do you really believe that generating 1m output tokens on sol really cost 100 usd?

These companies are printing money on inference. The only issue is the capex of expending on new servers but they are generating TONS of income.

The intelligence scores are absolute, not per $.

DeepSeek Pro is more capable than Luna regardless of the cost.

Works fine for me via opencode go.

Cheaper - not cheap!

Morph occasionally have lower prices than the standard rates, and:

https://telnyx.com/pricing/inference-api

Is one which is a bit cheaper... I haven't actually tried K3 myself...

I wish!!

No, I'm using it via OpenRouter in pi.dev - I just used it 30 mins ago... Providers (automatically selected): StreamLake and Baidu Qianfan.

That’s what I was seeing too, and my cache hit rate was below 25% during that same time, leading to significant burn of my weekly limit (Pro 5x plan) via all the uncached input. Doesn’t prove that the overload caused the cache failure, but it does seem to point to some common infrastructure cause. No such problems (cache hit or system overload) with Terra or Luna.

it's kind of hard to save a company billions in operating costs via software without it being part of Silicon Valley (culture)...

Not sure what you're working on as for coding it's still pretty bad at least for more complex projects compared to Opus.

You're right, they've continued the 50% off on the new 80% off price. So it's now 90% off yesterday's OpenAI price.

Was that Codex/subsidised-usage or API? I do get overloaded in Codex-account from time to time, but API is rock solid.

They obviously load shed a bit of Codex-sub during peak times, and for the amount of tokens you get for a sub, I don't mind. I just mean the API where you pay-per-token is rock stable.

They’re using the term colloquially. Deep research afaik is an outdated specific model.

Wow. This is absolutely wild. I didn't expect that.

If we get to anywhere near this speed for the equivalent of the current models... I don't even know what to think about that future.

Holy crap, I was not prepared for how fast it responded. I just wrote "Just wanted to see how fast you are! Can you write me a quick story about a tiger who lives inside a block of cheese the size of a house?"

I pressed Enter, and the response was instant.

> Generated in 0.037s • 14,205 tok/s

This is unbelievable.

So which shovels companies are the ones to watch for burnt in silicon models ?

Wow! You weren't kidding,

I just tried it too and 14,098 tokens in .05 seconds, I barely blinked and it was done. There was no typing at all appearing on the screen. It just showed up.

https://chatjimmy.ai/chats/01dc66a4-4b1b-4dea-bb5f-926855e37...

I think this is an unsolved problem. The most interesting thing I saw here is the Recursive Language Models paper.

https://arxiv.org/abs/2512.24601

There's also a great write up here by the author:

https://alexzhang13.github.io/blog/2025/rlm/

> Luna is an extremely strong model.

By benchmarks, which sadly is a poor measure. Yes Luna is a good model under certain circumstances. Whether it is great for general usage is another story. Sonnet is definitely better when prompts are more vague and it needs to decide things. Luna generally sticks to things very strictly and goes off in bad ways.

Yup, last month I did ~150mil tokens on DeepSeek v4 Pro for just under $3

I'm not who you responded to and I don't have any info on Google. Nor can I explain in detail due to NDAs. But multiple major players are working on something along the lines of what the parent is alluding to.

The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.

Could OpenAI be running a different model for high+max?

Wasn't the original proof something like "a program that loops indefinitely when the oracle says it will halt, and stops immediately if an oracle says it will not stop"? The whole point was the you can construct a contradictory program that cannot be decided by any oracle by its very nature.

```python

source_code = open(__file__, 'rt').read()

response = ask_llm("will this program halt, only answer 'yes' or 'no'?\n" + source_code)

will_halt = response == 'yes'

while will_halt:

   continue

```

You might need to reread that mate.

With the right kind of credentials, it's not about need, it's about want. Flip the roles and let yourself become an object of desire, an aspirational hire.

hugged?

Speed is the metric I'm currently most interested in. The models are smart enough.

Once speed significantly increases I think we're going to see some interesting downstream effects. The three things I currently spend the most time waiting on are LLM API requests, Rust compile times, and nix derivations. As AI latency approaches zero I think we're going to start taking a hard look at whether slow-compiling languages are adding enough value over Golang, Typescript, or even dynamic languages to be worth the slowdown.

Just wow, I made a similar request. Result: Generated in 0.042s • 14,201 tok/s

This is crazy.

"Stochastic gradient descent algorithm in Haskel"

"LMS algorithm in bash"

Just barfed it up lol.

Amazing.

I used this architecture for a while. The problem I have with it is that starting from one agent and fanning out keeps things mostly aligned with that single reasoning trajectory, even as you get a few layers into the stack. Every recursive invocation is a product of the caller's current state. Diversity doesn't really occur on its own unless the environment (tool calling) is complex/chaotic.

RLM might be more useful on the execution side than on the research side. In fact, these somehow feel like they might be exact inverses of each other in terms of what the ideal architecture looks like. At some point you definitely do need something in the middle that has it all sorted out.

Imagine the price of a $9 million NVL72 dropped to about $100, used 4 orders of magnitude less power, was the size of ARM cpu, be bundled with pretty much any electronic device, and ran as fast a frontier AI is today.

That's about how disrupting DSPs were to the industries they arose out of (over a very long time frame).

How would that disrupt the industry?

I can't speak for all cases, but the AI space is seeing improvements month by month, so it is beneficial to wait until it settles (a model becomes the standard in intelligence/price) before designing and mass producing an "LLM ASIC" of said model.

The big AI labs won't do that unless they are forced to, as they want you to spend more money on the big, expensive, frontier models (so they can live up to their valuation), so it's more likely that you will see this on smaller open weights models.

There is a deterministic framework that enforces the process. I use the raw chat completion APIs. No "reasoning effort" is applied on top of these requests (it's not even allowed anymore), so there's actually no way to influence this via prompting.

It's crazy. Are they doing any precomputing as you type, I wonder if you paste a block of text is it the same speed.

For what its worth the frontier lab models can surely be a lot faster if they wanted them to be but theyre supply constrained so theyre doing stuff like multi tenancy. Since you cant self host them no one outside the labs really knows speed as a solo tenant

That link isn't bringing up your chat, FYI. It just shows the default new chat state.

I'd like to imagine the things that can be done with this speed and the current frontier models.

Because their leaked financials point that way.

Does it actually yield results better than just sending it to 5.6 Pro?

Out of interest, are you using the DeepSeek plan?

(I've been using it via OpenRouter and it's much more than that, but still cheap).

Yes but there is a big, big market for subagents to consume lots of tokens cheaply and condense information up to parent agents. Luna would not be my choice for planning. But an explorer to comb through a codebase to find relevant parts? Or for enterprise retrieval, where it needs to search across many different types of data to see where to focus efforts for a smarter model? Or to wake up periodically to evaluate some conditions and determine if a bigger model should be spun up? Definitely.

I've previously found flash (for all the hate it gets) to be good for these kinds of things. Haiku was fine but it's ancient.

How will this affect the newly build data centers? What effect do you think it will have on memory prices?

> Using Sol or Fable for implementing is like having your Principal Staff Engineer with 30 year tenure routinely write CRUD functions for a REST API.

A Principal Staff Engineer who costs $2400 a year and never feels any work is beneath them? Hell yeah.

OK OK, usage limits

You can kind of get a sense by running these things at home - I'm currently running Laguna. One interesting thing is that per stream doesn't actually slow down that much with multiple concurrents, because the bottleneck remains the memory bandwidth until pretty significant request depths, and then eventually you hit the GPU's limits. It's one of the big forces that pushes for centralization in this stuff, the fixed costs to run one are huge, the marginal costs of additional tenants, relatively small.

Truth, it feels like we're in the dial-up age of LLMs right now. And this Jimmy AI is fiber.

> Yes but there is a big, big market for subagents to consume lots of tokens cheaply and condense information up to parent agents.

That's again not some "intelligence factor" here. Different agents work for different use cases. Luna wins some. Terra wins some. Sonnet wins some. Flash was really good at exploring.

So I'm not sure what your point is? There's a big market for everything. Even within the market you describe it's likely not a Luna-size fits all either.

I'm not aware of a DeepSeek plan, but I am using the DeepSeek API directly if that's what you mean.

> Revenue: $3.7 billion

> Cost of Revenue: $2.65 billion

That's how standard accounting rules for public companies would measure it.

No OAI is just very good at reasoning models

I pasted and instantly hit enter on this prompt: "I generated a filter set using REW v5.31.3 using real-world sweep tone measurements from the room I'm listening in . How can I use it as my MacOS output equalizer so that my spotify music is adjusted for this room and speakers"

and it gave a very reasonable answer in non-perceptible time.

No, it really does take ~0.03s to generate the answer. Try your browser's developer tools and watch the requests.

Seriously, if Fable or even Opus was this fast that would be a real game changer.

Fully interactive games where you can talk to every NPC by text or voice and have an LLM drive the story (with your own meta prompts to guide it, if you so wish). Maybe even have them generate assets on the fly too.

I’m still trying to figure out coding agents. I can’t even begin to imagine the things it would enable. Even the most mundane ideas like LLMs-in-HiFreq-trading have huge implications.

what do you mean "if"? of course we will, and the models will be smarter as well

$CBRS - Cerebras Systems

There is other in the space, Groq and Sambanova are both private companies attempting to develop their own technology.

I'm actually finding Luna to be an extremely strong model in practice.

I almost exclusively use it with xhigh or max effort, but when run like that it's been an incredibly cheap little workhorse for most development work. I'm still leaning on Sol for planning and debugging, but when it's time to start pumping out code I've been leaning into Luna (Max) and I've been enjoying it! And that was before the price drop, it's going to feel practically free at this point

You're right of course, LLMs provide a partial, unsound oracle.

The "halting problem is unsolvable" argument relies on the oracle not being able to output "not sure". But adding that option admits trivial oracles, like ones which output "not sure" for everything, so some are better than others.

The "real world" use most people have for halting oracles is as part of software safety, where if the checker outputs "not sure" you modify the software until the checker can decide if it halts.

Sure enough. I just tested it in incognito, and it is blank. I thought it would work because I wasn't logged in to Jimmy AI. It still shows for me though in my regular session so they must just have a cookie that makes it stick.

Here's a copy and paste prompt if somebody wants to just test it real quick to see what I saw:

Write a story about the fastest monkey who ever lived, his name is Jimmy and he is an AI superbot monkey that is part cyborg primate. He can travel through time and is psychic.

It’s crazy that you asked the very question whose answer spawned the sub thread, within the sub thread.

0.03 seconds is an eternity in high frequency trading

You need to be 4 orders of magnitude faster at least

RSI will be models better than Fable running faster than this, you won't even need a human in the loop to figure out what to do. The high level goal will be accomplished better than the human in an instant

Source?

The magic is in the fact that they essentially have an ASIC llm device. There is no other trickery. The problem they will face is that it is actually locked in silicon, so upgrading models will be difficult, and likely require new hardware each time.

If you have bcherny levels of token quotas, you can use Fable for everything =)

The rest of us need to optimise a bit.

My uneducated guess says, not much. For running massive models you still need a ton of high-bandwidth interconnects between many individual chips/GPUs/etc since you need to do math across a few TB worth of weights. That's simply going to require more power (and more die area in I/O, and therefore more cost). Being able to run small models in tiny power envelopes is incredibly useful to people, but I believe it will be covering a different niche than what datacenters can provide. Likewise, you'll still need crazy amounts of high-end memory to populate whatever goes in these datacenters.

The only thing that will crash prices is reduced demand (duh) or, more interestingly, increased production. In particular, if CXMT is able to get their DDR5 fabs up to a reasonably high yield, that could add some downward price pressure (as could government subsidies). As well, if Micron/Kingston/Hynix think that CXMT is going to start cutting into their market share, they might be willing to either increases supply or drop prices. Unfortunately CXMT looks to be taking quite a while to get their new fab up to max capacity so that may take a year+ before anything manifests.

If you're interested in following the (publicly available) info on these sorts of things, check out what companies like Axelera, DeepX, and MemoryX are doing today and have on their roadmaps, as well as the sorts of chips/SoCs Qualcomm, Kinara (now NXP), and Ambarella currently have announced (or have on the market). And remember, that pretty much all of these chips on the market today were in initial development more or less when ChatGPT first launched. If you knew what you knew today (or a year ago) about what requirements current- and next-generation models would have (from a silicon perspective), what might you do differently? Think for instance, host system interconnects, amount and speed of on-package or on-die memory, image/video decode capabilities, int8 vs fp8 vs fp16 vs bf16 compute units, etc. And, consider that most "AI" stuff in development a few years ago was all 15nm or 12nm - because who was gonna pay big money to get fab capacity at 3nm to run some object detection models? So most of the stuff on the market today is on very old nodes and therefore not super power efficient.

That's a hell of a prompt, you should make a post about what you're doing maybe, because I want that for myself.

Isn't it crazy that we can send a message, across the world near instantaneously and have a coherent reply, generated by a computer, sent back to your screen, in under 500ms in most circumstances.

I find myself getting caught up in the sheer speed of modern computing and networking. The fact I can play an online game with 10 other people is just insane.

Yeah. A model smarter than Fable by, say, 50 human IQ point equivalents, running in some kind of autoregressive process continously on whatever goals you put into the loop, on baked silicon, would be maybe my low end for the potential in the relatively near future.

Even if we had an answer to the halting problem, from a practical point of view, it's solving the wrong problem anyway. There's no point in distinguishing a function that would run forever from one that would exit after a century. They are both too slow.

Knowing that functions terminate is important for proof languages like Lean, where you often want to prove things without running the code at all. You're proving that one could, theoretically, calculate an answer, without actually calculating it.

Cerebras and AMD collabrated on some Helios system stuff.

Yes, I've seen this too and how Luna xhigh is so good that Terra doesn't really serve a purpose because beyond that you can continue at Sol medium. This can be the most cost efficient way, and especially now!

Is the size of the ASIC limiting factor or can they infinitely tensor parallelize? If thats possible then it would make it only a matter of economy of scale, there are good enough models already for people to invest in that kind of platform

Thank you for such detailed answer!

At some point I imagine you’d add a software layer on top that holds more current training and can be called as needed trading off for slower responses. There’s already work out there splitting models across networks. You could have the base on silicon, some stuff in memory on the machine, and another frontier tool in the cloud.

I got into Rust development via LLM last year, and being able to do things budgeted in nanoseconds is a heady feeling indeed. Real time video and audio analysis? Totally doable, plenty of time budget. 16.6ms is a long time, it turns out.

The company losing money does not mean the model inference in API is 70% subsidized- that’s a crazy leap in logic. Obviously the massive number of free chatgpt users are getting subsidized 100%.

There are already studies proving that a "stupid" model with a good harness + tool calling will outperform a "smart" model.

Things like this give me hope for a system that can be fully local and private, but also with the ability to be almost infinitely extendable with tools.

Taalas actually already support LoRA, basically doing exactly what you say.

The other thing that I think is really interesting about all of this, is that LLMs are already perforce behind the times with their knowledge cutoff, so adding an additional ~3 months for bake into silicon isn't such a huge deal, I think, for the ~10x more efficient and faster you get.

Yeah, really a fascinating time in computing. Once these chips start to become more common I'll be curious how people find ways to use them. One can imagine a world where really simple inference is available on dirt cheap chips found in toys and other low cost consumer electronics.