I expect future Chinese models to introduce even more of this type of bogus "safety" training.

Looks like if you are a white hat, then you will be fighting an uphill battle. Black hats will be fine, they will not care, they can just run a heretic model or specialty trained model.

Care to give more context to this? Seems interesting

Any prohibition on open source models will do nothing to fix the problem.. since attackers will never feel bound to the law. All advanced models must be available for defensive purposes.

I followed this example

https://minimal-agent.com/

but I'm running into issues with nested backticks so I'm thinking of making dedicated close tags per tool call.

Fable found a couple of good ones, then we lost Fable, so I tried GLM5.2 and it found two critical bugs that Fable had missed, so it got my seal of approval.

Deepseek v4 pro being in preview with less "safety" training makes it stronger for that reason. Thinking will be different and in the end, it will actually try to be useful. Just expect future Chinese LLMs to further push out "safety" guided LLMs. The future is bleak for open weight models. Prepare to have "guidelines" enforced unceremoniously to all.

A few people are running highly quantized models with limited context windows. It's still impressive, but not the benchmark level intelligence. Very few people could afford a rig for reasonable local performance at a reasonable quant, at full context size.

The antirez example is 2.6bit quant, 32k context, and few tokens per second... on a ~$7000 MacBook M5 (new RAM pricing).

If huggingface or whatever is forced to take down open source licensed weights, there’s always bittorrent.

Export controls are one thing, but the US doesn’t really have import controls, and there’s no copyright issue, so DMCA, etc don’t come into play.

It’d take the courts years to decide how to contort the law to ban open weight models, and by then, it’ll be too late (and also pointless).

But that's the whole point.

Fall out of favor with the admin and you lose access to the good American models, aren't allowed to use Chinese ones, and fall prey to the attackers and behind your competitors.

In multi-agent coding environments, GLM 5.2 is just shy of Opus 4.6 on average. Data at https://gertlabs.com/rankings

But when factoring in performance/cost, GLM 5.2 is the frontier model.

I don't think a $100 session is "typical". I use GPT for months. $20/m plus plan is enough for my daily work.

Overall, I still think GLM 5.2 is the much stronger performer. It's hard to tell the difference between GLM 5.2 and Opus at <120k tokens.

…probably already is one

Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years.

For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of money in local models unless you have a business where you're already paying for many employee's individual token usage.

I’m sceptical they could find the legal framework to do this even if they wanted to

They have legal authority to (a) prevent export of US goods/services; (b) ban imports of physical goods; (c) ban transactions (including purchasing services or license agreements) with foreign firms

But I’m not aware of any legal authority which lets them ban US firms from running a Chinese-developed open source AI model in the United States, if they are at arms length from the vendor, and aren’t using it for government contracts or regulated applications

Possibly they could order HuggingFace/etc to suspend Chinese accounts. But if someone in the US (or a third country) downloads the model from China then reuploads it to a US server, completely independently of the vendor - where is the legal hook to prohibit that?

US imposing export restrictions on a model from China?

So your question is really “if they’re giving free usage, why not take advantage of it?”

I do, so I don’t know the reasons not to, other than to experiment.

I kind of wanted to see if I can make a Matrix agent from scratch with Rust with GLM and it was surprisingly easy. Just make something for myself how I want it. Maybe I'll take a look on Hermes later...

Likewise, I don't understand how anyone survives on the basic plans. It's funny seeing these two camps not understanding what the other is doing :)

But, there are benchmarks for so many different kinds of ability, I don't know how to compare them directly against one another. Like, models that do well on terminal and agentic coding benchmarks tend to do well on finding security bugs, but it's not a 1:1 correlation, there are surprises.

Anyway, it isn't possible for any of the models, so far, to be trained on the Mythos bugs. We're getting closer to the point where I have to worry about that, at which point I'll roll forward and pull some newer CVEs from what they've published, assuming they keep publishing new bugs. (And, if they don't, it's trivial to switch to just random CVEs. But, finding out what Mythos is up to is interesting.)

Personally, I’m waiting for hardware to hit the secondary market before I buy something to run unquantized models like GLM. But I have no doubt that I will, at some point.

I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way more stable. Almost like buying a Ferrari for your daily commute instead of a Toyota or even a Mercedes.

I think there are several factors. Certainly marketing making us think we need the shiny thing which is rampant online and very smart people think they aren't susceptible to. There's a lot of really odd 'I trust Anthropic/OpenAI more than Deepseek' which tends to ignore, for starters, that you can run choose your provider and still save a ton. I also think there's some amount of addiction and brand loyalty where a Ferrari is one hell of a drive so that you turn your nose up at that sensible Toyota. Oh the other one I see used is like oh only fable can oneshot updating my embedded systems thing from 1975 to rust which is great but let's recognize how niche that is.

And it ends up just coming across as people are getting SO reliant on the tools so fast. Maybe it's ok to think and like read a few lines of code and work with these agents to convert your thing to rust or center your div. Even if coding is over which in some sense it certainly is, don't turn your mind into the wall-e people yet. I found myself guilty of this so often. It takes way more time and effort to do things via prompt and I wouldn't just open the editor and fix it because that dopamine hit of the magic the abstraction provided was so strong.

So I'm pretty much done using the 'best' (on benchmarks, if money isn't an object, etc etc) models available. After a year on Sonnet/Opus/GPT5x I'm having way better results with open weights models that don't get lobotomized weekly. I'm finding ways to do the crafting part of building software by focusing on honing my harness and workflow. I'm enjoying changing the oil on my Toyota after a year of almost flying off cliffs in my Ferrari and if I can check my ego it's a purely positive thing.

That’s not super surprising to me, but, given the apparent randomness of the stack ranking, is GLM actually worse than any of the Anthropic models? This looks like a 10-way tie to me.

On the contrary, I personally think these broader benchmarks are meaningless. I think personalized benchmarks are the way to go. They should answer "How does this model perform for MY use-case?" rather than trying to answer "How does this model perform across all coding environments?"

Case in point: I use Elixir which is not as popular as Python, is always a hit or miss with most SOTA models at the top of these benchmarks. Whereas, the ones in the middle of the benchmarks (like the GLM) almost always outperform even SOTA models from Google / Anthropic. However, this is relevant only for my use case and I wouldn't just advocate a model for everyone based off my use-case alone.

Anthropic won't even let you run "claude -p [prompt]" any more... They bill it at api rates.

So if you're trying to automate the ai (and seriously, that's the point) the subsidized plans are crippled.

My typical session ranges from $100-$400 - higher end when using workflows with lots of subagents. $100/session is expected when using the API without the subsidized subscription pricing. Most larger orgs have to use API pricing AFAIK.

[1] https://github.com/simple10/agents-observe

I'd blow through $20/month plan in hours.

I'm inclined to use DeepSeek V4 Pro the most, because it is consistently extremely strong, it's very fast, it's very cheap and has excellent caching and cheap-as-free cached input tokens (something like 80% of token usage is cached when I'm using it for security scanning). So, my probably "pair" of frontline security researchers will probably be DeepSeek V4 Pro and Gemma 4 31B self-hosted (another shockingly strong contender, competitive with the best models once you let it loop on the same file a couple/few times). But, I won't be surprised if GLM 5.2 turns out better than DeepSeek V4 Pro...it costs quite a bit more.

8 x RTX6000 GPUs cost $100,000 alone. You then need to build a system that can support those GPUs with enough PCIe lanes through a PCIe switch.

It's going to be $120K to $150K to build or buy a system to run this.

Not sure if you're being sarcastic, but I can run a quantised version of Gemma or Qwen on my 16GB M1 Macbook Pro that beats GPT-4 from 2023 hands-down.

I wouldn't be surprised if, in another 3 years, you'd be able to run something as powerful as Opus 4.5 or GLM-5.2 on standard consumer hardware - say a 32GB/64GB M7 Pro.

I also wouldn't be surprised if, 3 years after that, cheaper hardware and improved model efficiency means that there's a much smaller gap between what you can run on a consumer CPU (which, with memory prices coming down, could look like a 256GB M9 or M10 Pro) and $100k GPU cluster.

oil workers buy 100k trucks they do not-much with. why not a 100k in computer?

Did you program or did you gave the order to an agent to program?

Just want to express how amazing that is. Opus 4.6 is an amazing model. That an open weight model like GLM 5.2 competes with it is nothing short of outstanding.

I like reading benchmarks, but I take them all with a grain of salt. They're just to tell me if the model is worth even trying for my task. I've heavily used self-hosted Qwen 3.6 and Gemma 4 on a bunch of different tasks, and while the benchmarks consistently say Qwen is the better model, I simply don't find that to be the case for anything I do. I think Qwen is tuned for benchmarks, while Google couldn't give two shits about most of the benchmarks, they're just busy making unusually smart tiny models.

Or even just electricity costs vs token cost

If the real motive is profit, then open source models are likely simply not a viable means to that end.

The difference is how the model is used.

With Opus you can give it a long-horizon task (eg build an entire feature) and it will plan it out and implement it and almost always stay on task. This is what people mean when they say "agentic tasks"

With the lessor models the code is fine, but they need something else to plan what needs to be done.

GLM-5.2 is the third model (after Opus 4.6+ and GPT-5.5) that can do this agentic style work.

Notably Gemini 3.1 Pro is notoriously bad at this style work - the code is good, but it drifts off task most of the time. 3.5 Flash is supposed to address this, but I haven't had a good reason to try it.

I used to be obsessed with what's the best model. Then a while back when the new best model came out, I tested it on a task. I also tested its little brother (much smaller model from same company).

They both completed the task perfectly except the "best" model (the bigger one) cost 5x more and took 3x longer...

This also allows me to play with, and mix codex, claude cli, and others. This is my happy spot for the last two months.

We now use Sonnet 4.6 for a number of internal use cases we wouldn't have considered otherwise.

We test 11 popular/interesting languages (you can see the Languages chart to filter), but not Elixir -- although other evaluations have found that many LLMs solve more problems when working with Elixir [0]. Why models write code well in some languages over others seems to go beyond pre-training data (Python scores quite low for most models) and we don't fully understand it.

[0] https://elixirforum.com/t/llm-coding-benchmark-by-language/7...

Sounds like this is the year for coding.

Most of the gains right now come from tooling and process and any big post 2025 language model. The specific model isn’t that important right now.

> In May, we sent you an email announcing that starting today, the Claude Agent SDK, claude -p, and third-party apps built on the Agent SDK would stop drawing from subscription rate limits and move to a dedicated monthly credit. We're writing to let you know that we’re not making this change today. We’re working to update the plan to better support how users build with Claude subscriptions.

> What this means for you

> Nothing changes for now. Agent SDK, claude -p, and third-party app usage continues to work with your subscription exactly as it did before today, and there's no credit to claim. Your subscription limits are unchanged. When we have an update, we'll share it with advance notice before it takes effect

But hey you could save on heating?

The real gangstas are running 16x RTX6000s. Too rich for my blood, and the NV4FP quant doesn't seem to be that much worse.

We've been sat with basically the same PC specs for ~20 years - our current specs are within an order of magnitude of the ones we could buy back in 2010. This is not really constrained by tech, as we could have much, much, larger machines. It's more because there's no mass demand for much, much, larger machines - if it's big enough to run Office apps or VSCode then you're good to go. The exponential growth we saw in the 90's was driven as much by software demand as it was by hardware development.

I can see the next 10 years produce the same kind of push for larger machines that the 90's did. And we should probably expect the same kind of standards churn as our existing technologies for storage, memory, etc, don't scale up enough and new technologies become worth developing because there's demand for them.

Prices aren't going down, and consumer platforms are being shipped with less RAM so we can be sold cloud products. This isn't going to happen.

Can you please explain to me how you're going to fit 700bb-1T params in 64GB of RAM? You realize there are memory requirements proportional to model size?

Mythos level really doesn't seem that scary. And it would be a great way to take away the American labs international market.

I think it would make strategic sense for them to release more capable models than what American labs are allowed to make available to the world. It would help them grow their global soft-power and be a destabilizing effect on the American economy.

The input cache hit tokens are incredibly cheap for them, (incredibly high margin too, except for deepseek).

And input tokens are in the middle. Input tokens can be processed very efficiently.

Also his math is wrong. $100k gets you 22.7B output tokens at $4.4/M which is how much GLM 5.2 costs.

At 500/s 22.7B is just 500 days. Or about 1.54 years. Which is much less then the life of the hardware.

concurrency

In the future none of us do, so it's better to trial how the actually adorable models perform.

Which would be fine, but as we know, people securitize the crap out of their investments these days, and least some people probably leveraged themselves on some US AI companies, so now the risk is spreading outside of the sector to the economy in general, which is made worse by the sheer amount of spending on AI.

There are many layers of Chinese govt. But GLM is backed by Beijing municipal govt and Tsinghua University.

[0] https://gwern.net/complement

I wouldn’t call that “no money”

The only niche where it doesn't utterly dwarf the competition is personal computers and it looks like we're all getting priced out of that anyway

wild guess - I wouldn't be surprised if Opus 4.6 was run quantized for a while, and 4.7/4.8 have QAT for that nerfed size.

no one has a source, because no one knows closed model parameter counts. we have only heuristics which strongly indicate that Mythos is simply a big fucking model that any other lab could make an equivalent of.

The only ones who seem to profit are the ones running smaller Chinese models. Even NVIDIA seems to have to "reinvest" their profits into sponsoring companies to buy their cards now.

I use DeepSeek V4 Flash (high) and MiMo 2.5 (non Pro, because vision) to work on medium sized projects (~1mil lines of code, C#, Go, TypeScript) with great success.

And that is coming from someone who used Opus 4.7 and GPT 5.5 as workhorses before.

And I'm pretty sure GLM 5.2 is better than the lighter models I use.

My worflow is simple: plan -> clarify -> implement.

1) plan prompt template: I describe what I need and ask LLM to generate a markdown file containing an implementation plan plus at least 10 clarification questions for me to answer.

2) I answer the questions in the plan.md file.

3) implementation prompt template: I ask LLM to implement plan.md and tell me at the end if there were any deviations and new findings during the implementation (there ofter are).

https://taalas.com/products/

Isn’t the performance gap between quantized and full models indicative that even if you aren’t using it directly, the model knowing the colors in the Russian flag does have something to do with the intelligence you demand?

China could not be happier.

The same model is going to apply to the silicon supply chain as well is my guess. 1000th the expenditure in exchange for being a little behind the curve.

I worry it will have a very real chilling effect on research and development, since customers will probably very quickly switch to the thing that costs 1/10th as much, sucking out the ROI.

Over the last decade, the US has been way more unreliable than China. There's been a near constant negative impact from the US doing something.

At least with China, we are very good at winning trade wars with them here in Australia.

Can they actually though? Do they have legal authority to tell a payment processor that it has to block transactions of a legal US company, just because the company is hosting a Chinese-developed open source model? I’m sceptical

And what about companies (e.g. AWS) that let you “bring your own model”?

The weights are already available and downloaded, is it going to be a crime to have them, run them, make them available? Constitutional rights still exist (I hope)

Price and speed, for me. GLM5.2 is "good enough" for some tasks, but rather slow (on their coding plan). In the time it takes GLM to "read files to figure out...", gemini flash is usually finished. It's not SotA for coding, but it's fast and often "good enough" for normal tasks.

Same for me, I certainly don't have the same definition of success and failure either.

A more expensive model has *less* rooms for wandering around than a cheaper model.

If Claude wanders around during 10min until finding the most obvious solution, then I count it as a failure.

You don't. What they're saying is that today's small models (that fit on consumer hw) are better than yesteryear's top models. GPT4 was reportedly 8x 220B (~1.6T) MoE, and today you can run a 30-120B model that beats it handedly in real-world tasks.

Similarly for 4-20B models beating GPT3 (175B) and so on.

There is a sweetspot of "good enough" that the small models can reach, where you get equivalent tasks solved fully locally. They'll never touch SotA, but they'll reach 2-3-4 year's SotA. Which, depending on the task you need, it can be "good enough".

I know Google gives me free Gemini AI from my Google Drive plan. Microsoft probably already does too, didn't test. Apple is probably crafting some arrangements if not offering already.

My point is most people wont pay for AI. It will be bundled.

And I think AI is going to be free for all, with ads.

This is the logical end point of the fear-based way LLMs are marketed. You must want the best, because everyone who has the best can work faster than you, generate more — if you don't have the best, you are behind! Why would you want to use anything other than the best?

The thing is, once everyone has the best, the question is: how much can you spend? If you can't spend more, you are behind! If spending the most will get you ahead, why would you not want to spend the most, if you can afford it?

There is only one way through this, in the long run: work out a way forward that doesn't make you dependent on this cycle. If you can compete at all, without the spend, what happens is: they burn money and you don't.

FWIW so far I don't think the benchmarks prove very much about the actual experience, and you can discover this just as easily without spending any money. And we know this about benchmarks! Once a benchmark seems useful as a measurement, it becomes a target and it stops being as useful.

I think your strategy is right. It requires bravery, and as you say, it requires ego balance. But I believe it is obvious that the world will either come around to a more sensible, stable pattern or it doesn't matter either way because we're fucked. So opting out of this mad early cycle and choosing to be calmer and happier is a choice you can just make.

There are Business and Enterprise plans, both have discounting.

Are they?

I suspect AI labs are buying stuff not just for their own use, but to make local use too expensive to be an option :-( And they can always make the "best" frontier model even bigger (though only fractionally better) so it's always out of reach of local use, while consumer laptops have nearly the same amount of memory they had a decade ago.

    m                  o
    o
    d
    e
    l             o
    s
    i        o
    z    o
    e  2020 2022 2024 2026
    
    
    c                  
    h
    e
    a
    p             o      
    R        o     
    A    o                
    M                   o
       2020 2022 2024 2026

I've had no trouble getting the current generation of smaller models to do the same thing. Maybe it's more of a harness issue than a model issue?

Recently I've used both MiniMax M3 and DeepSeek V4 Flash to one-shot moderately complex applications from a written spec, and neither one got lost along the way

4.7 was so bad, I locked a bunch of my machines to 4.6.

I haven’t bothered locking the 4.8 machines to 4.6. There was a HN thread a while back where they run swe bench a few times a day and measure success rate and latency. It showed opus getting significantly dumber for the week before a recent launch.

It wouldn’t surprise me if they’re quantizing to improve margins or to hype models in comparative testing in order to defraud investors at IPO.

Or, maybe QA is hard. Anyway, I think they hit a performance wall sometime at or before 4.6.

I see no reason to trust Z.ai more than other vendors.

A single circuit using 10mm TPS would technically be enough to run what you’re describing. Might be pricey though, I’d probably take the excuse to get 3 phase installed so I could get access to the stock of used 3 phase machinery.

Not sure a project nobody knows or uses is much better in this regard?

Likewise, LLMs do not violate the laws of information theory, and therefore the only way to encode X amount of information in Y amount of bits where X > Y is by performing what is effectively lossy compression, and as X grows larger relative to Y the compression ratio must change to lose ever more information.

Yes, for the sake of making chatbots that are "conversational" in that they can interpret natural language as input and produce code as output you can easily benefit in incidental and unintuitive ways by training it on more natural language text. But for a given fixed parameter size, it's possible to produce a better model for a specific task by selectively not muddying its training set in the first place with things that are likely irrelevant to the task.

One group is trying to get the LLM to basically one shot everything and not properly reviewing the output.

Others are using the LLM to assist their human intelligence in a tight loop.

If you’re doing the former you really do need the best model available because that’s still right on the edge of what LLMs can do at best, and at worst you’re just shipping pure unmaintainable slop.

If you’re doing the latter then you can get away with a slightly less powerful model without it making a material difference because your human intelligence is filling in gaps

I am still struggling how to deal with sub agents and different roles for each model. I still think Claude or Codex are overall better models, but everything around them transpires such weird vibes, including, and this one kills me, that at certain times they feel like dumbed down.

I keep changing these things often, but I have basic subscription to codex (20$ plan) which I use with GLM 5.2 to do some high level planning of what I intend to do, and then leave Deepseek do the coding. Or something along those lines.

Point is, GLM 5.2 is now at a point where I cannot tell you if it's better or worse. I can tell you however one thing: no matter when I use it, it's consistent in what it does and how it works.

Then there is the Fable thing, but as with many things, I think the past has distorted the reality. It lasted two days, but Anthropic said it clearly for plan users it would only be there for two weeks. It was great for doing what you can already do with other tools: doing all the planning, and reviews, and launching a million subagents talking to each other. I sometimes wonder if it was really a new model, or just Opus 4.9 wrapped with some fancy model driven harness.

With the wealth of models available (open source vs closed, api vs local), I find optimizing the cost-efficiency of your token consumption an important part of business-oriented AI engineering. You don't need "the best" for every task.

Congrats, now you’re paying an engineer’s salary to make your engineer at best 20% more productive.

Better to hire another engineer, or two jrs, and build up your in house talent.

My productivity profits from the best intelligence available, a decent context size, and a batch size of four.

While my MacBook has 48 GB of RAM, not only do I want the above requirements at a decent speed, but I also need my machine to run the development tools and test suites, ideally without the fans blasting at full load.

For the foreseeable future I will stay with providers rather than local inference, apart from niche use cases.

It doesn't have a higher capability score than Fable, though. We break our coding evaluations into 2 parts, and "one-shot coding" makes up part of the index, where Fable significantly outperforms every other model, which is why it's ranked at the top despite Sonnet 4.6 having a slightly higher median (and lower average) in long-horizon agentic workloads. One-shot coding tends to be the most correlated with other companies' model cards, whereas agentic coding is partly about how well a model can adapt to a custom harness. Fable also refused some tasks.

Data at https://gertlabs.com/rankings?ow=1&mode=oneshot_coding

In the US it's common to get 200A 120/240V split-phase service. We're talking about the wiring inside the house, though.

How do you think everyone here is charging their electric cars at home and running our AC and electric cooktops at the same time if we didn't also have that? :)

You need to derate for constant loads here, and I assume you have to do that in NZ as well.

So, no, not a "uniquely US issue".

But the existing tech we're using for 16Gb probably isn't going to scale to 16Tb at a reasonable price point. And the price point is relatively inelastic - people are used to paying <$5K for their computers, and they're not going to go much above that. You'll get early adopters paying $10K or more for a machine that large, but not the early majority. And even then, obviously, $10K is not going to buy you a 16Tb memory machine.

So there's room for a new technology to come in, where there wasn't previously. This is what happened all through the 90's, and we churned through a bunch of standards and technologies to try and keep up with demand.

Some, and the market fluctuates a ton.

> corvettes

Only the oldest, most unique model years: nobody is buying (C4-C5-realistically C6) mid-90s or early 2000s Corvettes for more than what they paid for them, and they never will.

Both of those things' value drops like a rock as soon as you buy them and, at least for cars, they don't all appreciate. Most don't. Even so, they appreciate at an incredible slow rate.

I can't speak for watches but I'd be surprised if it wasn't the same situation.

At least the gpus can create value after you buy them before they are worthless.

We don’t understand AI or natural intelligence well enough to make such statements. As for self evidence, cross-domain competence in humans and the rise of generalist models over domain-specific ones (on competence, not cost) seems to pretty directly tank your hypothesis.

And hey, don't get me wrong, you can get pretty far with just prompting. But the subtle misses and (I'm looking at you GPT) the overengineered 20k line PRs to do a simple thing are going to cost you a lot if you're not vigilant.

They did this to themselves.

The compression is almost certainly in part specific knowledge getting fuzzed.

It's hardly self-evident, and your counter-example is hardly applicable.

The first 10^50 of pi is not the same as having BREADTH of information in the training data, which is the whole point not just any random "information that is irrelevant to your use case".

not to mention that the first 10^50 digits of pi compress to quite small formula, so not much information there to begin with from a shannon/kolmogorov perspective

As far as US EVs being subsidized early, if you take state and federal tax incentives, DoE grants and loan guarantees as subsidizes then that's true.

It's debatable (I think incentives applied to all suppliers not just US ones) but a reasonable statement.

As for Fable: I used it as much as I could while we had it.

It was a step change over Opus with my work.

I'm in Australia, so we're probably not getting access to Fable again. We're learning that a faster model + better harness/framework > smarter model. So being able to run GLM5.2 locally and super-fast would be great.

I assume (since they claim they are selling the batteries to AI data centers), they’ll produce some sort of EV >= F150 once the bubble pops, and we get a new president.

I've benchmarked it, and the "here's a repo, find bugs" approach finds far fewer bugs. Like, dramatically fewer. Models are good and contexts have expanded, but focus still wins with hard problems. You could probably tell the good models to make a plan to audit the repo, and it would end up making its own "loop" in the form of a checklist of files to look at over several sessions or via subagents, I assume.

It's not fast-changing, it's not abstract, it's just not that difficult, and where it is difficult, the AI cannot help you, because it is not capable of things you are capable of.

Learn CAD yourself. Honestly; I was sure I would never manage to learn CAD but it turns out to be interesting, rewarding, valuable and actually quite quick to learn.

An LLM certainly is not going to be able to do it better than you once you have a tiny bit of experience. (PCB design, perhaps, has a language to it that an LLM can make a bit more headway into, but as a non-PCB-designer I would still bet that it's more like CAD than code)

I generally started by reviewing but after a while (maximum in hours), I just can't keep up and resort to LLMs as sole reviewers.

While the single functions/classes/structs/... can be well though out the code tends to lack cohesion, and especially maintainability. For instance, it never thinks: "I could put this logic in an interface/trait so that if the requirements change I can simply add a concrete implementation that satisfies the new requirements (and potentially use one of these for testing)".

If that's anywhere near right then it seems like a no brainer.

EV is a separate thing. Vastly overmarketed for the technology as it exists today.

so Tesla technically is subsidized by US govt. SpaceX too. Without NASA funding, they'd be long out of business.

China and US ain't that different.

China realizes that being a tech and industrial powerhouse working on future tech is great for their economy. They bet huge on it. That's how they win.

Europe on the other hand is now a laggard.

In other words, they're not subsidies for Chinese cars being exported abroad. They're not even directly paid to the manufacturers.

But there are EU only providers for GLM5.2. For example tensorx. Depending on your definition of "secure" it may be acceptable.

Not sure if helpful but in my experience when something a bit more complex needs to be done, manually making it read the context I know the model will need for it to solve it well (like making it consume all the project docs first) helps with getting a more satisfactory result instead of only giving it the task and let it look around and consume the context it thinks it needs.

Will test your bug finding method in a current project of mine both with my "manual" context preloading and without.

SoTA models can do reasonably good jobs on each ticket, but over time the architecture of the application starts degrading without a human in the loop.

The entropy increases slower with better models but the trend is always towards slop

The memorization of say 100000 world facts through training texts, which enrich model associations all around, is absolutely not the same as rote memorization on 10^50 digits of pi. Not for a human, and even more so, not for an LLM.

An LLM trained with digits of pi and one trained with books and posts, even if they both have the exact same amount of bytes of training input, would not be comparable in any way in utility and reasoning capabilities.

>There's an infinite amount of information that we could shove into a model, and a finite amount of bits with which to store any of that information such that it can be usefully recalled or form useful logical associations.

Which is irrelevant. Anyway, the amount of information that doesn't form useful logical associations is even larger (e.g. actual human books vs possible permutations of characters and spaces). Just like those (random) possible permutations of characters aren't good for LLM input to get logical associations out of it, pi isn't either (logical associations of the kind we care for and expect, not of the kind related to pi's sequences).

Also it's not only not self-evident, it's also apparently wrong.

It has been hard to explain that they are in fact just creating toy versions and there is no way they can do it without learning the underlying architecture. But they just keep going wasting 100s of dollars , lost in a sea of bugs

For real work anything below 60 tokens per second is essentially unusable. That's not taking into account the prompt filling, Llama 3.1. 70b on DGX spark runs at about 800 tps running at that speed prompt filling a 512k context takes like 11 minutes.

I have not tried it but I will take your word on it. I don't think Qwen3.6 cuts it for large scale coding work. Reading issues, reading code sure, but biting into large issues no, it goes off the track consistently.

Depending on budget it may also be affordable to spin up servers to run it on demand.

You're making the assumption that anything produced by a human necessarily contains more useful information than random noise does. This is false. Even when only considering human intelligence, it's entirely possible to absorb information that makes you stupider, not smarter; learning is only valuable if you actually learn the right things.

Dabbled with OpenSCAD as we will. I decided to learn FreeCAD and what I discovered is that, even putting aside FreeCAD's many documented issues, parametric GUI CAD is not an imprecise, clumsy or fiddly way to work.

It is expressive, precise, generally capable of all the things that code-CAD can do and much more, and it's much, much quicker to work in, once you've learned a few core principles.

As you say, there is an underlying architecture; it's not just a sort of 3D paint package.

The problems the text-as-whatever crowd have are all Dunning-Kruger things in the truest sense.

People who are unaware they are unskilled in a particular technology are unlikely to successfully replace it with another. Particularly one that requires describing the problem domain in precise language.

Quite often when you see text-to-CAD discussions, especially here, there's evidence of profound misunderstandings from the people who think they are going to automate it. They assume their frustrations with the tools stem from limitations of the tools, not from the limits of their understanding.

As a person with decades of experience of code I have found learning how to use LLMs effectively to be much, much harder than learning CAD.

I'd say this exchange is a fine example of that :)