So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python.
Then when I want to research electronics components, I grab a 15B model of agentic research techniques, and add in 10B of electronics knowledge, etc.
I don't want general purpose models. They try to be everything to everyone. I want to click together a model that is laser-focused on what I am doing, and I want to run it locally
Specifically, creative writing driven by nerds dreaming about a future, without proper grounding in reality, constraints and all that stuff.
Which is kinda ironic given the topic. And also important to do, because we should keep dreaming. We should just also be aware of when we are doing that and mark it as such.
So newer data would be interesting.
(It seems a bit like an AI generated argument that uses old facts - something that happens to me quite often)
> There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks.
Future?
Even just recently I’ve read of two approaches to this problem:
Cactus have come up with Needle [0][1], which is their tool-calling focused 14 MB model (still an LLM!) – no world knowledge engrained.
And instead of say, tool call structure, VibeThinker [2][3] focuses on reasoning over world knowledge.
Combine these two approaches with a reliable search tool/a safe way of accessing the internet for the model, and you’ve got a probably slightly slower model for factual questions, which on the upside however doesn’t hallucinate.
[0] https://cactuscompute.com/needle
[1] https://news.ycombinator.com/item?id=49246804
There's no reason that an LLM should have a vast number of obscure facts encoded. It can go out to a search engine for such facts. But the LLM has to be clear on what it doesn't know.
(Google's pricing for search from programs starts at $2.50 per 1,000 queries. If an LLM reaches out to Google, it has to pay.)
Edit: I ran this article through pangram and it is “100% AI generated”. Cool.
(In retrospect it looks like they were pretty forward thinking!)
I know is editorialized, but a more accurate title to this content would be either :
Models Are Getting Ignorant on Purpose
or
Models Are Getting Less Knowledgeable on Purpose
To reason properly about the human condition (eg. World War) wouldn't you need to reason on some facts ? And then reason how some "facts" change the human behaviour ? How can you arrive via pure reasoning to predict how a collective of humans act ? We are not reasonable, humans are not logical deterministic machines confined to algebraic rules.
>On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions.
SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy".
>The part I find most promising is what this does to hallucination. When a fact lives in weights, a wrong fact is unfindable and unfixable.
This seems confused. LLM hallucinations don't come from the weights containing "wrong facts", they are artifacts that appear at runtime.
>When the fact lives outside the model, a wrong answer has an address. The model cites a document, so you can open the document. If the document is wrong, you edit the document
You can make any modern LLM explain its reasoning and find sources for its claims. None of this has anything to do with facts needing to exist in weights or in harnesses.
The internet is full of wrong information and I cannot magically edit it to make it all correct, so this doesn't help me.
>if a model is factually wrong a claim with a source is checkable and a claim from weights isn't.
Why? If a model's weights claim that Bart Simpson became President in 2020, why does this fact suddenly become uncheckable?
For example I prefer Kimi K2.6 1T parameter to Flash V4 0731 230B parameter, even if it is less intelligent.
Current AI is like the film company producing TV series or movies
Your question is like a story outline. You tell the film company that this is the movie you want. The AI film company then searches for existing similar stories. If similar stories do not exist or details are missing, screenwriters use imagination to fill in the gaps (remember hallucination? It's just a makeup.)
So you cannot solve hallucination of AI
What day, month, and year was Carrie Underwood's album "Cry Pretty" certified Gold by the RIAA?
If your idea of the smartest person in the world is the guy who always wins tuesday night pub trivia, this blog post is for you. It also gets it's foundational factual claim wrong (as seen via epoch.ai). Very on brand.
https://epoch.ai/benchmarks/simple-qa-verified?view=graph&ta...
https://logs.epoch.ai/inspect-viewer/c79c08da/viewer.html?lo...
LLMs work is being intelligent not having knowledge of everything is ok. But, they have to be intelligent enough (with some degree of knowledge) that where to find the information (search tools or any other tools for that matter)
I'd wager most people have less. In 2022 a 3080 might have 12 GB if you were lucky, 10 if you weren't -- and you paid for the privilege. A current RTX 5080 is only 16GB.
The author is factually incorrect here. Moving information out of the model weights and into the input of the model's context window in no way ensures that the model will accurately output content that was input from the context. This is true even when RAG is used to input exactly the correct data.
I'd always thought we'd eventually hotload loras or MoE experts.
It would certainly be useful on the robotics/VLA side of things as well; more limited mobile hardware, download and load/unload new skills as needed.
Tbf I also don't really care what facts my models have baked in (for llms at least). I care most that the model understands general logic and then general knowledge of some level is secondary. Reason being is that everything is RAG'd in anyway.
Models spitting out well established facts is cute but I don't really ever want to rely on say "electronics knowledge" that exists in a tenuous and vague form in the model weights.
Humans write books (and datasheets) for a reason. Books are RAG.
I think the vast majority of people do want general purpose models. They want to be able to ask it any question, or ask it to perform any task, and for it to do a decent job at it.
I agree that it's really hard (maybe even impossible) to build something that's everything for everyone. But your average (or even above-average) LLM user doesn't want to choose from a catalog to stitch together a model that does just what they need.
I do think for certain domains this is useful and will make sense: the model backing a coding harness doesn't need to know about the politics of 400BCE Rome. But I'm skeptical that many software developers will want to do what you propose, picking knowledge bases that are tailored to their current task or project. And at any rate, for web-based chat interfaces, most users just want to type a query and get an answer.
This is roughly what multi-agent systems are built for.
This is possible with models too, but "making one on the fly" is much easier with agent coordination rather than model weights, since they all speak the same language.
There is an IBM Mainframe vs Google Distributed system division here. Like Seymour Cray said - two oxen or 1024 chickens.
Chickens are harder to harness, so a lot of my work is in sled-dog territory for agent harnesses & command structures.
If I had to guess, the weights necessary to encode "how to program" are much larger than the final step of "output python."
There are "experts" which do divide parts of the model that are found to activate together for specific tasks, so they can be processed in parallel to join the result at the end, but it's nowhere near the granularity of a SwiftUI expert and a python expert. The difference in those things is so trivial from an abstract point of view that it would make no sense. They would be 99% the same.
Distillations also come into this but I'm highly skeptical you could make one guaranteed to only know programming and only in one programming language (especially with as small a sample set as SwiftUI relative to something like C) without its efficacy being hobbled by tunnel vision. Reminiscent of the SpongeBob episode where he empties his mind of everything except fine dining and breathing, then can't remember his name and goes insane. Beyond the basic concepts of general coding and the trivia of syntax, getting anything done requires a large intersection of disparate world knowledge and the ability to apply it to new situations.
The problem is that “finetuning” was a 2023 AI FOTM associated with products/demos that were almost exclusively using it for LLM character role-play/output style purposes (ie not in actual systems where they served a more functional role).
This made people think you could train models without replay/real evals by yoloing it with SFT (this is partially an artifact of that era being much heavier on autoregressive training and not so much evals). You really can finetune and get results but you have to treat it like a small ML training run, with real evals, and more intentionality than just “more examples”.
You can find pretrained and -instruct models on huggingface that clearly demonstrate what specialization/staged training runs do.
I’d be very wary of conflating finetuning with specialization/extending a model’s capabilities in general.
Turns out the world is made of simple, specialist processes, not generalists trying to achieve them. Adaptability may be of great benefit in evolutionary terms or for a walking anthropoid, but the majority of biology, chemistry, and mathematics rely upon specialist process for good reason. See also the old trope about robotics: that's what you call it before it works, otherwise it'd be a dishwasher.
The upshot is: use a generalist to create a simple solution once, and scale that. Don't deploy the generalist at scale, that's a waste of resources and an inefficient solution.
Instead we ended up with no finetuning. We give audio snippet to 2 AsR models, take 3 best transcriptions and ask the LLm to pick the best based on the context. That produced significantly higher accuracy in how an agent understands the users.
I wonder what’s tre latest in this field? Did we get a grip on this problem?
Quite ironic given the topic. It seems that the author’s model indeed contained too much knowledge about old Gemini releases, and did not do enough tool calling.
That's a sales tactic -- not a logical position.
>Why? If a model's weights claim that Bart Simpson became President in 2020, why does this fact suddenly become uncheckable?
Because in one case you have a source you can use to validate the fact, and in the other you don't. Though, as you explain earlier in your comment, the premise is misguided/hallucinated.
In terms of the value proposition of AI replacing knowledge workers, all value is in coding agents (coding agents as general agents).
(Apparently this is because the K80 is two separate GPUs on one card, but I still think it counts if you only have one slot to put it in)
Putting readers through this exercise disrespects their time. Even if as a writer you did the work of researching, reasoning, and fact-checking, you shoot yourself in the foot by running it through an LLM because there's no way for the reader to know which thoughts/research are from you. It demolishes the Ethos of the writing; readers feel they must do quality assurance on the reasoning, research, and facts.
>The internet is full of wrong information and I cannot magically edit it to make it all correct, so this doesn't help me.
My favorite RAG experience was asking Bart (or whatever they were calling Gemini back then) an answer to a question I knew.
It gave me the opposite of the truth (as was common with LLMs at the time).
But weirdly, it had cited sources for this "fact."
I checked the sources. Two of them, both AI SEO slop.
In this moment, andai was enlightened...
> You can make any modern LLM explain its reasoning
You can make any modern LLM create a plausible, self-consistent explanation that looks like reasoning, but it's not "the reasoning it used to arrive at that answer".
You just defined a liberal arts education.
https://linux.die.net/man/1/ls
I think I disagree. For some things, maybe that works - but think of a multi-agent system where one agent understands the code, and passes it off to the reasoning agent to figure out what the bug is. This system is going to suck. Because encoding enough info to figure out what the bug is would just be dumping every single line of the code.
So say agent 1 (reasoning) asks agent 2 (swift) to explain what is happening in File.swift. Anything agent 2 passes to agent 1 short of the entire code is a lossy transfer - and then the bug gets missed.
Yes, all three together would be even better. But it wouldn’t be if you had 100x more fanfiction, mostly synthetic, generated during RL to teach a model to be better at writing fan fiction. There are real limits to the amount of knowledge you can cram into fixed-size (downstream of hardware availability) weights. For a period scaling with data was basically “free” because we had the Internet and all the books/media that humans had already created; the data was accessible and limited (at least, the parts we think models should know about) enough and top-hardware big enough that we could basically compress the whole thing.
Post-training/RL are making this obsolete because they’re more about skill/capability acquisition rather than knowledge. They can generate much more data (most of it quotidian/useless, ie an agent made a typo in batch 382829) and clearly seem to cause a kind of mode collapse even in the most advanced frontier models.
We don’t need to make LLMs forget about SpongeBob SquarePants so they learn more about bash. But if I have a question about SpongeBob SquarePants, I don’t need to hear about load bearing seams prefaced with honest caveats after a model writes 400 lines of bash to look up SpongeBob’s family.
And there is probably a lot more SpongeBob knowledge we could put into models if we wanted to: interviews with the creative staff, a SpongeEnv/SpongeHarness modeling how the art/story team work together to create entertaining kids tv, a SpongeBench measuring entertainment value, etc. If a SpongeAgent spends 2000 years in Agent University learning how to Spongemaxx we probably don’t need or want to have it spend another 2000 years writing smoke tests
- Understanding of protocols like HTTP.
- HTML, JS, CSS, SVG, and everything "web".
- Understanding of databases, SQL, etc.
- Abstract code architecture patterns.
- Understanding the users' requests in English.
- Responding in English.
- Command line tool usage (agents/harnesses)
- Industry-specific knowledge that can be applied.
- Frameworks, SDKs, applicable libraries.
- Relevant legal requirements.
- Etc...
ie what everyone asking for this fails to immediately realize.
For a lot of use cases, you don't need a general purpose search engine – a search over a curated knowledge base works even better.
There are plenty of freely available data sets you can use, depending on the application; plus in many cases you will want to use internal-only knowledge bases containing non-public information (e.g. documentation for a corporation's internal systems and procedures). There are also many paid subscription domain-specific knowledge services available.
Providing not just any a baseline, but a correct and useful one, is ever more important the less the model is grounded in world knowledge – misunderstandings probably compound faster if there is no general grasp of (broadly) “life on earth”, or computers, or whatever.
And secondly, I think (consumer-oriented) search becoming worse and worse is a challenge that’s mostly solvable (but far from solved!) for the big labs: (Mostly) trusted or even editorialized/reviewed sources like published work, Wikipedia, etc. is something they could index internally, it doesn’t need to come from a random blog site on the public internet. Furthermore, there’s a whole slew of companies specializing in crawling-for-LLM (i.e., bypassing bot protections) now as well.
That's the right question to ask. For a while, it seemed that hallucinations went down as models got bigger. That may only have been because, with a big enough model, the desired data might be in the model, somewhere, which would keep the model from making up something. That's the brute-force approach to the problem.
This new article indicates that trimming down the model by pulling out seldom used info makes the problem worse again.
If LLMs had reliable "I don't know", and access to search engines, much smaller models might work.
After that Space colonization will come.
What makes you so sure your own brain doesn't work the same way?
We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, without even realizing we're doing it -- rationalization.
Like asking a human "how did you catch that fast ball coming at you?"
Reasoning scores keep climbing while per-token compute keeps dropping. GLM-5.2 scores 99.2% on AIME 2026 with about 40 billion parameters active per token. Qwen3.5 scores 91.3% with 17 billion active. DeepSeek V4-Flash runs 13 billion active. For scale, GPT-4 was rumored to run around 280 billion active parameters in 2023, and it could barely solve an AIME problem. At the small end, Qwen3.5 9B fits in 6GB of VRAM quantized and roughly doubles the score of the next best model under 10B parameters on Artificial Analysis's intelligence index. If you only looked at math and code benchmarks, you'd conclude that models are getting smarter per parameter at an absurd rate.
They are, on those benchmarks. Ask the same models a plain factual question and the picture flips. On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. The small models barely register. Artificial Analysis measures Qwen3.5 4B and 9B at hallucination rates of 80 to 82% on its knowledge benchmark, which means that when they don't know a fact, which is most of the time, they make one up. Ask the 9B for the birth year of a minor 19th-century mathematician and you get a confident, plausible, wrong answer. The parameter count didn't drop for free. Labs are trading world knowledge for reasoning skill, and the trade is deliberate.
Facts take space. Research on knowledge capacity (the "Physics of Language Models" series has the cleanest measurements) puts it on the order of two bits of factual knowledge per parameter. If you want a model that knows the birth year of every minor Wikipedia figure, the population of every Dutch municipality, and the argument order of every function in every npm package, you pay for that in weights, and it's a big part of why frontier models grew to trillions of parameters.
Reasoning compresses much better than facts do, because it's a relatively small set of procedures applied over and over: break the problem into parts, track intermediate state, check your own work, backtrack when a step fails. Distillation and reinforcement learning on verifiable tasks turn out to transfer those procedures into small models remarkably well. Phi-4 is 14 billion parameters, trained heavily on synthetic textbook-style data, and it's good at math and bad at trivia, which tells you exactly what its training data contained. That mix used to look like a limitation of the synthetic-data approach. It now looks like the design goal.
The knowledge that survives the trade has a shape. These models are generalists: they know a little about nearly everything and almost nothing in depth. Ask one about PostgreSQL and it knows what it is, what it's good at, and roughly how MVCC works, but ask which version added a specific planner feature and you're back to invented facts. That's the right layer to keep in weights, because breadth is what lets a model understand what a question is about, know what to look up, and judge whether a source is plausible. The depth is cheap to retrieve and expensive to store, so it's the part that goes.
A frontier training run takes months and costs hundreds of millions of dollars, and the moment it finishes, the facts inside it start going stale. Library APIs change, prices change, people change jobs, and half of what a 2024 model believed about the JavaScript ecosystem was outdated before the model shipped. Every fact you bake into weights has a shelf life, and the only way to refresh it is another training run.
The procedures don't rot. Algebra worked the same way in 1970 as it does now, and so does breaking a problem down or spotting a contradiction between two sources. A model that's mostly procedure and only lightly loaded with facts doesn't age the way a knowledge-heavy model does. Its training cutoff matters much less, because the current state of the world was never supposed to live in the weights in the first place. I think this is the best argument for the whole approach: it decouples the expensive, slow artifact (the trained model) from the thing that changes daily (what's true).
If the model doesn't know things, something else has to, and that something is the harness: retrieval over a knowledge base, tool calls, web search, a filesystem full of docs. I wrote earlier that Rust is a harness for agents, a source of cheap machine-checkable feedback. This is the same shape from the other side. The model contributes reasoning, and everything it reasons about gets supplied at runtime.
You can already watch agents work this way. A coding agent doesn't need to have memorized your dependency's API surface, because it greps node_modules or reads the docs before calling anything, and its answer is grounded in the version you actually have installed rather than whichever version dominated the training data. The recall that used to be a fixed cost in every forward pass became an on-demand lookup.
Follow the trend a couple of years out and I think we get a model with frontier-quality reasoning, Fable-quality, that runs on a single consumer GPU. The compute half is nearly there. DeepSeek V4-Flash reasons with about 13 billion active parameters per token, well within consumer-GPU range. What doesn't fit is the other 271 billion parameters sitting in its experts, and expert layers are mostly fact storage. That's the part this whole trade makes optional. Strip the knowledge out and total size shrinks toward active size, and a 20 to 40B model at 4-bit quantization fits on the 24GB card that's been sitting in gaming PCs since 2022.
The catch is that it won't know much. Ask it a bare factual question with no tools attached and the right behavior is to say it doesn't know and go look it up. Paired with a decent harness, that's most of what I use a frontier model for today, running locally with no per-token bill and no data leaving the machine.
The part I find most promising is what this does to hallucination. When a fact lives in weights, a wrong fact is unfindable and unfixable. You can't grep the weights, you can't diff them against last month, and correcting one error means a fine-tune that might break who knows what else. The model states the wrong fact with the same fluent confidence as a right one, and there's no artifact to check it against.
When the fact lives outside the model, a wrong answer has an address. The model cites a document, so you can open the document. If the document is wrong, you edit the document, and every future query gets the correction, which beats waiting for the next training run by roughly a year. Retrieval doesn't get you to zero, since a model can still misread a source or stitch two of them together wrong, but a claim with a source is checkable and a claim from weights isn't. A wrong fact in a knowledge base is an ordinary data bug, the kind we already know how to trace, fix, and write a regression test for.
There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks. The model just gets handed the world's current state at runtime, the same way a CPU gets handed a program.
It would be trivial to have a pre-flight convo with an llm to guide the user thru module choices. "Build a site" -> "ok, describe the purpose" -> "local council in XYZ location" -> "that implies you won't need localization since XYZ has a monolingual government" -> "english and catalan localization please".
Right now, you prompt and it builds using assumptions, and we prompt to adjust. I think it would be great to be able to pre-load a set of assumptions.
And of course the neural network series.
Even basic clients are now harnesses. A lot of chat interfaces are using memory systems, web search and other stuff under the hood.
Not as agentic as openclaw, but not a straight closed conversation either.
When you ask people who are rationalising poor behaviour about the scenario, but it is someone else doing it, they may arrive at a better answer. Can we use multiple LLMs to achieve self criticism and critical thinking?
Hearing has volume, direction, pitch, it's spacial processing etc
Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again.
The bitter lesson is this: You can simply ask a frontier model to do the things you suggested, in a few terse lines of English. Dump a few lines in AGENTS.md and you are good to go.
Your approach is to "fiddle with inadequate tools" for weeks or months until you can finally attain a pale imitation of what the frontier models can do effortlessly.
It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over.
I get it. You don't feel ownership over someone else's AI. You don't feel involved, you don't feel like you have agency.
It's like LEGO or IKEA furniture: study after study has shown that people enjoy things more if they "put it together themselves", even if fundamentally the thing is worse and/or still essentially nothing more than plastic made in a factory.
To your point, I agree that nominally there should be a way to give conceptual names to paths of weights, and when answering a question, notice which weights were and were not applied and retrospect on that.
That's not what reasoning traces as they currently exist are, though.
That paragraph sets me off. I’ll take Vim and Emacs over VSCode and Eclipse any day.
You don't _have_ ownership of someone else's ai, and that comes with real risks.
Security risks, privacy risks, business risk.
They might rug pull you, they might charge you more, or like atrophic, silently corrupt the answers, or code...
The labs are happy to jump on any emergent capability the scaling and training impart: generate prose, teach you things, cyber security, design, code, etc.
Do you really think that the frontier labs won't turn a popular capability, or trend they notice, into a first party tool if the ROI seems there? If it's your own private ai in your datacenter, you can keep it all secret, and not lose your business.
On the bitter lesson you're right of course:), wish I had a super computer to just scale that instead.
A) You can always self-host something like Kimi, DeepSeek, or GLM.
B) Just because you use a specific proprietary AI for programming doesn't actually bind you to that provider in any meaningful way. The authored code remains even if you stop paying them!
Of course, if you use AI as an active component in some sort of service, then the EULA, rug-pulls, etc... suddenly start to matter. That's a different story.
I can change which one I hear on a whim, so much that I can even get Brain Needle and Green Storm out of this video.