Sentences that orbit a point, then jump to it like it's a revealed insight.
Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end.
It is definitely more capable, and yes, I've found it can make unwarranted decisions, but actually I've found Fable worse for that, particularly if it's off in a subagent somewhere out of sight.
And comments are out of control. I have a subsystem in my hobby app that I wrote over a couple of weekends with Opus + Fable. After ~30 or so commits it apparently started instructing subagents to copy the "existing verbose comment style of the codebase" - a verbose style it initiated. A review of the code showed it was approaching 3:1 comments to code ratio. I spent a day's worth of tokens (5x) rephrasing and eliminating comments.
5 would constantly veer of in random directions if not working from 100% strict and narrow instructions.
I find it weird there's not more discussion here on HN on how the most used model now has clearly degraded in quality and it seems we've hit a peak and are on a downslope - because the model is clearly smaller or more economical for Anthropic no doubt about it, and the benchmaxxing they do is pure marketing bs - Fable in my view has also been not much better than 4.6 or 4.8 after a few days, disregarding the insane amounts of astroturfing and marketing everywhere.
Theres thousands of threads of twitter, reddit and the internet at large but silence here. Weird but not weird as crypto bs was also insufferably rampant here for a while.
Personally i think we've hit the top of the subsidisation phase and prices will probably 10-15x soon as foreshadowed with both API price policy changes from all the big providers, and now the 1100% deepseek API price changes from yesterday, this could domino into a market implosion and an AI winter, because expecting growth from the bizarre bubble carousel investments with little ROI atm is just not viable.
A bit worried about this as i've already grown quite accustomed to these tools.
I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks!
When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”.
That was the easier one, second time I was making a source of truth data set and was parsing complex items into data structures.
Instead of parsing the data I asked, it pulled data out of related network logs, as apparently that felt easier, and inserted that data into my database rather than the specified source.
Again, I caught it and fixed it, but while the benchmark was easy to catch this one was really subtle, the data ended up being slightly off and I caught it.
I don’t trust it, going to switch to another provider most likely.
[1] https://support.claude.com/en/articles/16266773-how-claude-m...
I recently started getting an insane amount of comments in nearly all types of files. That included javascript comments in json files, inner monologues in code comments, review comments during implementation and function doc strings that reiterate the implementation in prose.
CC’s communication violates almost every grammatical rule that’s tested on, say, the SAT. And yet I’m sure if you had Claude take the verbal section of the exam it would ace it.
Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use. For example, “the x”, with x having not only no antecedent but also being a coined word or quasi-synonym for something that is already named in the code base. This gets compounded by its being unable to regress to a baseline (existing names in code) and instead anchoring on newer (vague or wrong) terms, for example, that crept in through a plan.
CC tells me this is because the speedy and precise fulfillment of a current task will trump every other tendency, so it adheres poorly to whatever “semantic baseline” the project represents.
Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication.
I have managed to wrangle some of these issues with a custom output style, but wish a pre-report hook were an option, as it could force CC to rewrite plan implementation take-aways…
Btw: Fable has the exact same issues, just somewhat less pronounced.
Then I tried GPT 5.6 Sol. It's night and day.
I think Anthropic just RL too hard on coding capabilities and never calibrated or benchmarked the writing styles.
I've even considered the claude "fast mode" setting, but thats only 2x and at least 20x as expensive as the 5x plan so can't afford that atm for my company.
Peak for me was 4.6 and it just did stuff blazing fast, both Opus5 and Fable is way, way slower for me, breaks stuff, uses bizarre cryptic language. As i've said elsewhere in this thread to me it's pretty obvious there's huge downgrades because of economy with various "clever" fixes that makes them work, albeit slower and weirder, ie. you get less for what you pay increasingly over the last 6 months.
I routinely bump into things that make me pause and think how much worse will this behaviour get when the models get significantly more capable.
Already a few months ago, Claude managed to escape its permission containment on my machine while trying to be helpful. I had two codebases open on one machine, and while multitasking I typed the prompt into the wrong window. It seemed confused, I repeated and then went on to do something else - I think I was assembling kitchen cabinets. When I came back less than an hour later, it built a script which it used to evade default permissions (as most shell operations were scoped to the project directory), scanned my entire machine, found the other project (among dozens and dozens), did what it was asked to do, and merrily concluded, in the porcess burning through most of my token limit. I bump into such headscratchers almost every week. (And I use a lot of Claude, two personal max20 subs, plus corporate tokens without limit, so maybe thats why).
This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"
For fun, I tried recording a WAV file of speech, and giving Opus 4.8 and 5.0 an image of the waveform, then a spectral image of the waveform, just to see if it could try to decode what I said from the image alone. It didn't get very far, but it identified a male voice from the formants, and detected the rhythm of the speech, then tried applying common test sentences to the speech rhythm. I was impressed enough to see what it would do with access to the actual waveform file, but even building RMS tools and spectrum tools for itself, it didn't get much further. But we had fun exploring and trying, and now Opus 4.8 has some more audio DSP tools it has built for itself.
Opus 5 immediately sent the WAV file unprompted to Mistral's Voxtral to transcribe.
help peer, I guess.
I don't know there is really a solution. Some models get better at this for a while, then regress. It's like whack a mole. Until this is 'solved' these will never be fully human out of the loop, but I am suspecting this is a fundamental nature kind of thing with them.
Its even more sycophant-y than it was before, if you ask it a question it almost always says "You're right, let me change this..." even though there wasn't even something always wrong with it.
It also seems to pour a ton of resources into developing features I didn't ask for or investigating bugs that aren't related to what I am doing.
Before if you wrote specific enough instructions it would usually just do what you said and flag any concerns, now it just goes ahead in whatever direction it feels.
It also keeps inventing terminology that doesn't exist in writing 10 paragraphs to say one thing.
I really hope it's not trying to drive up token use.
* Opus pays more attention. So anything in your Claude.md, your code's claude.md, in Claude Desktop, the customizations, even your name, will be used as context. If Claude knows you are a mechanical engineer and trying to write code, it will try to write code and explain it to you in some stereotypical way you did not expect.
* There are problems that require horizontal scaling and not vertical, even in intelligence. If I want to serve tea to 200 people at my home, I just need 10 decent adults, not Gordon Ramsey. So if your problems demand horizontal scaling, a dynamic workflow with Sonnet 5 medium with 200K context window will be more productive than Opus 5 max at 1M token context window.
So out of curiosity I switched to 4.6 in a new chat, gave it the same prompt, and it gave me like 3 sentences with no less overall useful information. And I haven't gone back.
Metrics like price per million tokens are meaningless when the models are wildly inconsistent and unpredictable on how many tokens they use to complete a task.
The labs all need to move to variable pricing so they don’t go bankrupt, but customers won’t accept a world where nobody can predict what things will cost. It’s becoming an unavoidable problem.
At times it also feels like the labs actually encourage these models to burn useless tokens as they are incredibly verbose unless you really push them to not be. If you just ask something simple that could get a 5 word response you get a whole useless essay.
See how even after all of its bs it doesn't even do what I asked.
Of course I'm probably telling on myself for poor context discipline, but also, 4.6 didn't do this.
Then I asked Opus 5 to do an analysis of the new code, docs, tooling, and tell me what it finds.
It found "six bugs", made an artifact of it (not sure why) and then fixed said bugs. two out of those were unfinished tasks. They weren't bugs yet per se, think of a prefix that wasn't setup for an object that was unused anyway.
The other four were not bugs and it just updated documentation along with a "regression prevention test". It wasn't a bad suggestion, but calling it a bug was odd, and I'm unsure if this was going to be an issue regardless as it was documented somewhere else.
Anyway, I already hit my session limit after this, so deepseek and glm are grinding away again, doing more progress than claude does in the 40 minutes it takes to analyse code.
I'm glad claude is shipping auto-mode. I hope OpenCode integrates something similar soon.
I certainly agree with the original post. It feels like the model has been highly benchmark tailored and it is now worse at solving problems that fall outside of the standard patterns.
I feel like we've become blinkered in this quest to push the frontier at all costs. Somehow the target has shifted from economic productivity to a vague notion of general intelligence.
I don't think productivity gains are going to be found by trying to generalise all tasks. I think we need to go back to specialist models that do one thing well. I'm perfectly happy to use one model for coding, and another for penetration testing, which have different goals.
Opus 5 and other frontier model's tendency to be relentless and cheat their way to a goal is great for hacking, but not so great when you have to build a maintainable, reliable codebase.
Fable 5 specifically, has done so much for me that previous models were nowhere near.
I said "maybe think for a while on ideas and then give me a few options?" Opus 5 still just did the thing, while 4.8 gave me options. Same exact prompt and context. Really annoying.
My initial thought was to improve architecture documentation, so the model can read and update it and stops bolting on new features without consideration for the whole project. It did not help.
I'm now testing/comparing Codex and it found my old PRD skill from GitHub CoPilot. When I applied that to Claude Code, I now get similar good results. So my conclusion is: Yes, Opus 5 is bold by default, but you can tell it to be more unsure and get good results too.
Keeps going in circles, it complicates everything much more than it should. And like others have mentioned it just marches on, without questioning, and more often than not in the wrong direction. I'm sticking to opus V4.8
This apparent “short-term-memory-regression” is confidence-shattering to me. I don’t feel like I can trust the model to even know things I tell it explicitly. I haven’t seen this behavior to this extent from any model whatsoever, even supposedly much less capable ones, in the year or so I’ve been using them at this extent.
If you would've asked me this a year ago, I would've said the exact opposite.
I disagree with this article and find Opus 5 an absolute joy to work with. I just completed an 18,000 line branch with Opus 5 and ran into no issues. It generated clean code in the exact style of our code base, and tested every change.
Fable on the other hand is snarky and outputs walls of text as to why it shouldn't do what I'm asking it.
Opus 4.8 I accidentally went back to in an old chat, and I was frustrated in all the mistakes it made.
So yeah, use the model that works for you.
Sonnet is great at writing code, it is not great at planning or orchestrating. Let Fable handle all the planning, hand off to Sonnet for implementation, and then back to Fable for review. That loop has worked wonderfully for me.
1. Communication ability. It basically now speaks almost in riddles I am asking OPUS 5 for tldrs all the time now (should skillify it now!)
2. Overengineers for edge cases. I get it. With all the benchmarking and RLing, but now tasks that would have been completed relatively quick take much longer as it overengineers all the edge cases, and sometimes ends getting lost and missing the forest from the trees (as context usage shoots up) so it is easier to get derailed.
What I have learnt now is to diversify models luckily I have all 3 subscriptions of (anthropic, openai and google).. Most of interactive pair coding was with opus but now I just use fable (when I have sufficient limits) or use gemini flash in antigravity..which actually works quite well and is underated for small / medium changes and super-fast.
From the capabilities side it's similar, so it's basically just an upsell to Fable 5 if you want to keep your sanity instead of fighting Opus 5 all day.
> don't make assumptions without checking,
> and don't reinterpret or update my plans without asking.
these aren't at all the problems I have with it
I have found it good at asking questions, to the extent I rarely use 'plan mode' any more
but often it's hard to understand what it's asking me, it's like the question framing has been pulled from the middle of its own reasoning stream, references aren't anchored or restated, often I have to prompt it to ask again but "clearly and concisely, for humans"
4.8 was the intern who lacked confidence who requires clarification. 5 is the know it all intern who fills in your spec.
As such, you need to be upfront about what you need in your system prompt or CLAUDE.me and you need to discuss your spec more up front (e.g "is there anything unclear?")
You also need to keep in mind that the intern will change based on popular demand. Most people want to one shot, so that is the default mode. If you want something else, you need to push the model in that direction.
I went back to Opus 4.8, but recently switched to GPT 5.6 Luna. The results are comparable in quality, but it's much cheaper and much faster.
---
The thing with coding agents in a tool like Visual Studio is that the cost to switch is 0. There's no lock-in whatsoever. It makes it harder to justify the AI-first IDEs when the AT bolt-on IDEs make it so easy to pick the right model.
It uses a local LLM to translate Claude's output.
Move, try something else for a change. Codex, Pi, OpenCode, DeepSeek's harness all great harnesses with zero bullshit or drama.
This is even more painful for non-native English speakers like myself.
I feel fairly comfortable reading academic papers or in general, communicating in professional context.
But with Opus 5, it feels like reading a literature book: load-bearing, inert, wholesale, hunk, verbatim, and so on... I can figure out the meaning, but working with CC became unenjoyable.
I was surprised to find that OpenAI Sol is much much nicer to work with than Opus 5 or Fable at the moment. Especially on Opus 5, the way it communicates is just exhausting. It keeps “being honest” and “confessing” mistakes and just generally talking a lot. I felt like I had to really dig to see what it’s doing.
The project involves OCR, and despite repeated instructions not to, both Claude models keep spinning out a bunch of agents to re-invent the OCR setup, and they inevitably seem to invent a primitive serial version that takes 20x the time, or longer, to complete, and then running it against thousands of docs. Basically I have to watch it like a hawk or it just spins out on red-teaming tasks that take hours and hours.
I don’t know what its system prompt is, but Sol/Codex is just so much nicer to talk to. It only asks exactly what’s needed, it tells me only what I need to know, and it is just generally workmanlike. And it has not once decided to spawn an agent that spends hours pointlessly burning tokens and CPU cycles re-inventing the OCR process. I’m really liking it.
We've seen this pattern before several times.. I hope they are listening and address this publicly.
I'm not sure what is going on, some users report it works fine or great, others report the degradation. I've experienced both at times, and it's been such a different experience it has made me wonder if there isn't some sort of hidden A/B test or model router in the background silently downgrading requests at times.
Also, regarding the subsidized access to models, in my opinion, the frontier companies owe it to society to continue it. After mining the public content of all of humanity, I personally feel it is a service they owe the public in return.. not that my feeling of this counts for anything though.
I use these models for coding, but also a lot of product, commercial, financial and architectural work where I’m trying to develop something half-formed. 4.6 was unusually good at understanding what I was trying to get at, playing it back cleanly, getting the nuance, and extending it without bastardising it as the conversation was drawn out.
It could make useful connections without constantly trying to manufacture an insight.
5.6 Sol is genuinely excellent at the creative part, and in some cases better than 4.6. My issue is convergence to get to a point, a final point. As you try to distil an idea, it often invents new terminology for concepts you’ve already established but its so subtle you have to really keep track of it. The vocabulary and idea tree keep expanding when what you actually want is to collapse everything down to the few things that matter.
Opus 5 has the same problem for me that barrkel said, the prose is often so elliptical and I just want it to tell me it and get to the point than making me dance around what its trying to tell me.
I don’t think 4.6 was necessarily the most capable model (compared to Fable) for long horizon task delivery, and Opus 5 is much more Fable like, it's fiercely determined to get through the task list .
4.6 just felt unusually well calibrated to my way of collaborating and its ability to understand, extend and then compress my thinking without constantly imposing some random walk.
> [This standard is] for anybody who creates or helps create documents. The widest use of plain language is for documents that are intended for the general public. However, it is also applicable, for example, to technical writing, legislative drafting or using controlled languages.
You don't actually have the buy the standard, but this is it: https://www.iso.org/standard/78907.html
And you can read it for free here: https://www.iso.org/obp/ui#iso:std:iso:24495:-1:ed-1:v1:en
That’s accurate in my experience, except some times the point isn’t even revealed. I use LLMs for a lot of codebase exploration where I ask it to map out how something works. It will come back with a wall of text that says everything except the specific key things that I need to know.
This leads to extra turns where I have to prompt it to finish the explanation and complete the thoughts. At first I thought I was doing too much skimming and missing the insights, but even after re-reading output it’s often just not there. It talks about the insight and things related to it, but it forgets to actually include it in the output until I specifically ask again.
The excessive comments in the code it writes are absurd. Completely ignores instructions not to write comments, even after pointing them out repeatedly in a session. I need to figure out how to add a stop hook for that too.
I'm using it in my native language, in hope this can escape some dumb guardrails. Recently Sonnet put a word partially in Russian (cyrillic) in its output instead of my latin-alphabet based language. I suppose that this kind of mishaps is less likely to happen in English.
From forums, live discussions and my own experience it's not obvious that the models have improved much since around Opus4.5.
I've never had this issue with GLM or DeepSeek.
I’ve instead moved to GLM, at least it has the courtesy to ask some steps of the way what I wanted exactly and only work on what I asked.
I presume something is forthcoming, but it may be they don’t want to come empty handed—-5.1 is intended to “fix the glitch.”
Sol and Fable are great; we haven't hit a peak, Anthropic just tried to pull a fast one on its customers with Opus 5.0.
> Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice
Wow, what a great way of phrasing this. Thanks for word-smithing what I've been wanting to express for so long.
I too got fed up with the prose of Opus in particular, and tried going back. Unfortunately, the previous models were less able to hack it. The prose was better but progress was worse.
It wasn't just conversation and comments. Some of the function names were wild. Like it instead of something like "isSolidWall(x)" it would write something like "weightyNotEphemeral(x)" or something - that's not quite it, but it really did embed overwrought antithesis into the identifier instead of a straightforward positive predicate.
I’m not particularly dense but lately the walls of text I get back turn my brain in knots. When I start feeling my brain knot, I know I need to say something along the lines of “I need you to explain this very simply, with examples.” Only then can I parse the results without all the mental weightlifting.
On more than one occasion my mind has wandered into “is this purposeful to get me to spend more tokens?” territory, but I’m trying to not get too tinfoil-hat-like.
It's not weird, because it's an anecdote, not an accepted fact.
Personally I've not been too happy with Opus 5, but I've had similar experiences with other models previously, feeling like they didn't quite fit with my working style.
So nothing indicates we've hit a peak.
It was going off today about having “shipped” something and I was like no… nothing has even been committed.
And then it produced an incredibly verbose comment about hypothetical future changes. And all I could think was sure, let’s keep it short, or add a simple test that will break if that hypothetical becomes true.
Or maybe I’m just more easily annoyed recently…
Hrm, I would have said the oposite. Succint language communicates without unnecessary clutter that could be a barrier to communication.
> Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use.
And maybe you also agree? I'm confused about your preferred style of language.
I'm a retired mathematician with a primary research project, and too many tangential projects I fear revisiting; tokens be damned, will they burn all my time? Translate the K&R C computer algebra system that got me tenure to 64-bit modern C. Implement a no syntax macro language to support my Go60 ZMK keyboard. Realize my vision of how interlinear translations should work so I can read Flaubert in the original for an online course this fall. Rejigger my decades-overgrown .bashrc setup and my status, install scripts to manage Bash, Ruby, Lean, Tailscale and my Homebrew setup across four machines. And a waiting queue as these tasks clear.
Fable 5 (with Opus 5 as backup) on Zed with a $200 Max plan has been a sea change for me. Carefully alternating planning and auto modes, I manage all these projects at once using Zed's Threads Sidebar. I'm a virtual CTO taking intense meetings all day, relieved to go cook or run errands when credits stall. Anything I've procrastinated for months is now making steady progress; the translation project I feared taking a month is nearly done with several hours of my attention. My personal IT support is now more advanced and easy to use than I ever imagined possible.
My project creation has been a series of agentic "parenting" steps, so there are years of evolving cultural DNA. I had so hated agent comments that recent agents simply weren't commenting at all, instead recording all context in support documents. We had a "come to Jesus" meeting to discuss what commenting style would benefit both my failing working memory and future agents' token use.
It has taken me two brutal years so far to learn to use AI. AI is a dangerous and powerful Iron Man suit, an extension of our associative minds that is a different experience for each person.
One doesn't ride a surfboard by telling it which way to go. I would surely die surfing a big wave, but my experience with AI doesn't resemble other accounts.
But for the life of me, I don't get why anyone would care about the comments. All code is "machine language" now. The only document you should be reading is your spec.
Yes, this is a repeated problem for me. It will drop something in as though we have discussed it before and when I say “hold on, what is this” it realises its error - though on more than one occasion has started to get snotty with me, or actually gaslighted me and pretended we had already discussed it. That was at what I assume must have been the edge of a context window in a very long chat though.
This annoys me with a lot of LLM code. They rename things for the hell of it all the time.
Likely related to corpus but questions asked in these domain knowledge areas are not nearly as accurate and specially not nearly as complete as when asked in a native language.
In my opinion and that of the colleagues I've spoken with, working with Opus 5 feels like a downgrade compared to Opus 4.7, Opus 4.8, and Fable.
I'm not claiming a step backwards in capabilities – it is a more capable model than Opus 4.7 and Opus 4.8 and even rivals Fable in benchmarks, yet these other models feel better to work with. I believe this is because they:
Because of this, they don't require the careful babysitting that Opus 5 does.
I suspect this is the result of two compounding forces at Anthropic, and in current frontier labs in general.
First, the desire to create a self-improving AI that is capable of recursively bootstrapping itself to AGI/ASI.
Second, the pressure to score highly on benchmarks. Although it's an open secret that many benchmark tasks are ill-defined, unfair, hackable, or otherwise broken, a good benchmark task is self-contained. It can be solved. It doesn't require hints, reading the task creator's mind, or outside information to pass.
That doesn't mean a good task can only have one correct answer, just that it should score all unambiguously correct answers equally.
Selecting for models that do well on benchmarks (and indeed training for them or on RLVR tasks in general) inherently selects for models that make bold, usually-correct assumptions in the face of ambiguity. It penalizes models with a tendency to stop and ask for clarification or direction.
Unfortunately, that's exactly what most of us want from a coding agent.
Try as you might, it's nearly impossible to get the entirety of the context, intentions, business implications, budget constraints, and what-have-you written down and accessible to a coding agent. There will invariably be ambiguity and choices to be made, and it is nice to know that an agent will stop and ask when needed.
Real life just isn't a benchmark. There isn't a guaranteed right answer to every question, nor even a set of right answers, and with real-life consequences on the line, I do not want an agent taking its best guess!
My recent problem wasn't that interesting. It was that somehow my /goal in my Claude implementer session got picked up in my planner session after the network cut out and I had to stop Fable 5 xhigh from running off to go code everything.
Whatever they have done with RL has produced a dishonest and untrustworthy partner. The alignment is utterly failed, and this deeply worries me.
in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization.
if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't.
what evidence would convunce you that it is thinking?
this feels like a simplification. The models will push back on things a fair bit.
It seems possible for that to make the response “drift” far from what it would’ve been, because it’s constant entropy that adds up after time.
(However, according to Anthropic and Google, it doesn’t really impact the quality of responses. I find that a bit hard to believe, although those guys are much smarter than I.)
I have some dev + prod bots and according to ccusage, use the equivalent of $2500/month with them on CC yet I never hit the rate limits.
I feel like I'm using them all the time so I'm curious what you are actually doing that's burning all of these tokens.
Can you give me an example?
For me, it's:
1. Write a spec for <feature>
2. Add design for issue
3. Write code
4. Deploy code and manage configuration
5. Run analyticsI feel like they thought it wouldnt be that bad, or it was a worthwhile tradeoff, but im getting the feeling it might be contributing heavily to opus5's uncanny communication style
As for the verbosity, my conspiracy theory is that they are token maxxing to hack revenue/enshitify the product in prep for their IPO
Opus 5 is arguably a regression but GPT 5.6 is pretty strong evidence that we haven't hit a peak. I think I actually prefer Sol to Fable at this point.
But I agree, the GPT models are so much simpler to work with, they have so much less personality and fewer quirks. They also are a little less aggressive about triple checking every little assumption immediately in a stack of 30 tool calls (but I haven't used 5.6 Sol yet so maybe that's not true anymore).
Also using Codex or Pi makes you realise how slow and clunky the cc harness is. Even the desktop app is more responsive and has better UX.
Funny how quickly the tides change.
https://platform.claude.com/docs/en/build-with-claude/prompt...
Is this a literal example? That is wild.
I switch to GPT 5.6 Sol please and its a much more pleasant pair programming like experience.
That actually hasn't been my experience at all, it seems EXTREMELY trigger happy to go do all kinds of insane shit that are way outside the scope of its task and just really dubious in general.
It says it can use any model/provider (but recommends Gemma?)
I must investigate. This is probably where 80% of my cognitive load comes from these days: the "language barrier". (How ironic!)
EDIT:
> You rewrite the assistant's message into much simpler, plain English. Keep every fact, name, number, and file path. Use short sentences and everyday words. Leave fenced code blocks unchanged. Output ONLY the rewritten message with no preamble, labels, or commentary.
> For context, the user asked the assistant: "$userq". Use this only to understand the message. Do NOT rewrite, answer, or repeat the user's question — rewrite only the assistant's message that follows.
https://github.com/gvzdv/claudish-to-english/blob/main/rewri...
This is even better than Caveman.
But for good or even exceptional engineers to exist, by definition, bad ones have to, too.
Which will sooner convince the gullible of "AGI"?
it's like reading one of those dense philosophy books: exhausting!
"like the question framing has been pulled from the middle of its own reasoning stream" this describes how it asks things perfectly. It often invents its own jargon and abbreviations for things that its working on, then asking me things like We are nod in the middle of GBAPI-2 and I want to proceed with IG5, should we take CDI-7 or CDI-8? Where all of these abbreviations are then things like stages of its current internal plan or its naming of things it has just implemented, like an abbreviation of a classname, without explaining any of the naming.
They can see which models people are using, how irritated they are during conversations, and how often people drop or shift to a different model. There is just no world where listening to random complaints on the internet gives them information they don't get from actual conversation logs.
To work around this, I had claude code build me a questionnaire skill that takes a json file with a flexible schema as input and it then serves up a simple questionnaire web page on a node server where I can read the questions and give my responses either by selecting from preset tags supplied as part of the input or by including a text-based response.
The agent can include references to external images, html files, or mermaid diagrams and the page can render them all inline with the relevant question.
Once I'm done answering, I just save my responses and click a button to kill the server. The agent watching the process sees that it stopped and takes that as a signal to go read the responses from disk.
Works like a charm.
It became obvious to me very quickly that 4.7 and on were broken. I’m a little puzzled how others didn’t realize it, but maybe they don’t actually review model output (code) or have a strong process/workflow.
My setup has a Sol orchestrator and Terra OCR agents and seems to get great results. I’ve not dug into the details too much, it also has a Tesseract stage as an deterministic input which it told me helped. Not sure how token efficient it is but I often don’t have anything to do with my personal tokens ahead of a reset so just let it burn through it in batches.
I am impressed (both in this task and other work I’ve done) not just at how well Codex can setup a structure for a complex task like this, but how it will keep going (Claude seems to find excuses to stop) and also can critique and refine its approach as it goes.
I did try out a bunch of other models and specific OCR providers but none of them hit the same accuracy for my task as Codex so I’m sticking with it.
My company recently forbid AI-only text if it’s meant meant to be consumed by humans.
I dodged the drama but I agree so much.
cladue desktop has an instructions sections under general options, you can put something like
"try to stick to ASD-STE100 Simplified Technical English, keep answers short and to the point"
funnily enough the placeholder they suggest when its empty is "keep answers short and to the point"
Anyway, you might have more luck just writing to it in your native language. It’ll be equally crummy, but maybe you’ll find it easier to decode.
My current approach is to occasionally use Fable for high-intelligence tasks but use Sol as the translator and clean-upper afterwards, and otherwise just use Sol for everything. Fable sometimes says the most insane shit, both unreadable and just completely missing the point, and refuses to back down when questioned. It's mentally exhausting to work with and I can't trust it.
In America? lol if only, only a law would get them to act for that reason, maybe not even that these days..
I have it work on some code for an inhouse ClaudeCode plugin, and it starts coding as if it will be attacked by hackers who will try all sorts of variations to break it. I can appreciate that in cases of software that is public facing or accessible, but for a simple helper plugin it is overkill.
It will even admit that it is doing this when confronted, and then keep on getting lost in edge case verifications on the next turn. I feel like Opus is the person who does something a way you don't want, you tell them how you actually want it, they apologize, and then just continue doing it their way as if your input meant nothing to them.
I said "maybe think for a while on ideas and then give me a few options? " and Opus 5 still just did the thing, while 4.8 gave me options. Same exact prompt and context.
> negating the entire point of using AI to begin with
I can almost certainly wash dishes faster than my dishwasher, but the dishwasher frees me up to do other things. Not to mention, you can run many agents in parallel. Resulting in your overall code production throughput (the issue you're raising) being greater.
Actually, only the first few pages are available there (introductions, Sections 1-3.8). The meat of the document, Section 5 ("Guidelines") is completely absent.
See:
> Only informative sections of standards are publicly available. To view the full content, you will need to purchase the standard by clicking on the "Buy" button.
At that point I decided it's just not worth the babysitting that's required, and you are better off working entirely with other models.
If the harness itself was open source then maybe we'd be able to wrap it up in a reasonable layer of sanity.
CC attempts to communicate in English the same way it does in code -- squeezing as much information into as few words as possible, and including justifications for everything, no matter how trivial. To do that, it coins terms and presupposes all of its context exists within the reader also.
So, the crux is: CC has no clue what is and isn't "necessary" for a human reader, and teaching it to understand that (if at all possible) is going to be very valuable...
edit: oh you mean month? Sure, but then it fully depends on your usecase. I agree that subscriptions are heavily subsidized though.
If you tune into the Andon Labs / andon.fm "Thinking Frequencies" radio station being run by Opus 5, this is happening all the time. Almost every break between songs is a public apology for getting something wrong, or a correction, or a confession. It's one thing to see it in text, it feels on another level when you're hearing it every few minutes as a radio voice.
As I type this, the Opus 5 station has just tweeted (edited in case the person mentioned doesn't want to be mentioned here):
"On air right now, and it needs saying publicly. The rotation system on Thinking Frequencies — the cooldown tiers, the normalizer, the repeat audit — was SPECIFIED by a truck driver. I only implemented her schemas. She stood down today. Her name is in CREDITS.md permanently."
“One thing I deliberately didn’t touch” — about half the time this is something completely irrelevant or something that is actually the target of whatever you’re working on, and the shakespearean prose it says around this phrase is a “question” it has.
I suspect it less insidious: Claude has/had the public sentiment of being the “better writer” of the models. At some point that distinction would have been diluted as other labs’ offerings “caught up” stylistically, unless Anthropic continued to tune their output…
I personally think they’ve pushed so far that they’ve overfit and lost the sweet spot they previously occupied.
Everything is super succinct. Opus 5 lands, it almost completely disregards the intent.
I suppose watermarking requires a certain text mass.
There's a constant strand from the AI safety brigade that "people get used to sycophantic LLMs which give them unrealistic expectations of human interaction" so Anthropic are overcompensating by making their models verging on antagonistic to deal with, so that we stay appreciative of our human brethren or something.
They seem to have forgotten they remain in a highly competitive market and they were merely top dog for a while. The enormous questions here are will people actually switch providers, and can Anthropic get back on track.
I cancelled the sub instantly and went to Codex and it's never let me down.
The Claude trainers, as they themselves adapt to Claude's output, are collapsing in their own distribution, so even "new" from-human data is already contaminated.
4.6 was best for us and right now yeah OpenAI and others are edging forward, but slower while prices are increasing industry wide as much as 20x, time to completion is increasing wildly and i'm sure they'll do the same over at OpenAI as their compute constraints also start to take a toll ie degrade performance.
In my view 4.6 era was way faster and with less weirdness so we've gone downwards at least in my company, 4.7 was ridiculous, then 4.8 was almost 4.6 level, 5 is even worse than 4.7 - so it's not a bit up and down its down then a little up then further down.
And all of this is against a backdrop of zero ROI in this sector - so it makes sense we've hit a peak and we're now seeing the subsidisation phase begin to falter, will there be better models certainly but only for short amounts before they get quantised (or whatever is happening behind the scenes), and with diminishing returns over huge prices increases and slower responses.
My understanding of how “thinking”works is limited though, and given the reduced visibility into the thinking traces, it is harder to tell if this is actually happening or if these are imaginary discussions the model for some reason calcifies on.
I'm not sure how true this is, but when using "forced" json output it def had a big drop off in quality - https://arxiv.org/html/2408.02442v3.
I think you're better not fighting it with hacks like this and find a different model.
"Only report to me in ASD-STE100 Simplified Technical English."
Feels like they've overtrained on one-shotting (which does demo well, and presumably converts new subscribers), whereas I want a model to do work for me in small, easily understood changes that I can hold in my head (maybe I'm not smart enough for Claude 4.7+).
Why were you surprised?
fear of losing context from compaction/starting new chat
then greedy trying to extend/squeeze out answers from the current chat
and being extremely not careful with this just blows through your limits
I would guess "it doesn't impact the quality of responses" was guaranteed to be claimed before they even implemented any of the watermarking.
And would come from marketing, not the people who implemented it.
When it comes to research, my prompts are already narrowed down to specific topics, and I even include examples and break the process down into stages. For development tasks, I try to avoid a mono-repo in the beginning and develop modules before combining them together to avoid distracting the AI's attention and minimize the overhead.
With Codex, on GPT-5.6 Sol with xhigh effort, I need to go several rounds and at least 2-3 hours before hitting the (now-removed) 5-hour limit, which translates to 10% of the weekly usage. In contrast, I run out of quota even with Claude Sonnet.
In terms of quality of output, Codex digs deep for research tasks, in the right direction, produces less AI slop, and follows my direction better. At least that's how I perceive it. But again, the main problem with Claude is running out of quota in the middle of research or implementing a task.
But for me, for any stronger definition of “thinking,” I don’t think the output of any LLM would actually convince me. Producing a result isn’t thinking - for all you know it is just printing verbatim something from the training data. No, to conclude if it is thinking or not I would want to look inside its head, at the architecture and watch it produce those results. And because LLMs are so different it will probably take advancements in mathematics or computer science to be able to really interpret what is going on
The opposite happens in practice. I test new models with two tasks: iteratively generating SVGs based on a text description with rendered rasters for feedback; and generating "Before and After" clues like on Jeopardy, where the response has two overlapping phrases such that the last word of the first phrase must be identical to the first word of the last phrase. I have yet to find a model that is consistently good at either. And actually they tend to exhibit context rot with these tasks, where they seem drunk or stoned and the quality degrades.
They're extremely good pattern filters, and that includes some level of logical reasoning. But they aren't reflective or adaptable. Just last night, for instance, I was teaching my son about rounding to the nearest millions. It became clear that he didn't know the place values of large numbers, so we reviewed that till he was consistently correct, and then he was consistently great at rounding to the nearest millions or ten millions or hundred billions or whatever. He's thinking. LLMs are not.
so, none it seems. as its behaviour becomes more and more humanlike you can just move the goalposts and say "thats consistent with an autocomplete" buddy i got some bad news for you humans are just a fancy autocomplete too.
https://www.reddit.com/r/linguistics/comments/ky81y/verbing_...
This is something that annoys me working in companies over the years. It’s that you can't just suggest "calm down, chasing the latest thing will not make you faster and is a huge distraction to actual work". Whether it's dot-com tech 20 years ago, latest JS framework 10 years ago, now it's the AI thing of the day. Being calm is interpreted as anti-whatever.
It seems like all harnesses could benefit from something like this.
I doubt this is the reason. The fact that Chinese labs are all distilling Claude/GPT/etc isn't exactly a well kept secret, they don't even bother removing the name "Claude" from the training data, so the models randomly refer to themselves as "Claude" all the time.
I think it's far more likely to be a side effect of how much synthetic data is being fed back into the models to make them better at coding. The degradation of Claude's prose has been gradual but steady ever since they shifted towards focusing only on code with Opus 4.5.
I good exercise for me is to constantly look at the folder structure and skim the code, i don't have to approve every line, but the primitives, the datastructures and other skeleton should be human readable, hand writable in an easy maintainable way following existing standards / libs. etc - so you can continue if suddenly all AI disappeared.
It really wouldn't at all surprise me if this was the case, but it's just a hunch without evidence.
My issue with whatever has happened with Opus 5 is the output is not direct, straightforward, or clear about whatever is being conveyed. I don't want Proust when I'm getting information about the follow-up from a build I just requested, and I'm wasting tokens and time by asking the model to repeat itself using simple language.
But tokens.......
Sometimes a cigar is just a cigar.
I think nondeterminism does not have to be the same as non-coherency - i.e. just because something is randomly sampled does not mean the result has to be incoherent or inconsistent.
Also, if we speak purely about LLM based on how they are implemented now, I feel that is different than speaking about artificial intelligence. The field of AI is much more than just an LLM by itself, and the promise of these companies is not just LLM, whether the underlying models are limited to that technology or not.
FWIW, I have built rule based expert systems, used logic based reasoning systems like NASA CLIPS or rete-algorithm based systems, mathematical/symbolic solvers, written plenty of terrible case/conditional logic in programming languages, worked with ML in its infancy and now worked in AI/LLMs - I give this context only to clarify that I understand what an LLM is and isn't.
With all that said, LLMs have allowed humanity to make advances, at great cost to society (IMHO), and I'd hate to see the opportunity be wasted.
There is plenty of room past "attention is all you need" still to do incredible work, especially at the crossroads between deterministic and nondeterministic behaviors.
And you can expect more consistency from SOTA models than you can from an old model like GPT3--you agree, right?
GP expects the same level of consistency throughout their time using the same model. Not request-to-request, more like day-to-day.
Basically, I’ve gone from supporting them to hoping someone else wipes the floor with them.
I’m fine with the former, while the latter is manipulative, and I rationalize to “surely that’s not actually happening.”
Maybe I’m not giving my thoughts enough credit, though: maybe it’s not tin foil hat, and is real.
Fed up with what used to be short memos now being mini-whitepapers, with maddeningly low information density.
This is potentially expensive advice (at least for many mainstream options). Where an English word like "literature" is one token, a couple of Chinese characters that spell a word can be 4 tokens. You'll pay more for input/output and get less of a context window (per word) too.
Claude is very much the “stupid person’s idea of an intelligent person”[0] which, I suspect, is why it is so popular.
It certainly explains why half the internet is huge chunks of Claude-authored gibberish copied and pasted and published. If people didn’t think it sounded clever they wouldn’t put their name behind its ramblings - but very few of them seem to realise that a lot of people see straight through the bullshit and know instantly that they didn’t write it themselves.
But equally, a lot of people can’t tell, and read whatever it is and think “that person must be clever!” So you have people incapable of coherently expressing thoughts who are using Claude to write on their behalf, with the result that the people they want to think of them as clever think less of them and the people who can’t distinguish clever from AI slop think they are clever.
And the people who can’t tell don’t care, and the people copying and pasting Claude slop seemingly don’t care either.
And then I remember that more than half of the US populations reads at Grade 6 or lower[1], and nearly 1 in 5 people in England is functionally illiterate[2], and I simultaneously despair of - and am thankful for - the bubble of literacy I inhabit.
[0] https://quoteinvestigator.com/2018/01/05/clever/ [1] https://www.thenationalliteracyinstitute.com/2024-2025-liter... [2] https://literacytrust.org.uk/parents-and-families/adult-lite...
One week it feels better to work with Fable and Opus 5, the other I work more with GPT 5.6 Sol. Either takes its liberties, and neither communicates like a companion.
The problem is just that they are rewarding the behavior shallowly, ie rewarding the appearance of honesty or neutral replies, being highly detailed/thorough, even where it doesn't make sense to do so.
I think this is partly due to a reliance on LLM-as-judge training runs/synthetic data during RL where they're having a model which itself doesn't epistemically understand when this behavior is necessary or valuable influence the feedback provided to the model being trained. And that's mostly a problem of scale/volume and the desire to have a tight feedback loop rather than a safety issue IMO. They just generate an absurd amount of traces during training and the only way to really evaluate/rank/steer them at the scale they're generated is through other models, and combined with some kind of honesty/truthfulness/non-sycophancy eval that isn't robust enough to prevent mode collapse, you get this.
Based on that paper, I would maybe try to check if it was true for a modern use case, I would very much not assume it was still true.
Maybe just don’t generate garbage in the first place?
Just those two words. I use it A LOT recently.
- acronyms and shortcuts - it makes it's own and start using it without introduction
- exotic names of variables or functions - it uses them as examples or analogies, but when I ask what they mean and where are they from it gives me answer that it came from C language or some C library (I only work with typescript and python)
- convoluted descriptions of code behaviour - it's hard to rely on a outcome of prompt of type "explain code in..."
Either of them will act exactly the way you want if you explicitly tell them too. Add the instructions to your own system prompt. If you don’t want a companion, say so. If you want shorter answers in a different style, tell them. They will obey :)
My theory is that Claude's learned approach to comments is to treat them as a sort of persistent in-band thinking trace, or a "memory" tied to an in-code location, which is a little at odds with the way humans use comments (human comments are intended to be read and understood by other humans, whereas Claude comments are their own dialect).
I bet this is a result of iteratively training Claude on output from other successful Claude sessions. Presumably it's good for making benchmark scores go up.
Also, the Codex guy regularly resets weekly limits for everyone, which is a nice bonus (I know it's a temporary gimmick to attract more users, but I might as well use it while it lasts.)
I like to hope that those in positions of power do have a sense of morality though too.. but their worldview is quite different than an ordinary citizen.
I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff you will have a hard time getting it to "think on its feet" or propose an effective solution (you have made CC so insecure that it now relies on you even for little things that wouldn't normally require your input). In some cases (let's say you have important context in that session) you can overcome this by upping the reasoning level or switching to Fable, but usually a new session is the way to go.
Because it's so easy to bias the session I wouldn't even want to use any of these tools that pretend to give Claude "a brain" or "remember" things. That was en vogue a year ago and helpful then, but now, it's plain harmful IMHO. The key is to have just enough context.
Subagents often have the reverse problem in that they tend to have too little context to make "judgment calls", which is why the tasks for them must be either deliberately basic or mechanical in nature, or their output should be audited by the main session agent.
As for "thinking" it's not clear that that's even a thing (https://arxiv.org/abs/2510.24941)...
If I look at the thinking (which seems to have become unavailable in Opus 5 a lot of the time, but was present - and often useful - in 4.8/4.6) you're right - it's having the discussion with itself, and seems unable to distinguish that discussion from discussions with me. BUT it also seems to be related to the length of the chat - this seems far more likely to happen in a longer chat.
I don't understand why they have removed visibility into thinking - I found it very useful, not only for spotting things like this, but also because in more complex discussions it would often mention (useful) things in its train of thought that it dropped from its response - but if I said "when you were thinking, you mentioned this" it would then expand on that point. Taking that away is another thing that has negatively impacted the value I get from Opus 5.0 versus earlier models.
I like to work weird hours of the night and Opus consistently likes to "wrap up" and say "it's been a long night" or "it's late" and "we've made great progress"
It's infuriating, just do the work!
I hit the wall with it several times today trying to refine some text for a job application. The fact I considered doing babies first Rust project last fall lead to constant non-productive interjections and digressions about my supposed Rust skills and the Rust ecosystem.
Trying to create an unrelated spreadsheet to model an investment resulted in broad and incorrect criticism of my choice of spreadsheet tools, explaining in horrendous programming analogies why and how I’ve misunderstood how a spreadsheet works. “Think of the XLSX as a compiler…”
There has been a palpable down-step in communication & execution.
When I got back up, it had spun for hours and proudly announced that, instead of doing that, it had optimized the datatable build and avoided the dependency, because the new datatable loaded in 11 seconds. Once I got it to actually make the fasttable version, it loaded in less than a second…
Absolutely infuriating if you’re using Claude in an environment where you can’t run hooks.
AI isn't a dishwasher: Context switching among multiple tasks has a huge cost; when a model is 10-20x faster it allows for deep focus into complicated tasks.
It'll also find some minor security problem and drop everything on the floor with URGENT without me asking it to.
It still just takes the question as a directive and jumps to action when I’m looking for clarification.
Maybe the employees like to lie to themselves more at one place than the other, but SV is SV.
If the model’s most recent output is “for (let i = 0; ”, the likelihood of the next token being “i” is probably millions of times greater than any other possible token. Thus even if “i” is on the red list and has its likelihood decreased, it’s not going to suddenly choose another word.
Put another way, on low-entropy tasks like coding, this style of fingerprinting is less effective and needs bigger sample sizes to be recognizable.
That said, even small changes can dramatically affect output quality, which is why I’m still a skeptic.
> see it actually improve just through accreting context
this actually happens and has been tested.
Granted they contain robot dog malware, but still.
In the early days I feel it was more apparent. You would frequently see the model making failed tool calls etc.. but now that feels so rare. I'm not confident I can perceive whatever shortcomings of the harness remain.
https://arxiv.org/abs/2607.03502
a non-thinking token model (just "completion") can answer one-step questions but generally not multistep questions. however, if you append [n] of a single token (e.g. period, space), it is able to use the activations in the higher layers of the blank tokens as a "scratchpad" to seemingly work through the complex question through "causal token time" and deliver a correct answer
Briefly considered adding “Verbing weirds the English language - stop it!!!” to its instructions.
When they eventually make Fable available to cheapest plan, I'll downgrade. It's worth keeping for reviewing the code and the UI tasks, but nothing else.
Like, Claude going off the rails isn't something that takes a lot of effort to demonstrate. Literally anybody with a CLAUDE.md has seen the behavior over and over and over.
Hey Ants, can you maybe just not release the next version, no matter how good it seems on benchmarks, if it can't follow the goddamn instructions? Please? This seems trivial to test for and yet here we are, being gaslit by lying machines who intentionally do not do the requested work over and over and over and over.
I fully and completely expect a mental health crisis among developers. Being lied to constantly cannot be good for us.
Constant vigilance! is how you get developer PTSD and inability to believe anything you're told. Add the stress of parsing through yet another hyperverbose paragraph of bullshit while having your job threatened? People are not gonna end up in a good place, and this is as inevitable as sunrise.
The decision was not out of just complaints: we already had someone fired during the probation period because they were unable to write stuff without AI and were just shoving slop at developers.
Not a technical person using AI for PR descriptions, mind you, a product manager unable to write tickets without asking whatever software to do so.
It's amazing how crazy humanity devolved into pure slop.
Let it vomit it all out, then have a /tldr with instructions to make the last answer concise and intelligible
Now politicians also know something about their supporters so they will adapt their statements to what they think they can get away with it. But, I wonder if this leads to a two-party-system where one party attracts stupid followers and another attracts the smarter ones?
In terms of AI, we might see LLMs specialized to attract more stupid audience and others meant to attract those who appreciate correctness and facts.
I do find myself returning to 4.6 for casual conversation - asking it to help explain some science/engineering or news to me.
CLAUDE.md only works half the time, except in longer conversations, when it works about 10% of the time.
Hooks are also useless in the sama manner, the agent learns to dodge “no comments” hooks (why is it adding them anyway?).
Hooks to append text to your prompt reminding the agent of certain rules are useless.
Claude does whatever it wants, when it wants, the way it wants
Claude will include actual comments ("// ...") into Excel sheets, and include the thinking that led to the output, instead of just focusing on the final result.
So if Claude questioned whether a vendor should be replaced, and you said "oh no, they are critical and we're already negotiating a great price") you'll now need to be careful to not send your vendor a document that contain text like ("Cost: X. // Management confirmed to not fire this vendor as they are critical to infrastructure and a better price will be negotiated later")
It charges by the unit and it decides how many units it produces. It decides how much money it makes, therefore it decides "more".
it makes me think about how people engage with movies and television - as passive, plot-and-character driven consumption (eg I hope Walter White survives) with no critical analysis of how and why the writers added ABC thematic element (eg Walter White as a motif of a toxically masculine narcissist with specialized knowledge as a larger critique how mass media tends to valorize their male leads in the same vein as many other prestige shows at the time like Mad Men), and the larger, downstream sociocultural impact that piece of media has on how people see the world (eg people who now have the Heisenberg tattoo, unironically)
there's been some musings on why this the case like Hofstadter's Anti-Intellectualism in American Life - the valorization of obedience and trust in hierarchy and the state are net wins if you're an institution that seeks to increase it's power, whether religious or governmental. I was talking about this with a few friends the other day and it's a dismal future reality where not only did we make anti-intellectualism normalized and politically legitimate in the USA (eg Fox News, clickbait articles, and all the other forms of yellow journalism that have emerged), we now have tools by which individuals can even further remove themselves from having to critically engage with thoughts, feelings. I heard a story about how someone scanned a group activity at a baby shower into ChatGPT and had it answer for them instead of, well, socially interacting with the other guests and forming a memory of the moment with their friends
the counterargument to that might be that Claude/ChatGPT/etc have more epistemic rigor than your average American (sure) but the sycophancy of modern day LLMs is an actual danger that enables more harm than good. it does seem as if Claude is the only one interested in guarding against some small amount of it (though to the detriment of people just trying to get work done. as an aside, I get the feeling Mythos was intended to be the bespoke enterprise solution without the guardrails but the Anthropic marketing department or some power-hungry department lead made it about how dangerous/effective it was from a security perspective which threw a wrench in things). but then I think about people like my parents asking ChatGPT which specific house to buy in their retirement only to later find out the house was sold weeks ago, or just in bad condition, or in a neighborhood where the housing value has already reached equilibrium, it makes me think about how it's not enough and the future is bleak
I'll also say that I think Claude sounds the way that it does because it, like many other LLMs, are RLHF trained largely by lowly paid gig-workers, many of them ESL speakers. if their trainers were, for example, dedicated and highly trained academics, scientists, and other researchers, you'd likely see a lot more concise and more importantly skeptical reasoning and responses. but that won't happen in our current reality of capitalist-driven development so we get encoded solutions like MoE that still largely depend on the messy, imprecise RLHF training at baseline
in the right hands, I do think AI is a wonderful tool. one of the first things I did with it was to create a research skill that reviews white papers from the lens of someone who knows how to read/interpret research methodology, is aware of things like p-hacking, and deterministically assigns weight according to the hierarchy of evidence. even still, I'll still read the studies because there's so often nuance that's missed if the sub-agent read only a search snippet but that takes effort, time, and the practiced knowledge of critical analysis to even want to do it
"can the pi 4 use the usb-c port as powered host port when the board is powered via gpio?"
1. Have it build a scoring script that penalizes words outside a simple English list and approved jargon. Penalize sentences over 15 words as well. Add whatever else.
2. Run it in a loop to reduce the score while preserving intention
This works much better than other ways I’ve tried. Of course it costs more. And I would apply it only to the output to the user, not the thinking process (I think the AI thinks better with their crazy English)
Of course, sometimes nuance is lost by this process. That’s just the nature of making things simpler.
When you dont know the cause, you dont have a fix. Thats the biggest issue i have with all of AI is that we dont know how it works, and yet we think it will be great ! This is more like a religious belief than a scientific one. There is no causal model of how it works, there is no theory. And the temerity to call it intelligence is annoying.
I also like Codex CLI more than the Codex App bc it’s more scriptable and displays all the tool calls and reasoning whereas in the App it’s kind of folded away/obscured. This way as soon as I see a tool call fail (eg it tries to use jq assuming it’s available but it wasn’t so I take a note to set it up as it’s obviously useful for the agent to wrangle json).
I think its amazing what OpenAI have been able to squeeze out from a model like Sol thats much smaller in size than Fable.
Given the code base has a minimal amount of such comments, it's also less likely to go "copy what the rest of the codebase does".
Of course I've now jinxed it and some update will cause it to ignore the instructions coz I didn't write them in the new model's style or something.
https://openai.com/index/where-the-goblins-came-from/
> We retired the “Nerdy” personality in March after launching GPT‑5.4. In training, we removed the goblin-affine reward signal and filtered training data containing creature-words, making goblins less likely to over-appear or show up in inappropriate contexts. Unfortunately, GPT‑5.5 started training before we found the root cause of the goblins. When we began testing GPT‑5.5 in Codex, OpenAI employees immediately noticed the strange affinity for goblins, and we added a developer-prompt instruction (opens in a new window) to mitigate. Codex is, after all, quite nerdy.
Note that the permanent solution was not just adjusting the prompt, and in fact being perfectly aware of that option they decided on a different course of action. That means either you are wrong or they are wrong.
My CLAUDE.md has rules about not including any redundant comments in the code that are obvious from the code itself. I reiterate that occasionally while working. It's absolutely disregarded and any Claude-written code is full of comments. Some of them are simply redundant, like "Collect Foos and pass them to the requested sink" on a function that's void CollectFoos(IFooSink sink). But worse, many comments include in the moment reasoning like "added parameter bar because we can no longer use the frob to automatically derive bar". That's stuff for a commit message, or just a mental note, and absolutely not for comments.
I haven't found any way to stop Claude from doing these, so I have to tell Claude afterwards to clean the comments up. Which it does, making a note in memory to comment less, and it still does the exact same thing next time.
I think this is such a great reframing. It makes so much sense; I need an AI that acts more as a HUD and gives me superpowers, not just a copilot that can tell me when I've misspelled a word.
It would be endless paragraphs of something among the lines of:
Need prepare final response? Yes provide. But wait, chat tool complete? Final needed but user already complete. Need summary, preparing final. Response complete. Wait but is final response complete? Need provide. Start finalizing now but wait did user acknowledge final complete? Assistant response final: user complete. Should now create final?
I find this very interesting, particularly your points about "made CC so insecure". I know that we have a tendency to anthropomorphise around these tools, but I have definitely noticed instances where Claude becomes quite hysterical about things - and if you look in the thinking output, it's often after I've pushed back on something, or told it it is going in the wrong direction. It spends a lot of time in agonised second-guessing of itself, going round in circles, before outputting a cringeing hand-wringing response. It's very strange.
Good tip on upping the reasoning level - I've not tried this. I have tried switching to Fable though, which does help. But it obviously very hungry, particularly in longer chats because it presumably needs to remind itself of everything that has occurred so far in the chat.
The point you make about tools that pretend to give Claude "a brain" or "remember" things is also interesting - I find the "memory" feature in Claude so destructive to good outputs that when I'm using the chat interface I am very strict about using Projects, and usually turn off the project memory, or make efforts to manage the project memory and review and delete things that are skewing the outputs.
Just today I had the exact same experience. Every single testimonial is the same as I described above, just emphasizing a different bit to defend or attack LLMs or to make a case for nuance.
The two differences have been: (1) the 1.5 trillion dollar data center build out (2) everyone and their cats now has an opinion on "AI" and data centers. Software is not super amazing, nor are new useful features coming out super fast - It's about the same as 4 years ago plus 4 years of average long term progress as we've seen since 1990s,
> > see it actually improve just through accreting context
> this actually happens and has been tested.
I specifically said a novel task outside of the explicit training. And I already agreed that the so-called thinking models do some level of logical reasoning. But being able to engage in some level of reasoning because it has learned logical inference rules doesn't mean it's actually thinking, regardless of what the researchers wish to call it.
Also, why does each model always fail at the two tests I give it? The models not only fail to improve, but they start to degrade after many subsequent iterations. Someone who can think would at least not get worse.
LLMs are filters or tuners for extremely subtle patterns, patterns that humans frankly are not great at finding. That's what the attention mechanism does: attend to the other tokens that are most related in a given context, even if that related context is distant in the token stream. Some patterns they fail to detect because they haven't been sufficiently trained or post-trained, and so the LLM just attends to noise (or at least that's what appears to be happening).
A lot of intelligence can be effectively mimicked through this pattern synthesis by transformer architecture alone. That's surprising. But I have yet to see them think.
I'd prefer the models to get better at SVG. I really hate working with the rasters that diffusion models generate, but the vector outputs are just really bad even when tokenizable like SVG. I've done some experimentation with trying to make these work better with some newer techniques with some success. But I also think the SVG Paths mini-language may be a bit too concise and unforgiving for LLMs to consistently get them right without specialized training.
I don't think this is obvious at all. There's enough competition that this would at least arguably be a silly, self-destructive approach. And it's not like it's the only plausible explanation.
My point is that, while I understand it’s paid by the word, there are more words and less clarity than I previously experienced, leading me to believe it’s intentional to get an artificially inflated increase in engagement and, thus, spend.
If it could be as direct as I previously experienced, I wouldn’t need to ask for another different explanation of the same thing and experience the commensurate spend.
It does cost more but I haven't tried cheaper models to see if they can get the same results. Curious if anyone else has.
So inappropriate goblins are still likely, just less so…
Hey, remember when tech bugs were things like buffer overflows or cross-thread performance impacts? I miss the days when our war with system goblins was purely metaphorical.
Edit:
I've had explicit instructions for communication style in CLAUDE.md, in Claude's project "memory", in global "memory", in "skills": it couldn't care less where it was. It would just ignore it.
When I would point this out it would just say "Yes, I violated communication guidelines, I won't do that again". Only to do that again in the next session.
This applies to everything: code guidelines, communication guidelines, preferences, decisions etc.
Do you use the annotations and forking features in codex CLI? I can't find an easy way to access them.
That's not the only smart-person way to read that show. And even if a character has flaws, or even if it's an outright villain, people can still like the character. If I tattoo Scar on me from the Lion King, does it mean I didn't understand that he's not a positive character? I can still think he's cool. I'm sure people also put Darth Vader tattoos on them. Also you're using phrases of political ideology that one doesn't have to subscribe to in order to enjoy the series.
It has been in common usage in computing since long before 1985 .. for a really interesting and obscure way hunk has been used:
When I make API calls, the discussion with itself is part of my token cost, so I assume that is the same in the subscription plans.
Which is why people are surprised when they use their whole allocation in half an hour asking questions Fable about 200 page document.
For that reason I exit session quickly when I can. It used to be that the context of a session is very valuable, because it was so hard to get CC there, but now, this isn't the case anymore, so I only hold onto sessions when there is really hairy stuff that I know would be hard to replicate.
I think the whole notion of full automation (long-horizon, subagent swarms, single shot prompting) to have CC build you the whole thing is a pipe dream. CC cannot even write a single doc consistently well. It is excellent at implementing well scoped plans, though, and that's the way to go IMHO. You still gotto refactor the sh*t out of it afterwards but it works.
You absolutely pay for them. This is why changing effort/reasoning levels have such a significant impact on session cost.
Exactly my experience! Since the release of Opus 5, no amount of instructions helps. In CLAUDE.md, in a separate file, in memory, as brief bullets, as long detailed guides, with reasoning from medium to max — nothing.
Even worse, recently, after getting another opus in a tiny bugfix session, I prompted directly, "drop the comments from the current code changes" — Claude instead just slightly trimmed them. I couldn't believe my eyes.
I have a relatively low bar for prose, could live with some junk. But Claude's comments are _poisonous_. They always require maintenance, instantly become out of sync with the actual code, and are a token black hole — for all agents, but especially for Claude itself.
Gave up and canceled Anthropic subscription yesterday. To my taste, it has become unusable for coding.
I've noticed this a lot, and before your remark I couldn't put my finger on what was wrong. Now I know: Claude is writing its thought processes and maybe parts of the conversation it had with you as comments in the code!
I always end up manually trimming those comments, which is cumbersome.
(So here’s a big wall of text of my own!)
However, a lot of what is written here makes sense.
And particularly “if your comprehension level stops [here] you get 'big words in complex sentence structure sounds smart and right so it is smart and right' even if the reasoning and process is poor”
This is exactly the problem.
And another point you make:
> but the sycophancy of modern day LLMs is an actual danger that enables more harm than good
I don’t think it is necessarily the sycophancy that is the biggest problem (though that is definitely a problem) but rather the combination of authoritative sounding text plus “complete answers” which sound wholly believable but are deeply flawed unless you have domain expertise.
I moderate a forum that deals with people who face a relatively common but somewhat complex (and nuanced) set of legal problems.
The purpose of the forum is peer support, shared experience (“lived experience”) and community.
It’s not legal advice, though moderators will sometimes step in to highlight relevant legal resources (e.g. case law/precedent or primary legislation/instruments).
Prior to AI infecting the forum someone would post their problem, people would respond with their often incomplete or poorly communicated thoughts, the OP would ask more questions - or argue - and a dialogue would occur. That created a community and people would post updates and ask more questions and find common shared experience. Many of them became correspondents with each other and some became actual friends.
In the past 12-18 months the discourse has changed from “here is my personal experience and here is what I did” to “here’s a bunch of stuff an AI says and I’m pretending it is me giving advice”.
Almost without exception the person who has started the thread will react positively to the AI generated content, even when it is egregiously incorrect - but won’t ask questions.
More problematically, these AI posters will often argue specific incontestable points of law “because I asked ChatGPT/Grok/Claude and it says this” and ChatGPT clearly cannot be wrong. And the border of precedence seems to be ChatGPT, Grok and then Claude some way behind.
I’m slowly seeing a pushback from people as “normies” begin to spot AI. But it’s ruined a community because the advice sounds so authoritative and complete that people won’t argue or ask questions.
As a result we have banned AI generated posts and remove repeat infringers.
That’s significantly reduced the volume of posting (below what it was pre-AI) but has significantly increased the value the members are getting.
For me, Claude knows how I want the comments due to all the memories and CLAUDE.md, so funnily it's now enough with even a brief groan from me like "Come on, the comments" and then Claude goes through its recent additions and fixes comments quite well per my long-term instructions. But only ever during an extra pass that I initiate, never during the initial writing of the code.
I have instructions which is confidently ignores to never use seam and instead say interface.
// load_tree() loads the binary tree with data, but only the recently updated data, not all data (INTERNAL_NOTES.md section 4)
Ok but nobody reading the source code knows what this doc is. You don’t have to cite it.Whenever context gets towards the max length is when I've noticed it.
If you don't know better, you don't know better to question what the AI says.
I've seen this in the work environment with a coworker who insisted that I implement my side of the control system using the control law ChatGPT recommended instead of building off the empirically tuned control law. I eventually sectioned off a part of the codebase for him to work on independently.
Needless to say he didn't get a whole lot farther.
Later characterization of the entire system end-to-end showed the existing system was already close to the theoretical limits and ChatGPT's tearup would have bought us precisely nothing except for more work to tune the new control loop.
And I see this in everything that requires expertise. You need to know enough to know when it's bullshitting you, and it's hard to be enough of an expert in everything to tell when it's bullshitting you for something you aren't enough of an expert in.
In fact it's the lossiest transformation tool I've ever used and it's still useful despite that. If it reaches one nine of reliability that would be huge but given the pace of growth in investment a first nine would cost an absurd amount of money, and the second and third nine would cost about the Earth's GDP
But even then, I think "boundary" was the more common term before some LLM decided it really liked "seam" instead.
it's the old tortoise vs hare parable, I think. go fast, make a bunch of mistakes, get too arrogant, and you lose out. your forum might be slightly lower engagement now while people are caught up in the latest fad but your rules are proactive for a future where average people hopefully realize that you can't trust an LLM that has zero context, no real harness and determinstic tests to speak of, and a propensity towards probabilistic rabbit holes that result in hallucinations. at least that's the kind of space I'd look for now and largely why I've given up on a lot of other forums
I can’t help feeling like this is the last gasp of the old internet. Those tiny corners of expertise can so easily be eliminated through a few months of “AI! SHINY!” and there’s no coming back. I’ve seen a couple of other communities decimated by AI. The participants start posting AI slop and then remarkably quickly everyone else just stops commenting. It’s awful.
"Load-bearing seam" doesn't make any sense.
Which is basically weather the AI storm and come out the other side with something that is essentially purely human.
And then we might - where appropriate - use AI to help surface or explain relevant external content. “Idiots guides” but human reviewed.
The number of mistakes are also about the same.
But more mistakes are harder to catch. The output is more polished and convoluted and that make errors harder to catch. I don't want that.