I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.
I believe this works in two different ways.
First, information compression. The use of professional language helps describe problems more densely with minimal information loss/distortions. Verbose output by LLMs (e.g. ELI5) tend to incorporate local chat context, which can destabilize the context (e.g. out-of-topic, irrelevant nitpicking on writing style and wordings) and lead to faulty logic and even hallucination. LLMs are not good enough to look through all the noise, so, sometimes, it's helpful to refine the input data before performing actual tasks.
Second, boosting logical pattern-matching. Using professional language helps drive logical reasoning through simpler pattern-matching b/w texts. This is not about whether LLMs can reason or not; it's about how high-level reasoning is guided by preconception. Even humans tend to consume only textual surface of highly complicated theories (e.g. Adam Smith's "invisible hand"), and use them casually during conversation. It's similar for LLMs: if the conversation is conducted entirely in professional language, LLMs can easily incorporate external professional information into its reasoning. If the text is written in amateurish tongue, translating it into professional language can introduce errors and distortions.
So, yeah, keep your conversation professional, tidy and tight. A large volume of unprofessional text helps no one.
Someone with less skill or expertise might still get good results, sure. It would just take longer and it wouldn’t be pretty
The information system required to encode the aesthetic preferences needed to make a technology experience not suck is likely in excess of what any near-term solution will offer. Knowing when to say "no" is perhaps the most important skill here. You can't just say it arbitrarily either. You really have to mean it and be willing to fight other humans for it.
It's important to note though, from the perspective of the LLM's objective function, that this is not a failure at all! LLMs are designed to match patterns. You give a jumbled mess of incoherent ideas, it will faithfully reproduce a token stream of incoherent ideas. It's only when the model output is subjected to the real world that it fails.
I've seen this at work (as eng manager/lead/principal/whoevenknowsanymore) – all the big APIs give you stats. We see how much people burn in tokens and we know how much output they produce. There is a pretty strong inverse correlation between token burn and output.
The more tokens people burn, the less likely they are to produce a good outcome.
It's less of "signaling expertise" and more about actually having said "expertise".
In my experience with LLMs it's not uncommon to be having a deep conversation about making pasta, only to be told, after asking for a sample recipe, to get a bucket of paint and a bag of concrete. Of course these hallucinations are way more subtle and easy to miss for someone who doesn't have deep domain knowledge.
I don't even debug anymore on those projects. If Claude tries to add debugging logic in my code, I tell it not to and just provide additional information and it can usually find the solution faster that way.
This is when working on my own projects. When working on projects created by other people, it's a different story and I have to fight it constantly to stop it from implementing hacks and workarounds... It uses much more tokens to implement basic features. It's more work for both the AI agent and myself.
The project's existing code makes up most of the context so if the code is not great, you have to write long detailed prompts to set it on the right path. You have to make it clear that the existing code isn't good enough and your expectation is higher.
In this case, it usually gets better with more back-and-forth... At the beginning, it can't do anything because you keep pointing out a problem whenever it tries anything at all, but eventually, after a lot of criticism, it starts becoming more careful and adapting to your standards.
So yeah, even same person doing the prompting can lead to two very different experiences depending on who built the foundation.
So my conclusion is that the expertise comes from both the existing codebase and from the person doing the prompting... And TBH, I would say the codebase/foundation carries more weight than the person doing the prompting.
Pretty sure I could put an idiot on one of my codebases with Claude Code and they'd do a decent job.
I’ve found the most utility in having it do “small stuff that I could do, myself, but it’s faster to have the LLM do it,” and in analyzing intractable bugs, like memory and threading problems. It’s really good at analyzing a bunch of code, and seeing a small typo that results in something like a strong reference.
In both these cases, my own expertise is vital. I’m asking it to act as a consultant; to give me advice and material to be integrated into a whole that I am architecting.
I guess part of it, is that I haven’t been able to completely “give in,” and wholly trust the LLM, like I hear many people do (profitably, I guess). I’m used to having my sleeves rolled up, and my hands in the dough. Catching some pretty severe mistakes, from time to time, has reinforced this perception, on my part. I wouldn’t catch these, if I didn’t know what I was doing.
- ..Does this polynomial map have any symmetry or other structure that makes this cancelation less miraculous?
- Given this structure can you see the non injectivity in a transparent way?
- ..But why is the jacobian from x u r to P Q R just a monomial?
- ..Is there a general theory of such twisted jacobians and do you have any sense why those particular dilation weights were used?
- Given this weight structure, why exactly is x given by a cubic equation from P,Q,R?
Also, looks like Terence Tao was doing lot of work and asking LLM to verify. This is inverse of the LLM trend, where LLM does the work and humans verify.
Labs have to compensate with post-training in RL env that embeds these expertise well, which is non-trivial both in terms of domain knowledge and technical expertise.
https://www.newscientist.com/article/2580932-extremely-basic...
The referenced Terence Tao chat did not lead to new breakthroughs.
If you use the b words the LLM would be somewhat constrained to content that has them. If you don't use them it will use content that doesn't have them.
The company website (subject) will never become a static html document. If you request exactly what static html does it should probably point you to a wysiwyg website builder.
Your prompt might look almost the same as the one from somebody with a good understanding of the requirements but no technical competency, plus maybe one or two sentences. Those one or two sentences dramatically change the results, and what those sentences are differs from prompt to prompt.
These internal "attention weights" are highly influential parameters of how I work with LLM.
E.g. when working with Rust I often hold strict control over structures and lifetimes. But when lately I've been doing token-based bind generation I didn't care about anything outside of high level patterns like RAII and ultimately - API ergonomics which was verified in consumer app.
I've been in position of porting real-code to vibe-code platform and seeing non-technical people prompt-stream (they were shared across accounts) I know why they engaged engineer to run this work.
Their efforts took 6 weeks, I ported app within 4 days and (to be honest with myself) without LLM I that'd be 3M+ work pre-LLM.
In short: I observed same effect as claimed.
Maybe, but FWIW my first thought when I skimmed Tao's session was that he probably has a personal system prompt requesting this style.
E.g. even if you get it into "talking to an expert" mode I've found AI waffling through filler like "given your background in Linux kernel engineering, I'll skip the surface level and go straight to the technical meat". You do have to explicitly tell them if you don't want this.
Is it possible that the effectiveness of an LLM user with respect to the expertise of the user is like a sigmoid function or at least a step function in that shape? That is, one has to know something like the basic concepts and the vocabulary to bootstrap a programming project, but one does not have to know too much to do lots of meaningful work, and then again one needs to be en expert to build something extraordinary. Since most of the work is somewhere middle, most of us mere mortals are still concerned or stressed out for the possibility that LLMs will squeeze out too many job opportunities.
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
E.g., medical history taking protocol always says to start with open ended (albeit structured) questions, and converge towards more closed/specific ones when you're sure you've extracted the broader surface and you now want to close in on a differential diagnosis.
If you start open and go with the flow but then just let the patient talk without any structure or subsequent attempt to converge, there's a risk that the patient might spend 60 minutes taking about their fluffy dog at home, which wastes time, and doesn't get you anywhere nearer the diagnosis. But, if you skip the open questions and go straight to yes/no diagnostic questions, you will definitely miss the fact that they have a dog at home that they're worried about, and that they'll be self-discharging against medical advice in the next hour to go tend to their dog.
So while to an outsider, the conversation might look effortless, in reality the doctor requires considerable skill to be able to strike a balance between open vs closed prompts, as well as the ability to critically sift through the outputs, and decide which outputs are relevant to pursue further and lead to a fruitful direction, versus those that can be safely discarded to remove potentially distracting noise from the conversation (and all while attempting to keep this interaction within a limited number of prompts due to operational time constraints).
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
Prompting feels a lot like this conditioning phase to me. You start with an LLM in unconstrained mode, basically just a "soup" of knowledge. If you prompt wisely, you immediately condition the LLM into "your space of (domain) knowledge".
What comes out is an extended version of your existing knowledge.
"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"
https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7
https://xcancel.com/__alpoge__/status/2083855298239078748
Tao's chat was for him to gain intuition, not to solve the problem from the outset.What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem (without searching for "how to center a div", or whatever your exact problem was, and copy&pasting the answer and moving on), also becoming more knowledgeable in the process.
The rest of the short blog post has some good points, but the first sentence sounds like it's targeted at the percentage of developers who did StackOverflow copy&paste to close Jira tickets, never becoming experts.
Delegating to LLM-ish AI is just a natural evolution of that. The question is whether they can still add value if kept in the loop.
The article author suggests that the answer is to be expert, and is addressing people who... "either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet."
A real world example might be: I can find any movie DVD you want from our warehouse, but you need to tell me the name of it. Don’t know the name? Tough luck.
Contrast this with a “browse” interface: the options available are presented to you, and you can pick from them. Relevant contextual information is already on-site. The DVD store has shelves of potential movies you can rent, and you don’t need to know their names ahead of time.
The interfaces of future AI will be more browse oriented, with a query viewer available in the settings for advanced users.
Where my angst comes, is worrying that no one will ever get that experience anymore. They might have had some eventual success, who knows what monstrosity a much less guided LLM would have done, but experential learning may be mostly a thing of the past. And it creates a real tension between the person with experience and the person without.
Anyone can point at a public GitHub repo and have an LLM write the documentation for it. Whether that documentation is good requires understanding that codebase.
There's no way around expertise unless we're talking about strictly mechanical tasks.
I do think LLMs are incredible at helping to build your expertise. You could point it at a codebase and say: "Explain how this API works" and interrogating the LLM until you get an explanation at exactly your level of understanding.
1. AI cannot do something.
2. AI starts being able to do something, but one needs to prompt it carefully, so one needs to be an expert, see, we will always need human experts <--- this article is here
3. AI just one-shots it.
Why do people still need to say this for each and every task? It's just reliving the bitter lesson over and over again.
So yeah, sometimes LLMs reward expertise, sometimes they don't. I guess either way it helps to have it.
I loved graphing calculators until I learned tools like Mathematica and Matlab. Still waiting for the Mathematica version of LLMs.
Agents / loop engineering / whatever is hot with the AI Twitter kids still isn’t it.
(And even for things that were in the training data you don't have any guaranty they will be reproduced correctly, there is just some chance something meaningful comes out, or it doesn't, it's random.)
The whole idea to use a next token predictor as "answer machine" is completely flawed. This can't work like advertised, and that's by construction.
The example math is boundary-pushing and definitely not a solved problem. But most of us work on CRUD backends with a React frontend. Those are more or less solved problems that have well-documented solutions. For those kinds of tasks, LLMs just reward usage.
I can count on one hand the number of times in my career I've needed to solve a problem that's not described on Stack Overflow.
An expert can lay a different kind of frame to prevent the llm to fell out of its way of being generally too verbose, and that can transfer as well to code generation and complication.
Inevitably, this is the new tradeoff to make, above average quality comes from asking for more, and knowing what to ask for comes from expertise.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.
I'm now learning how modern frontend development is done (for both personal and professional projects), so I may not know the specific tools, technologies, or terms but I can say "whatever the equivalent of XYZ is" and the models will translate for me. If I say "run pytype" it will tell me "we're using mypy - i'll run that checker for you".
If you can express what problem you're trying to solve, that will get you most of the way - and then you can refine by asking questions. "I think I need something like Redis for caching things - do people still use that? Is there a simpler more modern version that is the new standard? Do we already have company docs suggesting what to use?"
What harness did you use?
In e.g. claude, there are two modes:
1. Spit out code 2. Draft a plan, ask questions, GOTO 1
You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so
Particularly with something static, I don’t think they’d fail to get a result.
But without domain knowledge I think they’d misunderstand prototype with finished product.
Without knowing what it’s doing, it’s hard to know what it’s not doing.
Claude code is entirely vibed. Someone posted some of the prompts they used: barely comprehensible typo-addled half phrases.
This seems to be the opposite of your experience.
Tiny, isolated, but awesomely useful CLI scriptlets, for me, seem to be the sweet spot. Little shining rays spreading out from the veins of my own familiarity.
The downside, the Achilles Heel of LLMs, so far as I can tell, is using the system to assist in maintaining large, sprawling, and largely pointless legacy codebases. Somewhere you have to keep many many many stupid things alive. I swear I can almost sense Claude's frustration with some of this shit. Then you get frustrated, and then Claude wants to agree with you so it acts even more frustrated, and the gyre thus widens. You're just cussing at everything with a machine. Which can be fun - Claude is often surprisingly funny - but not productive.
[1] I know absolutely nothing about positively everything, but have the attention span of a squirrel.
Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow. Now it's instant. As if your very first Google search just solved the problem for you immediately.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
In fact, I never enjoyed frontend programming because it was such a pain to deal with matters I considered trivial yet so frustratingly hard to do right... like centering a div. And yet the slightest misalignment is visually jarring and forces me to get a bit OCD about fixing it, which made it even more frustrating.
I questioned the whole premise of the situation: is working around a bad developer experience something worth spending my time on? Unless I actively wanted to get in there and fix the situation, not really. So yes, in those cases I would outsource my problem to a colleague or StackOverflow and move on. And as a career choice, I preferred to do more backend dev.
I would posit that that was the type of expertise that did not matter. The type of expertise that really matters here is good UI design. That is entirely orthogonal to the drudgery that is implementing and debugging webpage rendering, and I am eternally grateful to LLMs for freeing us from it.
You can extend that line of thought to the entire article. What really matters (and what LLMs reward) is domain expertise rather than technical expertise.
>Yes. Nearly everything you described has a standard name. Here is the vocabulary, organized by what part of the page it describes...
Goes on to identify Header, Navbar, Stucky header, hamburger menu, hero, CTA, Above the fold etc.
>So your described page is: header/nav -> hero with CTA -> card grid -> footer. That is the single most common landing page structure in existence, and that is fine. Being conventional is a feature, not a failure.
I've had the same conversation with an electrician wiring a car charger: we are more likely to succeed if I use his terminology.
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
Prompting is conditioning, that is what it is. The visual of a GP (like the thing you get if you google image search “Gaussian process”) is a great metaphor for what prompting an LLM is doing.
The output of the LLM is the logits which is sampled - plucking out tokens from a distribution. The input of the LLM is data which constrains the logits. That is what it is.
That’s also how you know that AI will never “solve” intelligence (the way the boosters say it will) without some general mechanism for this conditioning process. The ultimate mechanism would be embodiment; the crappy mechanism we have now is something like openCLAW.
As the old joke goes, a mechanic charges you $5 for hitting it with a wrench and $495 for knowing what and where to hit.
"LLMs reward expertise" is the title, not that "LLMs only make things possible for those with expertise"
I'm fairly certain the article is directed at professionals, or at least the AI companies are basing their valuations off of directly taking a slice of that professional "productivity".
A concrete example: GPU performance optimization for a kernel. This was (and still is) a very niche domain with not many top-notch experts. But kernel performance and characteristics are easily verifiable. You can run the agent in a closed loop for it to improve iteratively (and people are already doing it, coming up with kernels better than human-written ones).
You see Tao's example because:
1. He is curious (so he asks detailed questions, which are not necessarily needed in a closed-loop optimization).
2. Verification in math is harder. Many math tasks used in RL are easily verifiable. But for advanced open conjectures that require long proofs, you cannot trust the proof directly from the LLM (so it's not as easily verifiable as basic math problems or code). The model needs to write it in Lean, and you still need to make sure the Lean implementation correctly captures the specification of the problem. So you still need a human for verification in advanced math. But I don't see why you would need this in domains like performance improvement.
But when a 30-some year old shows up at a rheumatologist with joint pain they will likely go to unusual (at that age) but not unheard of rheumatism/arthritis, not hypermobile spectrum disorder. When a woman goes to a GP with period pain they will be prescribed mild pain killers or anticonception pills and fobbed off, until a decade and much suffering / many more issues later they get diagnosed with endometriosis.
I see vibe coded apps as requirement documents. Rarely do I have to engineer.
If my job gave me some actual tasks, then maybe I'd engineer something. But at home? Vibe coding all the way. I'm open to engineering, but I need a compelling reason such as: the app is fundamentally broken and an LLM is going in circles. When the only user is me, there are not many performance issues to think about or fix, so that helps. Moreover, certain systems don't need to exist (though they might soon since now I have a smattering of apps that I need to manage).
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say
at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting"
will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs
If you ask a non-programmer to install Claude Code, just installing it will be a challenge, then opening the shell and interacting with it. Things as simple as copying and pasting can present roadblocks if you've never used a shell before, and things intuitive to programmers like using up-arrow to go back to a previous prompt would never occur to someone in the field.
Claude Code seems so simple and natural of a UI to programmers, it's easy to forget how much it builds on.
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
Having to read through a structured resource describing something to figure something out has intrinsic value that an LLM is not going to provide you with.
Careful now: https://www.youtube.com/watch?v=TMoz3gSXBcY
But that's how you learn...
On the other hand, I've been using it to make small changes to a ~4000 line codebase, and it takes a lot of wrangling to keep changes in scope.
Today I'm translating a 5,000 line VB6 codebase to C#, and I've been spending the day chopping up the job, passing it to Claude, and manually validating it.
Note that this isn't too different from, say, how software engineers are expected to be good at, and make good use of unit tests. But most probably don't (either because they never really cared to fully develop that skill, or their organisations applied contrary pressures leading to tech debt). But it is a recognised skill.
My main point was that, it is, in theory, a skill that doctors are expected to train (or at least pick up on during their practice), and therefore the same prompting principles that seem to apply here in the context of LLMs also interestingly seem to apply to medicine and history taking when "prompting" and interacting with humans.
If it doesn’t go away and they come back, you start considering more serious issues.
It’s expected that uncommon non-emergent diseases will be diagnosed over multiple visits.
1. "find the code that does X"
2. go read that code
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.
(FWIW I think people betting their whole companies on AI are trusting shitty one-wish genie goblins, but the terrible irony is that anyone "technical" with years-old knowledge is talking about something else entirely in today's context)
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
We'll of course you're gonna be frustrated if you start with the hardest problem in HTML :/
Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these.
An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution.
As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something.
Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away.
(Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately). She uses that laptop daily now and says she enjoys it over Windows by a mile.
I counter with the platitude that common things are common - especially in fields like primary care, the amount of wasted effort one would expend in pursuing unusual explanations for every presenting symptom is considerable. We thus have to examine patients over time and trust that they will tell us if things have indeed not followed the course of the initial diagnosis.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo.
Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.
The wildest thing is, there's no evidence that I can find that static stretching does... anything? It increases pain tolerance through the range of motion, but not any more than just asking people to try harder etc. And it doesn't build up, so after you stretch once for a given day, you're done.
Edit: after looking into it a bit further there's actually a cochrane review that stretching does bupkis, fascinating. It's also the first "Good" evidence graded cochrane review I've seen.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
"update an AGENTS file with relevant information"
To be able to navigate that faster on longer tasks. Meanwhile, the lay person does not even conceive of the LLM as a file reading entity. To them, its machinations are its own, so these types of "dumb" (simple) solutions are not even on the deck of cards.
In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.
Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver — PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing — you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.
This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.
A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently-discovered counterexample to the Jacobian Conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.
There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:
However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multi-paragraph response, suggesting alternate approaches or formulations, and identifying what “looks weird”.
Terence Tao is a better mathematician than I am a programmer. But the idea here — that domain knowledge makes you better at using LLMs — is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say “no, I think it could be simpler here”, or “but don’t we already do X?”, or “can we express this problem in these familiar terms?“.
This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “does X work here?”, or “given Y and Z, why A?“. I can’t ask those questions about the Jacobian Conjecture, but I can ask them about the systems I own at GitHub.
If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.
The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.
edit: this post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI do have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.
If you liked this post, consider subscribing to email updates about my new posts, or sharing it on Hacker News.
Here's a preview of a related post that shares tags with this one.
Powerful AIs might escape containment by releasing themselves as open-weight models
Before large language models, people who worried about AI safety often talked about the “boxing problem”. It goes like this. Suppose some genius figures out artificial intelligence in a late-night coding session on their laptop. Because they’re a genius, they’re smart enough to disable internet access on the laptop before turning it on. In order to escape to the outside world (and begin self-replicating) it would need to convince its creator to “open the box”. Would that work? Could a sufficiently smart AI convince anybody to let it out?
Continue reading...
Edit: https://xkcd.com/2501/
I gave him a link to Ghostty, a link to the claude code copy/paste pipe to bash thing, told him how to cd/ls/pwd into the folder he had locally from the GitHub app, and he was off to the races. I told him to type `claude` to open Claude Code in the terminal and gave him a prompt to use about being a non-dev getting his environment set up to the point of being able to pnpm dev and test, and Claude took it from there. The repo's readme had a setup section which it followed to install brew, asdf, pnpm, etc. With the GitHub MCP he's now opening PRs the same way devs do.
The stuff that I produce in this way would probably be considered by many to be unusable trash. But it solves the problems I have, and it does so with exactly the amount of precision that I demand.
When I built a PWM fan controller for a pro audio amplifier, I was very particular about some aspects. I wanted maximum resolution from the DS1820B temperature sensors (which is a relatively slow mode where reads take ~750ms, and often the bot is primarily interested in fast), and resolutely-consistent PWM output (so software PWM was a non-starter).
It was very important to me that the fan speed ramp smoothly and without audibly-discernible steps, so the target output goes through a low-pass filter to smooth things out and the final PWM value gets recalculated at a completely-overkilled rate of 1KHz.
Power consumption was a very deliberate non-concern: The power used by the MCU is ~nothing compared to that of the whole of the system, so optimizing towards reducing it was never my goal.
At the end, it's a rewarding little project that is all wrapped into a state machine that burns clock cycles like they're free (they are free!), and it works very well.
There's parts of this thing that I do not understand at all, and that I have no desire to understand.
But if I hadn't been so particular about the parts I did care about, then: An underspecified one-shot prompt seems like it would probably have just produced a loop with a lazy 1-second sleep at the end, since being sleepy and power-efficient was a feature that the bot kept working to reintroduce.
I spent a lot of time working to dismantle the bot's proclivities to be this way, and I probably would not be happy with the end result if I had just let it do its thing.
Differently-stated: It could have been an unsupervised one-shot prompt, and the result almost certainly would have done the job of keeping the amplifier cool. (I just would not like it.)
I reminds me old WYSIWYG and unlike Figma it has full HTML/CSS capabilities available.
How I work with it:
- I ask agent to extract part of app into Design, let it even use playwright-cli to get full rendering of the particular view.
- perform design session in Design.
- once design system is perfected I go down to Claude Code dungeons, do /design-sync.
- perform on the stack implementation session.
Actually you don't need Claude Design UI for any of that too. Just ask any coding agent to prepare local mock HTMLs and iterate over them.
However for some reason it had him deploy a single HTML file with all the assets encoded as a huge base64 blob in the code that required a massive amount of JavaScript to extract and render.
It all boils down to naming things and cache invalidation, /s
Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose.
I probably generate 20-50 lines of code for every 1 line that I keep.
This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
Pre-AI I'd say I've gotten over the hump 3 times with CSS, only to lose it again by the next time I had to use it in volume.
But there’s a lot of pressure to get up to speed as a new hire and it’s easy to move fast with AI
It’s harder to internalise concepts because you don’t go through the struggle of understanding them and finding the mental links you need to remember later.
I notice this with people around me - all of them are doing more things, but I am also catching more issues when reviewing docs and code.
Obviously YMMV.
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
I made a lot of use of the TI-89 era CAS in college. But IMO the TI-83 era manuals taught me more about both math and computers than the subsequent generations could have.
Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals.
Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.)
These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
None of this is related to intelligence, desire or potential. It's about context and experience. The vast majority of people use their computers as consumption devices, like a TV. If I asked you to "make a movie" where would you start?
This is obviously not the case. There are mountains and mountains of boring, unimportant stuff that LLMs can do for us now.
For example, at work I can now make a nestJS dashboard without having any idea what nestJS is. I can just tell the LLM what I want, and it makes it so. This allows me to make my tool useful to people without having to become an expert on something unimportant.
No, of course not. Because all of that got abstracted to higher-level instructions decades ago.
You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?"
And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
E.g., I get the whole "if you hear galloping think horses not zebras" adage, but I guarantee you, if someone comes and says "hey when I was in Africa I saw a black and white striped animal galloping really loud", I bet you an uncomfortably large percentage of the "horses not zebras" crowd would still favour a horse over a zebra diagnosis, despite the overwhelming posterior.
Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests, and it becomes a big problem, whereby uncommon conditions are treated as common, thereby often escaping detection altogether, and driving down their apparent prior probability even lower, causing a diagnostic vicious cycle.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
This is the target user of these chatbots.
The people who make the tools that generate your assembly instructions need to learn it. Just like the people who make the browser rendering engine and push CSS forward still need to learn it.
The people who don't need to learn asm never needed to learn it. If you wouldn't code in asm now, you wouldn't have ever.
This " logic" is so irrational.
Yeah, I have a pretty good idea of how we go from code -> IR registers -> x86 registers and how the underlying hardware has more registers than that, etc.
As for SPARC and POWER, I have not yet gotten my hands on one! But as someone who writes compilers for fun I did learn about most of that, yes.
I write C# in my dayjob, and having written high-ish performance C++ and Rust for a couple years before that, and because of all the curiosity-driven projects I've done (like writing useless compilers), I did learn a lot.
This has helped me massively in my job, especially when optimizing our C# sensor data ingestion backend to process hundreds of thousands of measurements per seconds with explicit backpressure etc. it was great fun because I know what I'm doing.
I'm not trying to "uhm actually" you, but when you call yourself a software engineer, I do expect that you know some of these things after some years in the industry. The only way you wouldn't is a lack of curiosity, or because you genuinely had no idea that you CAN learn these things.
Anyway I'm not so sure "static" is a viable boundary between app and not app. A static page that does any sort of API request doesn't suddenly become an app imo.
Maybe web development, despite plentiful tutorials, just isn't quite the same.
I would say we are very close to the end of things like WordPress and templated websites. It's pretty easy to make a custom page.
Chatgpt can absolutely do the things here: it can give you files, it can integrate and show web pages that you develop, etc.
There is no need for an "agent", the chat version works just fine and is actually easier for novices.
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
Can you share the source? This seems dangerously wrong.
I see a specific review making the very narrow claim that it can have detrimental effects on power/strength activities immediately following static stretching.
But by and large I'm finding a great deal of evidence for a wide range of other benefits, particularly in range of motion and injury prevention. I can't find anything widely damning, and I certainly don't see any reviews contesting the validity of the very vast body of research supporting the many benefits of static stretching.
I'm doing it for knee pain. It's possible it's a placebo because there is a slight strength training component to the whole thing (it's several exercises), although the knee pain has persisted through a lot of strengthening of my leg in general. This falls under "don't care" as the program as a whole works and I'm not worried enough about the details to try to optimize it. All the stretches fall under "dynamic stretching under tension" so maybe it's not something covered by the study you reference last.
Presumably the Cochrane study you’re referring to is “Stretching to prevent or reduce muscle soreness after exercise”, which as it’s title suggests, investigates a narrow question and does not support you claim that “stretching does bupkis”.
You could probably replace me with a minimum wage worker to do some manual testing and copy-paste errors from the console into the LLM, and still be fine.
Let's see how long it is before the next round of layoffs, I guess. For now, the money's fine and the work's boring but ok.
But, no, the LLMs rewarding expertise line is pure cope. Software is not really skilled labor any more.
(obviously, throwaway -- I'm not interested in potentially advertising this to my employer)
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
Are you kidding? WIMP and GUI democratized computing!
Someone has to know it. And to advance it to the next level requires domain level expertise. Or are we allowing LLM's to determine the next CSS snapshot/version/level/whatever?
The moment you interact with an LLM it gives you feedback that you’re doing things right. It feels like a gradual climb instead of a series of abrupt jumps. People really don’t like feeling like they don’t know what they’re doing, and the terminal constantly reminds you that you are making mistakes.
1. The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas.
2. The user was the sycophant in this relationship, never steering the session(-s) into producing something tangible.
The premise supposes that somehow the session(-s) never even tangentially touched implementation/deployment ideas and the user has never typed something like "that's enough, how to make this appear in my browser?". While not impossible, the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
I feel as though the gap between the theoretical power of LLMs and what the average user knows of them and their capability have already widened so far that it’s irreconcilable.
This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.)
Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
well not very settled then is it
1. LLM thinking 2. RLHF 3. The latest frontier models
that does anything to change this fundamental "suggestibility" of LLMs.
But who knows, maybe I'm wrong.
The idea of an AI that one-shots a task presumes that the one requesting the task already knows what the task is, and no additional expertise.
Similarly, when you ask an AI "please summarize this text" it means you already know what summarization is as a concept.
They are either using some generic web frontend, ala chatgpt, some local app like claude desktop, or programming app like cursor.
Each of those will detect that you are "building an app" and will spit out code in one form or another. You have to try really hard and be very explicit that you want the output in some other format than code.
What most likely happens is a normie has no idea of how to build something that does nothing first - they just start describing the end state.
I'm going to try this with my wife later today. I bet she'd sort something out since she's been a manager forever and phases out instructions maddeningly
I often ask it export a single html file, for an external collaborator or simpler sharing. But I wouldn’t deploy that to production.
I wonder if they asked it to deploy a html file.
But this is exactly the kind of hidden domain knowledge / expertise that changes how you use the tool.
I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...
Welcome to Software Development, Lindsey from HR - here's your first database!
People just don't really understand how these things work yet, and they don't know what to ask for, I'm hopeful that they eventually do become more tech-literate, but not sure yet.
However, I personally agree with the author and I don't think his argument is necessarily motivated out of an anxious fear. On the contrary I think it may be motivated out of a sense of extreme exhilaration and empowerment.
Because experts (like myself as a programmer for 15+ years) who are using AI in many fields are suddenly empowered and much more useful than we were before AI. My employability and value has gone up and not down, precisely because of being able to apply my expertise with AI, which people without expertise simply cannot do. I am a professional programmer and also owner of my own startup.
Let me give you a concrete example that I am dealing with at my startup. I'm a small business owner. Before AI if i wanted to produce production quality video for marketing it would taken such a huge budget and such a large team of people (or an expensive agency) that I wouldn't even have considered it due to the enormous cost. I'm talking about Apple quality video production which takes millions of dollars to produce.
Not anymore. A single competent person with AI can replace an entire marketing video production department or agency. But expertise is key here: knowledge of film terminology to be able to describe the effect you want, and ability to use video editing tools effectively, as well aesthetic taste. I as a programmer with no filmmaking experience don't even know how to write the prompt which makes the video that i want because I don't even have the terminology. But a person with that expertise has suddenly become more employable and more valuable to my business because I as a small business now have the capability to create Apple quality marketing videos.
So AI actually created a new job for an expert that would have otherwise not existed because it was outside the budget of small businesses. Previously somebody like that would have been employable to only a few large production studios but now they become employable by almost any small business.
This is why it has the title (for me currently reading "What can I help with?" but this varies a lot) and the text box itself has the placeholder text "Ask anything". Sometimes I get big friendly suggestions about what to ask it, placed on screen near that text box.
> WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
I don't think it's constraints, per se: almost nobody looks at the font list and goes "oh no, too many options!"
Rather, GUIs are there to organise your options visually, group them in ways easy to intuitively get. There's a bit of fashion-induced rot here, e.g. I'm old enough to remember when it was always unambiguous when you were looking at a checkbox vs. a radio button, and now there's a blurry middle ground of collections of boxes with ticks in them that act mutually exclusive, but the point of a GUI from a UX POV is not the same as how software in general drifted as it got both more users and more developers and more opinionated managers and middle managers and designers who only cared about shiny rather than usability.
I want a green lawn and big bushy shrubs in front of my house. Do I need to know the intricate biology of my soil and habitat? No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
> Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests
There isn't a way around this: if you order the test and a value is abnormal, you are now committing yourself to treat a thing. We should not be ordering tests if we aren't ready to follow their results to the logical conclusion. So I would disagree that this is a problem in the way you've framed it.
We don’t use react, which Claude design seems to trend towards. We use Phoenix / liveview.
We have a shared design system, which keeps the visual elements in line. And then just prototype on design, collab, discuss and arrive at what we want to ship. And then engineering take over and rebuild via hand / claude code.
But the tools aren’t directly connected.
The value has been in the separation. In iterating on the prototype without impacting the codebase, dev cycle, etc. And solving problems/unknowns earlier.
There were always tools for this, but Claude design just feels more accessible and therefore gets used more immediately.
And the fidelity of the outcome (and the assumptions it’s forced the make) are more valuable and faster to achieve than Figma.
But again, VisiCalc is intentionally limited, it's not a all-powerful environment, on purpose. It's all about intentional limitations, making computation easier to reason about.
It's very useful that there's an intermediary that knows how that stuff works so I can build things without thinking about it in excruciating detail the whole time, I can dip down lower and learn stuff when it's relevant (like cache access and nested arrays) but I can also not do that in many situations.
There's also the argument that you can do engineering without understanding the underlying science as seen in th pyramids, the beautiful old european churches, etc
the generalists win overall, except of course for specific cases where specialists are great
I started using harnesses because they are good for when something breaks and it's not trivial to investigate so I'll have the agent tell me what's happening, then using that to produce my own change
E.g. you'll note my zebra example was not about whether one should additionally request a photo of the animal just to make sure it is indeed a rare animal. It was arguing that given existing differentiating information that points to an uncommon condition in the first place, one should not dismiss this on the basis that horses are still more common than zebras in the general case regardless. Under this uncommon information, the prior of thinking about horses should go out the window, and one should absolutely focus on zebras (at least as an additional differential). I assume you would also agree with this conclusion.
But of course, in real medicine things are not as simple as this contrived example. So the point I'm making is that, from what I have observed, there seems to be a bias towards decisions based on "prior" rather than "posterior" probabilities, even in the presence of additional differentiating information which should have prompted additional differentials to be considered. But this is different to saying one should constantly seek additional evidence to include or exclude additional differentials that are unlikely in the first place. That, I agree, would be a waste of time and resources (and potentially harmful for the patient).
Having said that, I somewhat disagree with the phrasing that we should not be ordering tests if we aren't ready to follow their results to the logical conclusion. This is a bit like saying, "I don't want to check for X because if I do and it checks out it will derail my current management plan"; but then this is putting the cart before the horse, since it's the findings that need to dictate management, not the other way round. I do think it is reasonable to say that one shouldn't be ordering a test if the expectation that it will change management is sufficiently low to justify the time/cost expended for it -- and where this expectation is a function of both the likelihood of the finding (given current information!), as well as its potential to change management. But this doesn't mean that if you do find an unrelated inconsequential incidental finding you are now required to divert all resources to it.
Conversely, if an incidental findings does prove worthy of treatment, then I don't see the problem with committing to treat it, as long as you don't lose sight of the original complaint either. Obviously that doesn't mean one should go looking for incidental findings willy-nilly though.
You should absolutely keep stretching to help with your injury, and there is decades of “sport science” to back that up.
[1] https://www.cochrane.org/evidence/CD004577_stretching-preven... - there’s another one about joint deformities which is far more interesting, but less relevant to this discussion
Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
We've all accepted that code-generation has been required and accepted for decades.
That’s been the way I do it.
I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.
And others want the green lawn and big bushy shrubs, but don't want to learn all of the surface level things to make their lawn thrive so they hire a service to do it for them. And there's nothing wrong with that - not everyone enjoys the yard work, but they still want the thriving lawn and shrubs... and they are happy to write a check to OpenAI... err...Lawn Doctor every month to get that result since the time they save by not dealing with their lawn, they can pursue things they do want to do.
Not everyone wants to (or needs to) learn every detail along the way of getting the results they want.
The one who learns about x86 registers is much more likely to continue to evolve
This reduces the cycle time on issues and reduces bugs and performance regressions. If I pitched that last sentence to your boss, chances are he'd take it.
What i described is what happened. I don’t appreciate the undertones where you are insinuating that i’m lying for whatever reason.
> The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas
You call it steering. I would call it falling into that grove. Probably tiny things in the initial message made the first response more likely to be a clarifying/ideating type. And once that happened the conversation was gaining momentum in that direction and neither participant was trying to guide it in a different one.
> the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Exactly. The LLM itself sidetracked her. They were just talking about cool features they could add, and at no point did she put down her feet and say “stop asking more questions and just write the code”.
It can be a combination of many things. Attitude (some people hate to be rude, and not answering a question feels a bit rude). It can also be that she enjoyed the process of unpacking and elaborating on the idea.
The meaning of the story is not that no lay person can possibly develop using AI. That would be silly, and untrue. I know clear counter examples. The point is that if you don’t know what you don’t know it is harder to steer the AI in the direction you could very easily with the right lingo.
I wanted the thing to be as quiet as possible at all times, and to always minimize the ways in which it can annoy.
Fans, as frequently implemented, are often annoying. They're often noisy all of the time because they lack controls.
Sometimes, they have dynamic controls that suck. Like a Juniper switch that runs the fans at a screaming 100% that you can't have a conversation near during its minutes-long boot process, and then quiets down to only a dull roar (but never any quieter).
Sometimes they suck in other ways: It starts quiet, but then it ramps up in audible quantized steps as cooling demands increase. The very definition of these steps is distracting.
Sometimes, they're just too responsive to be tolerated in the company of humans. The fan speed is always hunting and changing the pitch of the noise it produces, and sometimes this adjustment happens at a discernible and very regular interval that just makes it maddening to be around.
Sometimes, they're arguably even worse than any of that: The fan cycles on and off. It is a jet turbine, or it is silent. There is no in-between.
Sometimes, they're very clever in the worst ways: They ramp up instantly in response to dynamic events, and then slowly wind back down. That's awful in a power amplifier but it happens anyway, wherein: Every loud amplified sound is punctuated by the dying roar of a fan, even when the dynamic event was actually a short-lived nothingburger.
---
Anyway, this one is buttery-smooth. It responds to temperature on a curve. It has no perceptible steps as speeds increase or decrease. It's deliberately lazy and smoothed-out in its response; this amplifier has several pounds of aluminum heatsinks so instant response just isn't ever useful, and it's also least-annoying to deliberately eliminate rapid speed changes altogether.
The fan is a big Nidec screamer that I got from a late-century, grossly-overbuilt Dell Precision desktop. It can reliably spin so slowly that it's essentially silent (as is useful for an engineering workstation like that Dell), and it can also move enough air to fly itself right up and off of the bench.
The controller automatically finds the minimum viable speed for the installed fan, so it avoids being stalled at low duty cycles. In this way, some air is moving regardless of which make/model of 4-wire PC-style PWM fan is used down the road, making potential bush fixes more practical and functional. (It also does stall detection and recalibrates if that's ever necessary for some reason.)
And there's no lookup tables, because lookup tables imply steps and steps are bad. Besides, I'm not trying to save a million nickles on a million units here; shaving pennies isn't part of the program and a single $3 MCU board is cheap for my [qty. 1] application, so that what it gets. By extension, it has way more than enough grunt to get everything done and it just computes it all over and over again.
Like the output %, which gets rejiggered at a rate of 1KHz: I could have probably been happy with 100Hz. Or 10Hz. But the MCU is already chosen and it can do 1KHz just fine, so... that's what it do. It doesn't matter that it is inefficient; efficiency wasn't a goal. :)
And, because it's vibe coded: Of course this big rack-mount power amp from 1986 has a wifi-accessible web interface for its cooling system. It seemed like a pocket computer would be best way to provide a way to twist some of the cooling-related knobs when used in the field, since this is a functional prototype that sometimes gets used in literal fields. (It was dead simple to to get the bot to put that part together. The networking stuff might have been the easiest part.)
Sometime in the late 1990s, a Rice Rockets website appeared, making fun of people decorating their under-powered import econobox cars (not rockets in any sense) to look like racing cars with features like rear wings (on a front-wheel drive, lol), exhaust modifications ("fart cannons"), stripes, stickers, rims, steering wheels, etc.
If the cats decorating the desktop are Hello Kitty, she is really ricing it.
I vibe coded an iOS conference schedule app recently, built on top of my own rust UI framework. I started with claude design. I gave it the requirements, and showed it screenshots of other conference schedule apps I like which have features I want to use. I also gave it some visual references for how I want the app styled. It came up with some workable designs. They were a bit 'webby'. But, fine. The high level breakdown of UI screens and navigation between them was excellent.
Then I gave all the HTML files it produced to claude code, along with the documentation for my UI framework and told it to port the code to my UI framework. The first working version was rough. It copied a lot of the unintentional webby look and feel. It worked around missing features in my UI framework by rolling its own janky reimplementations of platform features. For example, instead of using UINavigationController, it rolled its own. It made its own (kinda bad) tab based navigation bar. The app didn't work properly in dark mode, because it was hard-coding a lot of colours. It took a bit of back and forth to fix all of this stuff. But I'm really happy with it now. It looks and feels great.
It's just a pity I couldn't share the app at the conference. Apple took a few days to approve the app in Testflight, and by the time they approved it, the conference was over.
I assume everyone else is playing with the same AI tools that I am, and getting similar results. But a lot of people I talk to seem to have no idea that this is possible right now. They're amazed when I show them my schedule app.
I think the etymology is from "rice burner" cars [0]:
> Riced out is an adjective denigrating a badly customized sports car, "usually with oversized or ill-matched exterior appointments".
So the barrier is still kinda there to just grab the instructions and follow them, its just incredibly easier to follow them.
Is this an actually serious question? Am I losing my mind here?
I didn't tell my HR friend about Lovable: she found it on her own. Of course someone's not going to Google for Lovable if they've never heard of it, but they might Google for "how do I build a website without knowing anything about it?", or ask ChatGPT the same question.
They're also, most likely, getting endless ads for AI services that help you build various kinds of software shoved into their faces all the time - these ads may not couch the value ad in exactly these terms, but that's fundamentally what they're advertising.
Not everybody is like this but there are plenty of people in the world who, when they have a problem, are quite motivated to find ways to solve it off their own backs.
It just is not believable or interesting. Even if it did happen, the reality is it just doesn't matter.
My HR friend found Lovable off her own bat. I imagine she probably Googled or asked ChatGPT something like, "How do I build a website to do BLAH without knowing anything about building websites?"
The point is people talk, they ask questions, they Google, they talk to ChatGPT, and if they have a problem to solve they're often quite motivated to find a solution to that problem off their own backs.
If someone asks me for advice on how to get something built then I'm going to recommend a tool that suits them and their situation, whatever that may be. In this specific situation, if they know nothing about building software, I'm certainly not going to sit them down and have them try to follow the most jank-ass way imaginable of building software with an LLM when I know much better tools exist that are built with people like them in mind.
Seriously, what is with the overly narrow assumptions in the replies I'm getting this morning? You're the third person who's tried to set this same fraying paper tiger on me. Can we all just wake up and think about the issues a bit more in the round, please?
It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
I think the main point is, if you're building frontend web apps, you should probably know how HTML, CSS, and JavaScript interact to some decently high level. Can you look stuff up on the margins via AI, or StackOverflow, or whatever? Sure.
If you're building <something else>, you should probably know the core tools and concepts at the appropriate level to build them.
No we do not need to know every aspect of CPU branch prediction and whatnot to make a webpage.
That's so far from being true it's like saying that humans are currently an interplanetary, post-scarcity species.
https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20cr
I count 500+ of them.
Little known fact maybe, but Google (also YouTube) search has been pretty great at this well before LLMs got really popular. I think it already started when they were changing from "keyword search" to "ask us a question search" but I'm not sure.
At some point I figured, if they want me to type a whole question, I might get even better answers if I ask the question like I was a complete idiot.
> https://www.google.com/search?q=pls+how+to+make+the+steam+pl...
> https://www.google.com/search?q=i+wnt+terminal+to+say+where+...
I don't think it actually gives better answers but it sure makes me grin every time
Yes, it is. I just Googled that exact question (and I promise I'm not trying to be obstinate when I say I'm Googling the exact phrase!) and it's automated AI response was to use WiX or Squarespace. Prompting it further with "What if I want it to do bespoke things that Squarespace can't do?" it responded with using Figma to design the website UI and then pass it along to either Framer or a "professional developer".
I do genuinely think that this is a discoverability issue. Of course, if you prompt it further with "Could I use AI to do this?" it dutifully responds that it can help with generating HTML, but that's three layers of difficulty to eventually get whatever default model Gemini has for signed-out Google searches to even suggest HTML.
LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report with the right jargon will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
I wish it was different, I actually enjoyed getting paid to learn, but I play with the cards I'm dealt.
The files you serve to the browser are static, not the contents of the page itself
Updating the dom can happen with only individual assets, so it’s a static site
It could also be that you've seen a lot of social media memes regarding "cope", observed their ability to provoke strong emotions, and confused this for meaningful insight. I see a lot of AI commentary these days that is clearly being spread for its virality rather than its truth value.
I started on this path literally about as soon as I could read thanks to the family having bought a Commodore 64 for my older siblings, but also perfect timing in that when I got to this age the sibling whose room it was in had just gone off to university.
Most people are not like this, in much the same way that they're not going to read the T&C end-to-end (another thing I've done) or learn enough law to actually understand what those words mean (a step too far even for me).
The vast majority of people (blanket statement, I know..) do not come from a culture where embracing curiosity, asking questions, or trying to break things down is the norm. Developers, tinkerers, etc., sure... you can reasonably make that assumption. But not everyone. A cultural practice of critical thinking and problem solving is HOW you KNOW to ask WHAT questions need to be answered FIRST, in order to solve a problem or progress toward a solution (if more information is needed).
You might even make the argument that everyone should have these skills, and I would agree with you. But the missing link here is a culture or cultural practice that provides those things (the WHY), and an AI/LLM will not provide those things in absentia, without "prompting", or build up that infrastructure in meatspace for a given set of users. Ignore this at your own risk.
Race-inspired to look cool, but not actually functional. Picture putting one of those giant fake air intakes on the hood of your car.
for example you can service static sites from S3 that have HTML/CSS/JS but no API or DB
I think it’s valuable enough to justify the price, and I want it to use the better model, as much as possible.
> This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise?
Learning from LLM written code is significantly and meaningfully different from struggling on your own or learning from more experienced human co-contributors. Especially in a large, complex, iteratively developed codebase.
Worse, once you're in that situation you are now at the juxtaposition of: "I did this. I understand what my reasoning was, and now I know why it is wrong and how to fix it" vs. "An LLM did this, I don't know why it did this, I'm not sure what it was trying to do or what pattern it followed and I'm not sure how to make it better because I didn't write or understand the original implementation either"
Is it impossible to learn and gain experience this way? Not at all. But it's definitely not equivalent.
You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
Honest curiosity about language and language use.
In fairness and full disclosure: I have believed until now that the language comes from rice-burners and related, and I have never liked it. If I were still in communities that used it (unix desktop crowd), I might proactively steer newcomers towards your acronym as a kind of reclaiming.
I've had people criticise me for having had the opportunity to learn in that way, as they did not.
> You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
No, why would I think that? I don't know them, the only thing I can say is in their favour: moving country to better your situation is difficult and them getting as far as they did is a demonstration of putting in a lot of effort of the exact type I praise by default.
Any negative connotations "rice burner" once had was lost when the term shifted towards referring to cars instead of humans. But now re-recognizing that the enhancements are inspired by the East Asian race turns the connotations back to humans. Isn't that a regression?
Reintroducing this to be something about a population's race reinstates the derogatoriness. You can be derogatory towards humans. Minimizing the cosmetic enhancements to be being inspired by the East Asian race and not valuable human achievement brings us right back to the same place we were when Japanese cars started being introduced into the North American market, diminishing the human contribution. It is a regression.