That wasn't my take - it was more broadly a 'given past statements and accompanying behaviour, we can't trust future statements to align with future behaviour'.

Can you more explicitly describe what the X and Y points you allude to are?

Gary Marcus has been writing a piece like this roughly every 2 months for 20 years, and he gets a lot of attention because he seems respectable (he is not), and many papers are looking for a respectable source that has this opinion.

The crux of the article was altman trying to get a bailout, and when people called him on his bullshit, lying thay he never wanted a bailout. You cant trust the man

Every “6 months ago I said this!” added little to the article. In a real investigative report it would lay out the facts and fairly objectively state “6 months ago, Sam Altman was accused of …”

The article as a whole just seems libelous? Almost personal?

Yeah that and the rush to IPO OpenAI....

Why comment then?

Gary is a bit of a stopped clock that always says the same time and is right occasionally. His basic position is these neural network things are not much good, they won't do whatever the current claim is.

He’s playing the Nouriel Roubini game (another NYU adj prof).

Last time I checked Nouriel was partying up in the Hamptons so being a permabear is lucrative.

Nicely done! An ad hominem attack, the lowest form of discourse.

He might be fun to play a board game with but not in real life.

Spot on!

The "I don't need to answer your simple questions on profit and loss statements" sentiment was odd. Likely odd enough to spur institutional investors to dig deeper or attract short interest.

This site(entropytown) has a certificate problem: https://a.dropoverapp.com/cloud/download/c3bc66a4-40a3-49fe-...

The VCs and bloggers have also shifted tone and are hedging to save themselves from embarrassment. It used to be “AGI is just around the corner” and now it’s “Just because we haven’t figured it out yet doesn’t mean we never will” (Andreesen) or “Actually bubbles are good” (Stratechery).

Gary called it a long time ago but I think the Dwarkesh and Karpathy podcast is when the shift started.

Care to enlighten those of us that know nothing about Gary's track record?

You could just read the primary sources. The things hes showing people to have said.

The gist is altman was trying to be slick about requiesting a government bailout, got called out on his bullshit, and then decided to lie and say he never wanted a bailout

The implication being the bubble is getting closer and closer to popping, since even altman is thinking about how to survive the pop

Why is he not respectable?

It looks like the actual crux was Altman's plea that the backtops be granted... Not to OpenAI? The document linkedfrom the article that had the actual ask, rather than cleanup over deliberate misatributions by others, was to... "Extend eligibility", for the AMIC money already carved out, to companies that are producers of things such as "grid components such as transformers and specialized steel".

So: Altman did not ask for OpenAI loans to be guaranteed, nor did the CFO. It was on behalf of others drawn into the needs of the industry the AMIC grant was supposed to support. Self interested by OpenAI? Sure! And also not about to make the top 10k leaderboard for "sleezy things companies do".

I think you must mean Sarah Friar, CFO of OpenAI.

You can criticize OpenAI for many things but this was blatantly misquoted.

They were talking about backstops for chips, to get fabs constructed in the US. If anyone else said this, it would have been considered a great idea, both republicans and democrats talked about the same thing to get TSMC production into the US, but everyone pretended that OpenAI was talking about data centers in this context. They weren't/

I'm with you on that - people use the wrong terms. Bailouts are supporting things like GM or failing banks because the government is worried about GM workers losing jobs or bank depositors losing money.

Altman's 1.4T isn't like that - it's a proposed new investment in stuff that doesn't exist yet and there would be no job losses or the like if it fails to exist. They have been talking about potential government support for the new ventures, partly to keep up with China which uses similar government support. I'm not sure if it's a good idea but it would not be a bailout, more a subsidy.

This is the same kind of bullshit rationalization they used to say that the bank bailouts of 2009 weren't really bank bailouts.

They were bank bailouts.

Unsecured government loans are either bailouts, entitlements in disguise, or (usually misguided) attempts at broad economic stimulus. This definitely isn't either of the latter two.

Hilarious how there's vigorous discussion happening under a post with 200 votes and 100 comments, while it's been flagged the whole time.

> No substance, no skill, just wheeling dealing

That's their only real skill. There's an entire class of people who specialize in exploiting society's hard work.

All of Sam’s shenanigans go back to that one popular post on here many months ago about how AI has no moat. LLMs are a commodity, Chinese companies are just releasing them for free. Sam has gone all in on OpenAI and needs to secure his company, he knows it could be another decade until they discover an innovation on par with tranformers and there’s absolutely no way they can go from burning tens of billions of dollars a year to profitable by selling a rapidly commoditized technology.

It's not a report, it's a blog post. Of course it's personal.

The Ben Thompson take attempting to find a lemonade in the lemons is the most terrifying: the bubble will pop and leave us with… excess electric generation capacity? (And bankrupt power producers?)

We truly live in the dumbest timeline.

Unfortunately the shit flinging is recursive. I’m not looking to read more conjecture. And people are pulling an Elon v2 with Altman and the “any day now he’ll make a fatal mistake!” nonsense.

I’m not holding my breath for hot takes, but I got what I came for: sama said some stuff, the thread.

Something to note about this craze and Crypto is that the world is full of bozos who do not think deeply whatsoever.

He posts some variant of this kind of anti AI piece roughly every two months for over 20 years. He's been wrong so far. Eventually he'll get lucky, but his track record is abysmal.

Somehow people like him seem to survive just fine. Adam Neumann still has infinite money and Billy McFarland is still getting investors for various schemes.

Gary Marcus has a very particular view of human psychology that was popular for a while, especially in the early 2000s. Major proponents include Steven Pinker, Noam Chomsky, Jerry Fodor. This view was heavily influenced by symbolic computers when symbolic computers were new and so held some mental share leading into the dotcom boom. For several reasons, one of which is the replication crisis, this view is no longer nearly as popular.

One of the major beliefs of this view is that LLMs are essentially impossible because there's not enough information in language to learn it unless you have a special purpose language-learning module built into the brain by evolution. This is Chomsky's "poverty of the stimulus" argument.

Marcus still defends this view and because of this bias is committed to trying to prove to everyone that LLMs are not possible or at least that they're some kind of illusion. There's a sense in which they threaten his fundamental concept of how the brain works.

In proposing and defending these views he appears to me and others to be having a sort of internal crisis that's playing out publicly where his need to be right about this is coloring his judgment and objectivity.

You cant be this naive. You have to read between the lines. Altman has laready shown hes willing to boil us frogs with his attempts IPO a non profit. Hes slimy, he doesnt just say things and stand by them, he tests the waters and lies about ever wanting to swim if its too cold.

He called for general bailouts, which would benefit OAI far far more than others since its spending the most, and then he got backlash, he lied and said that OAI doesnt want bailouts at all.

You cant be so literalist to just take the man at his exact word and not try to determine his motivations. Especially this guy, whos made a career out of dancing around silicon valley's "no assholes" policy

My goodness is there a huge difference between TSMC and OpenAI. TSMC moving to the US is a benefit to all US fabless companies and fabless companies that want to do business in any sensitive US industry, and they have an incredible track record as an industry leader. Open AI's chip manufacturing is much more risky and much less payoff. I wouldn't want the government to pay a penny towards it. Especially because an estimated cost for a leading edge chip is maybe $500 million. If they can commit to $1.4T, they can do it on their own. It's almost an omen they don't feel comfortable spending that without a guarantor.

Those are typically structured as loans no?

Most likely. Who better to know their financial status is fried than the CFO

Unsecured government loan to a successful company to fund acceleration and growth is a "handout".

A "bailout" is what happened in 2009, in the sense the banks would literally have collapsed without it (and they probably should have).

OpenAI is not going to collapse without these loans. Huge difference.

Also for the record, not rationalizing, because I'm not in favor of either handouts or bailouts.

I think one thing I liked when I was in start-ups was working for folks straight out of grad school that genuinely cared about a problem, had some advanced knowledge about a topic and were just trying to turn it into a business.

I felt like those were the kind of folks I could get on board with; not someone with an MBA that needs head count to reach their KPIs. Someone that cared about solving a problem and was genuinely smart.

I think every company would like to have that. OpenAI is in a hyped up position that it can get it, so they go for it.

I don't buy that they can create AGI by investing trillions in training models and infrastructure.

If you ask me, this is just more money spent on pollution.

The need to replace humans to be profitable, just sounds like the end goal is to destroy the planet with datacenters and hurt people generally.

Sounds like a net negative for the planet.

Cause he hasn't built anything or found a job since he sold his last company ages ago.

The CFO literally asked for guaranteed loans, tell me that’s not what she meant, don’t tell me not to believe my eyes and ears. Not only did I hear it, but David “AI Czar, king of the all-in bros” Sacks heard it the same way along with everyone else who heard it.

Believe it or not, that's actually a walkback from the $5 to $7 trillion he was pushing for almost two years ago.

https://arstechnica.com/information-technology/2024/02/repor...

LLMs are a commodity but ChatGpt is a brand, for most people AI means ChatGpt. They are not going to use Chinese models.

ChatGpt is also building other products/brands like Sora to capture more mindshare.

Linux is free and yet people use Windows.

You regularly make personal attacks against individuals on the internet?

Maybe I’m just old, but I don’t see the appeal?

If you’re trying to convince people, then you should probably have a convincing argument. Otherwise it feels like kiwifarms-posting with a megaphone.

The need to say “fuck you” to Elon and one-up him.

Here's the juicy stuff mentioned in the article: https://storage.courtlistener.com/recap/gov.uscourts.cand.43...

You buy puts on TQQQ.

> what prompted OpenAI recently to need this 1.4T investment?

Capital denial to competitors.

Altman definitely has enough wealth of his own to survive no matter what. But if you look at how he operates, it is pretty clear that what he really wants is power, and he is usually extremely good at getting it. But OpenAI is rapidly losing their leadership in the space, and it turns out that their product has no moat. Meanwhile, new Chinese open source SOTA models just keep coming on an almost daily basis.

I guess survive is the wrong word. Sonce thats not what hes trying to do. Its more like he wants to figure how not to lose his dragons horde of gold, and even grow it in the pop

Chomsky and Pinker (I don't Fodor), are entertainers and their theories are good only for PhD pumpers. How human brain works and the associated neuroscience has absolutely nothing to do with LLMs which simply are empirically defined function approximations.

The problem with Sam Altman is that he is a shyster and leech feeding off the hard work of thousands of programmers and engineers.

They are going all in, promising so many deals to so many companies that if OpenAI fails, the entire US economy will explode. Expecting the government will bail them out to prevent such a disaster.

Absolutely nothing. They don't, they didn't, it's a poorly stitched together attack job.

The 1.4 T amounts to a broad nearly decade long capex plan, not liabilities.

The loans and backstops etc were a request, not for OpenAI, but on behalf of manufacturers of grid equipment, manufacturers that OpenAI wouldike the government to consider as eligible for money already carved out by the AMIC national investment in chips and AI, and also probably more money as well-- it's a separate group of tangential industries that weren't initially considered, so why not ask? Sure it would help keep the wheels moving in OpenAI and the broad AI and US semiconductor industry, but it's far away from and ask by Altman for a bailout of his company.

I’d also add these lessons I’ve personally learned:

- Greed is blinding even to intelligent people, especially greedy people in groups

- Society is incredibly vulnerable to lying, we mostly rely on the disincentive that people usually feel bad about it, but the ones who don’t can get away with anything.

- There’s really only a subtle difference between many successful startups and Ponzi schemes. Sam Altman’s grift is only one level more sophisticated than SBF or Elizabeth Holmes

So many goons on here who keep flagging submissions like these, or trying to discredit them with superficial criticism ("he keeps saying the same thing!"). It's sad this behavior doesn't get modded.

Edit: lol at the downvotes. You're just proving my point, goons.

Has he been wrong about everything? I doubt that. It's also a logical fallacy to dismiss his current argument because he's made wrong arguments before. What he's saying about Altman lying is true or false independent of anything else he's said prior.

Yep; it will just screw the employees, the investors (or at least those that didn't suddenly get clairvoyant and find an exit right before it hits the fan), and many, many apps relying on their APIs.

> LLMs are essentially impossible

> trying to prove to everyone that LLMs are not possible or at least that they're some kind of illusion

This is such poor phrasing I can't help but wonder if it was intentional. The argument is over whether LLMs are capable of AGI, not whether "LLMs are possible".

You also 100% do not have to buy into Chomsky's theory of the brain to believe LLMs won't achieve AGI.

> LLMs are essentially impossible

This is the worst misrepresentation of Marcus' argument I've seen so far.

> One of the major beliefs of this view is that LLMs are essentially impossible because there's not enough information in language to learn it unless you have a special purpose language-learning module built into the brain by evolution. This is Chomsky's "poverty of the stimulus" argument

The argument is that there is not enough information available to a child to do this. So even if we grant the dubious premise that LLMs have learned to speak languages in a manner analogous to humans, they are not a counterexample to Chomsky’s poverty of the stimulus argument because they have been trained on a vast array of linguistic data that is not available within a single human childhood.

If you want to better understand Chomsky’s position, it’s easiest to do so in relation to other animals. Why are other intelligent animals not able to learn human languages? The rather unsurprising answer, in Chomsky’s view, is that humans have a built-in linguistic capacity, rather in the way that e.g. bats have a built in capacity for echolocation. The claim that bats have a built-in capacity for echolocation is not refuted by the existence of sonar. Likewise, our ability to construct machines that mimic some aspects of human linguistic capacity does not automatically refute the hypothesis that this is a specific human capacity absent in other animals.

Imagine if sonar engineers were constantly shitting on chiropterologists because their silly theory of bats having evolved a capacity for echolocation has now been refuted by human-constructed sonar arrays. The argument makes so little sense that it’s difficult to even imagine the scenario. But the argument against Chomsky from LLMs doesn’t really make any more sense, on reflection.

Chomsky hasn’t helped his case in recent years by tacking his name on some dumb articles about LLMs that he didn’t actually write. (A warning to us all that retirement is a good idea.) So I don’t blame people who are excited about LLMs for seeing him as a bit of rube, but the supposed conflict between Chomsky and LLMs is entirely artificial. Chomsky is (was) trying to do cognitive science. People experimenting with LLMs are not, on the other hand, making any serious effort to study how humans acquire language, and so have very little of substance to argue with Chomsky about. They are just taking opportunistic pot shots at a Big Name because it’s a good way to get attention.

For the record, Chomsky himself has never made any very specific claims about a dedicated module in the brain or about the evolutionary origins of human linguistic capacity (except for some skeptical comments on it being a gradual adaptation).

So upon finding success, he has escaped the struggle that people are forced into against their will. How unreasonable of him.

She said "guaranteed loans" and everybody heard "bailouts" and that's what everybody is reacting to.

Guaranteed loans are a completedly different thing and are still debatable, as I expand here -- https://news.ycombinator.com/item?id=45839060 -- but that's not the debate I'm seeing anywhere.

All this transformer tech is a commodity, no one will care about the brand if an alternative is free so it’s a race to the bottom.

Yes people use Windows. Go look up the history of how that came to be, it had nothing to do with their brand. Sam is looking for his IBM.

I remember and it lends some credence to the notion that Altman is just making up big numbers.

ChatGPT has literally no brand moat. Nobody recognizes, or even cares about, the brand. They care about the interface - chat.

If you just take those chinese models and slap them on some decent looking website, nobody will know the difference. People take brand recognition far too seriously.

If Linux was actually indistinguishable from Windows, you bet your ass nobody in their right mind would install Windows. But they're actually different things.

> LLMs are a commodity but ChatGpt is a brand, for most people AI means ChatGpt. They are not going to use Chinese models.

This is still an unknown.

> Linux is free and yet people use Windows.

The moat isn't even 1% of the one that Windows has - and even Windows has been losing ground on the consumer side. While slow, the rate acceleration is definitely there. Compare Oct 2020-Oct 2025 vs the preceding 5-year period.

I don’t buy this argument about brand, in tech history has showed customers will switch overnight if a better option appears and there is no network effects or ecosystem lockin. BlackBerry had a brand too.

Well, sama managed to convince a lot of people to give his company billions, is making apocalyptic predictions that some CEOs take seriously etc. Making sure people at large realize the guy has a very loose relationship with truth, for many years, seems like public service. It's only libel if you spread false statements which Marcus is careful not to make.

Maybe you’re missing the context of who Gary Marcus is

Respect can be earned through many means, not just through goalposts set by financiers.

He was just anchoring.

https://www.pon.harvard.edu/daily/negotiation-skills-daily/w...

An operating system is a lot stickier than a website URL.

Google has stayed on top for 25 years because they're better and free. LLM providers will have to compete on price doing expensive inference.

Netscape was fairly known by the public during the .com bubble, it's now a distant memory at most.

A blog is an opinion piece. The subject of interest, Sam Altman, is a public figure and CEO of one of the fastest growing tech companies. He's testified in front of Congress on AI regulation and has a lot of pull and influence on regulators. Some of the things and actions he's taken in the past are controversial, thus, thinkpieces get written. The AI industry is quickly en route to trillion dollar plus territory (already there if you count Nvidia as an AI company). There's a lot of money and emotions at stake for the AI gold rush. When someone is at the forefront of these types of things, like other public business figures with controversial tactics (Musk, Gates, Jobs, Kalanick, etc) it draws attention.

> You regularly make personal attacks against individuals on the internet?

Isn't that basically the history of the internet?

Billionaires and corporations too big too fail are bad for democracy. They have an oversized impact on society and the government.

I pity them. They are not as far ahead in life and understanding as they want to be or are supremely deluded.

I just wished the bozos actually replied to the post instead of hiding behind a button.

point here is that his signal/noise is very poor, and the reason why people upvote him is because they agree with the headlines

- Thinking the present is somehow special and unlike similar past events or trends.

Narrator: it was in fact a bailout they were asking for.

That would do nothing to prevent such a disaster. It would guarantee it.

The correction is good in the long run. It will settle to sane understanding of AI's capabilities, competent applications, and the unsuing AI winter will be when real research is done rather than "we fed its garbage back into itself hoping something else came out"

It will ruin some lives. I hate that the american economy runs on these speculative cycles. If we tempered our expectations to what was sane, but still bullish, we'd have far less to fall

We weren't talking about AGI. It sounds like that's a topic you are interested in. It's a fine topic just not the one we're discussing.

And my phrasing was wonderful and perfect.

Nothing in my comment is a misrepresentation. It may help to look up some of the words I mention in the comment if you're unsure what they mean.

Also Gary Marcus doesn't make an "argument" in the singular. He's a researcher with decades of public work and private work.

There was a large literature on language acquisition prior to the invention of LLMs that showed that Chomsky's argument likely wasn't correct. This is in addition to the fact that he significantly underestimated the amount of linguistic input children receive.

There's too much to hash it out here in HN. You can try to save the LAD argument by a strategic retreat, but it's been in retreat for decades now and keeps losing ground. It's clear that neural networks can learn the rules of grammar without specifically baking grammatical hierarchies into the network. You can retreat to saying it's about setting hyper parameters or priors but even the evidence for that is marginal.

There are certainly features of the brain that make language learning easier (such as size) but POS doesn't really provide anything to guide research and is mostly of historical interest now. It's a claim that something is impossible, which is a strong claim. And the evidence for it is poor. It's not clear it would have any adherents if it were proposed anew today. And this is all before LLMs enter the picture.

The research from neuroscience and learning theory and machine learning etc have all pointed toward a view of the brain as significantly different from the psychological nativism view. When many prominent results in the nativist camp failed to replicate during the replicability crisis, most big name researchers pivoted to other fields. Marcus is one of the remaining vocal holdouts for nativism. And his beliefs about AI align very closely with all the old debates about statistical learning models vs symbolic manipulation etc.

> Why are other intelligent animals not able to learn human languages?

Animals and plants do communicate with each other in structured ways. Animals can learn to communicate with humans. This is one of those areas where you can choose to try to see the continuities with communication or you can try to define a vision of language that isolates human language as completely set apart. I think human language is more like an outlier in complexity to the communication animals do rather than a fundamentally different thing. In that sense there's not much of a mystery given brain size, number of neurons, sociality etc.

> The argument is that there is not enough information available to a child to do this

Yes, but children are the humans who earn language in the typical case. So you can replace "child" with "human" especially with all the hedging I did in my first post (e.g. "essentially"). As I said above Chomsky is known to have underestimated the amount of input babies receive. Babies hear language from the moment they're born until they learn to speak. Also, as a parent, I often correct grammatical and other mistakes as toddlers learn to talk. Other parents do the same. Part of the POS is based on the premise that children don't get their grammar corrected often.

Gary Marcus is not changing the story financiers are telling each other about AI. He has been telling the same story without making a dent in their stories. And that is because he is not in the arena. What he says, does and thinks doesn't matter.

Google stayed on the top because of business deals with Apple and Firefox and then they got the biggest marketing tool called Chrome.

Chatgpt can become an Ad company just like Google probably bigger than Google.

Absolutely. It's anchoring in the service of a massive con.

If a foundation model provider allows fine-tuning on confidential data but the result is locked into the platform, wouldn't that be extremely sticky?

Yes because MS gave away IE for free, why do you think all AI providers including ChatGpt give AI access for free for limited cases?

Fastest growing charities. Oh wait, no he weaseled out of that.

Well, you know, unless you're a better class of person, then every single thing you post is a backhanded personal attack on the reader.

I don’t think that’s the case. The relatively small cabal of people driving this stuff are all pretty weird people who live atypical lives and are hella smart, aware, and unconstrained.

They don’t care about the wake of destruction, and likely believe that it is virtuous to bring it on.

That's not really true. It may be legit strategy. If they realize they overhyped the whole industry, and everyone is tied to them, and they have strong political allies… if you also realize it's going to crumble. You might just see bailout as only viable option so you make yourself even bigger to fail.

If it turns like this it's extremely cynical and one would wonder how could Altman stay ou of prison (i am sure he will not go to prison).

Have to agree. The market needs a correction. And history for the states definitely seems to encourage it, if not flat out reward it.

Your reasoning seems clear enough to me. cmiiw, you’re saying Marcus says LLMs don’t really understand language and only present an illusion of that understanding. And that illusion will noticeably break at a certain scale. And to be honest when context windows get filled up to a certain point, they do become unintelligible and stupid.

In spite of this, I think LLMs display intelligence and for me that is more useful than their understanding of language. I haven’t read anything from Chomsky tbh.

The utility of LLMs come from their intelligence and the price point at which it is achieved. Ideally the discussion should focus on that. The deeper discussion of AGI should not worry the policy makers or the general public. But unfortunately, business seems intent on diving into the philosophical arguments of how to achieve AGI because that is the logic they have chosen to convince people into giving them more and more capital. And that is what makes Marcus’ and his friends’ critique relevant.

One can’t really critique people like Marcus saying he is being academic and pedantic on LLM capabilities, are they real, are they not etc when the money is relentlessly chasing those un-achieved capabilities.

So even though you’re saying we aren’t talking about AGI and this isn’t the topic, everything kind of circles back into AGI and the amount of money being poured into chasing that.

What is it that happened roughly 2 months ago? It seems like a disproportionate number of all the low-effort comments on the anti-ai side of things are coming from accounts that age.

But you need to agree that we can single out one specific argument he is most famous for, right? And when Josh Wolfe is saying that "Apple Gary Marcus'd LLM's reasoning ability", it does refer to a very specific issue of whether LLMs can lead to AGI or not.

I don't know whether he's right or wrong, but deep down I feel the amount of money funneled in the Transformed architecture might have blocked other approaches, both potentially promising and others doomed for failure, just because LLMs' quick wins.

> And my phrasing was wonderful and perfect.

Do you use LLMs to post on HN? I'm asking seriously.

Yes, lots of people have argued that Chomsky is wrong about various things for various reasons and at various times. The point of my post was not to get into all of those historical arguments, but to point out that recent developments in LLMs are largely irrelevant. But I'll briefly respond to some of your broader points.

You mention 'neural networks' learning rules of grammar. Again, this is relevant to Chomsky's argument only to the extent that such devices do so on the basis of the kind of data available to a child. Here you implicitly reference a body of research that's largely non-existent. Where are the papers showing that neural networks can learn, say, ECP effects, ACD, restrictions on possible scope interpretations, etc. etc., on the basis of a realistic child linguistic corpus?

Your 'continuities' argument cuts both ways. There are continuities between human perception and bat perception and between bat communication and human communication; but we still can't echolocate, and bats still can't hold conversations. The specifics matter here. Is bat echolocation just a more complex variant of my very slight ability to sense whether I'm in an enclosed location or an outdoor space when I have my eyes closed? And is the explanation for why bats but not humans have this ability that bat cognition is just more sophisticated than human cognition? I'm sure neural networks can be trained to do echolocation too. Humans can train an artificial network to do echolocation, therefore it can't be a species-specific capacity of bats. << This seems like a terrible argument, no?

Poverty of the stimulus arguments don't really depend at all on the assumption that parents don't correct children, or that children ignore such corrections. If you look at specific examples of the kind of grammatical rules that tend to interest generative linguists (e.g. ACD, ECP effects, ...) then parents don't even know about any of these, and certainly aren't correcting their children on them.

Chomsky has never made any specific estimate of the 'amount' of input that babies receive, so he certainly can't be known to have underestimated it. Poverty of the stimulus arguments are at heart not quantitative but rather are based on the assumption that certain specific kinds of data are not likely to be available in a child's input. This assumption has been validated by experimental and corpus studies (e.g. https://sites.socsci.uci.edu/~lpearl/courses/readings/LidzWa...)

> Babies hear language from the moment they're born until they learn to speak

I can assure you that this insight is not lost on anyone who works on child language acquisition :)

I turned 11, and my mom let me get a hacker news account!

Nope.

I would appreciate if you and the GP not personally insult me when you have a question though. You may feel that you know Marcus to be into one particular thing but some of us have been familiar with his work long before he pivoted to AI.

A realistic child linguistic corpus for a 2 year old starting to form sentences would be about 15 million words over the course of their lifetime. Converted to LM units that's maybe about 20 million tokens. There are small language models trained on sets that small.

Some LMs are specifically trained on child-focused small corpora in the 10 million range, e.g. BabyLM: https://babylm.github.io.

Keep in mind that before age 2, children are using individual words and getting much richer feedback than LMs are.

Humans can and do echolocate: https://en.wikipedia.org/wiki/Human_echolocation. There are also anatomical differences that are not cognitive that affect the abilities like echolocation. For example, the positioning and frequency response of sensors (e.g. ears) can affect echolocation performance.

There are two problems with your post.

The first is that your phrasing "that LLMs are not possible or at least that they're some kind of illusion" collapses the claim being made to the point where it looks as if you're saying Marcus believes people are just deluded that something called a "LLM" exists in the world. But even allowing for some inference as to what you actually meant, it remains ambiguous whether you are talking about language acquisition (which you are in your 2nd paragraph) or the genuine understanding and reasoning / robust world model induction necessary for AGI, which is the focus of Marcus' recent discussion on LLMs, and why we're even talking about Marcus here in the first place.

You seem more familiar with Marcus' thinking on language acquisition than I, so I can only assume that his thinking on language acquisition and LLMs is somewhat related to his thinking on understanding and reasoning / world model induction and LLMs. But it doesn't appear to me, based on what I've read of Marcus, that his claims about the latter really depend on Chomsky. Which brings me to the 2nd problem with your post, where you make the uncharitable claim that "he appears to me and others to be having a sort of internal crisis that's playing out publicly", as if it were simply impossible to believe that LLMs are not capable of genuine understanding / robust world model induction otherwise.

I'm sorry, I didn't mean to insult you. To explain the reason: you seem to use some particular wordings that just seem strange to me, such as first saying that Marcus position is that "LLMs are impossible" which is either false or incredibly imprecise shortcut for "AGI using LLMs is impossible", and then claiming it was beautiful.

I didn't mean to attack you personally and I'm really sorry if it sounded this way. I appreciate the generally positive atmosphere on HN and I believe it more important than the actual argument, whatever it may be.

Yes, humans can echolocate to a limited extent, just as some animals have very limited analogs of human language. That was the point of the comparison. It is no more sensible to view human language as just a more complex variant of vervet monkey calls than it is to view bat echolocation as just a more complex variant of whatever limited capacity humans have in that area. There is continuity viewed from the outside, if you squint a little, but that's unlikely to correspond to continuity in terms of the underlying cognitive mechanisms. Bats, for example, can make precise calculations of distance based on a built-in reference for the speed of sound: https://www.pnas.org/doi/10.1073/pnas.2024352118

Children don't get 'rich feedback' at all on the grammatical structure of their sentences. I think this idea is probably based on a misconception of what 'grammar' is from a generative linguistics perspective. When was the last time that a child got rich feedback on their misinterpretation of an ACD construction? https://www.bu.edu/bucld/files/2011/05/29-SyrettBUCLD2004.pd...

LLMs trained on small datasets don't perform that well from the point of view of language acquisition – even up to 100 million tokens. There's not a very large literature on this because, as I said, there are many more people interested in making a drive-by critique of generative linguistics than there are people who are genuinely interested in investigating different models of child language acquisition. But here is one suggestive result: https://aclanthology.org/2025.emnlp-main.761.pdf See also the last paragraph of p.6 onwards of https://arxiv.org/pdf/2308.03228

The other point that's often missed in evaluations of these models is their capacity for learning completely non-human-like languages. Thus, the BabyLM models have some limited success in learning (for example) some island constraints, but could just have easily acquired languages without island constraints. That then leaves the question of why we do not see human languages without such constraints.

>Children don't get 'rich feedback' at all on the grammatical structure of their sentences.

They probably do get parents and the like correcting them or giving an example. Kid says we goed fish, adult say yeah we went fishing. I taught English as a foreign language a bit and people learn almost entirely from examples like that rather than talking about ellipsis or any sort of grammar jargon.

It seems brains / neurons / LLMs are good at pattern recognition. Brains probably quicker on the uptake than LLM backpropagation though.

That particular example is irrelevant to poverty of the stimulus arguments because no-one has ever suggested that kids acquiring English lack evidence for the irregular past tense of ‘go’.

See above for some examples of the kinds of grammatical principles that can form the basis of a poverty of the stimulus argument. They’re not generally the kind of thing that parental corrections could conceivably help with, for two reasons:

1) (The main reason) Poverty of the stimulus arguments relate to features of grammatical constructions that are rarely exemplified. As examples are rarely uttered, deviant instances are rarely corrected, even assuming the presence of superlatively wise and attentive caregivers.

2) (The reason that you mention) Explicit instruction on grammatical rules has almost no effect on most people, especially young children. So corrections at most add a few more examples of bad sentences to the child’s dataset, which they can probably obtain anyway via more indirect cues.

If corrections were really effective, someone should be able to do a killer experiment where they show an improved (i.e. more adult-like) handling of, say, quantifier scope in four year olds after giving them lots of relevant corrections. I am open minded about the outcome of such an experiment, but I’d bet a fairly large amount of money that it would go nowhere.