Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.