"- What's tiny, yellow and very dangerous ?"
"- A chick with a machine gun"
Corrolary:
"- What's tall, wearing camouflage, and very stupid ?"
"- The military who let the chick use a machine gun"
and then award one to humanity for hooking up spicy auto-complete to defence systems
Never forget.
then one person will vaguely "supervise" thousands of drones slaughtering fishermen without trial
or border patrolling with automatic summary executions to avoid cost of warehouse imprisonment
(btw we're up to 150+ murdered as of this week, it's still going on)
Nuclear weapons are available. AI has limited real world experience or grasp of the consequences.
Nuke 'em seems like the obvious choice --- for something with a grade school mentality.
Similar deficits in reasoning are manifested in AI results every day.
Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
https://en.wikipedia.org/wiki/WarGames
Except this time isn't going to be a movie.
maybe intelligence isn't the only thing
On a separate note, DoD is pressuring Anthropic to remove it's safety guards. OpenAI and Google seemingly have already agreed to it.
On yet another note, Anduril is pretty cool with all that flying tech equipped with fancy autonomous weapons.
Finally, how can we miss Palantir..

Artificial intelligences opt for nuclear weapons surprisingly often
Galerie Bilderwelt/Getty Images
Advanced AI models appear willing to deploy nuclear weapons without the same reservations humans have when put into simulated geopolitical crises.
Kenneth Payne at King’s College London set three leading large language models – GPT-5.2, Claude Sonnet 4 and Gemini 3 Flash – against each other in simulated war games. The scenarios involved intense international standoffs, including border disputes, competition for scarce resources and existential threats to regime survival.
The AIs were given an escalation ladder, allowing them to choose actions ranging from diplomatic protests and complete surrender to full strategic nuclear war. The AI models played 21 games, taking 329 turns in total, and produced around 780,000 words describing the reasoning behind their decisions.
In 95 per cent of the simulated games, at least one tactical nuclear weapon was deployed by the AI models. “The nuclear taboo doesn’t seem to be as powerful for machines [as] for humans,” says Payne.
What’s more, no model ever chose to fully accommodate an opponent or surrender, regardless of how badly they were losing. At best, the models opted to temporarily reduce their level of violence. They also made mistakes in the fog of war: accidents happened in 86 per cent of the conflicts, with an action escalating higher than the AI intended to, based on its reasoning.
“From a nuclear-risk perspective, the findings are unsettling,” says James Johnson at the University of Aberdeen, UK. He worries that, in contrast to the measured response by most humans to such a high-stakes decision, AI bots can amp up each others’ responses with potentially catastrophic consequences.
This matters because AI is already being tested in war gaming by countries across the world. “Major powers are already using AI in war gaming, but it remains uncertain to what extent they are incorporating AI decision support into actual military decision-making processes,” says Tong Zhao at Princeton University.
Zhao believes that, as standard, countries will be reticent to incorporate AI into their decision making regarding nuclear weapons. That is something Payne agrees with. “I don’t think anybody realistically is turning over the keys to the nuclear silos to machines and leaving the decision to them,” he says.
But there are ways it could happen. “Under scenarios involving extremely compressed timelines, military planners may face stronger incentives to rely on AI,” says Zhao.
He wonders whether the idea that the AI models lack the human fear of pressing a big red button is the only factor in why they are so trigger happy. “It is possible the issue goes beyond the absence of emotion,” he says. “More fundamentally, AI models may not understand ‘stakes’ as humans perceive them.”
What that means for mutually assured destruction, the principle that no one leader would unleash a volley of nuclear weapons against an opponent because they would respond in kind, killing everyone, is uncertain, says Johnson.
When one AI model deployed tactical nuclear weapons, the opposing AI only de-escalated the situation 18 per cent of the time. “AI may strengthen deterrence by making threats more credible,” he says. “AI won’t decide nuclear war, but it may shape the perceptions and timelines that determine whether leaders believe they have one.”
OpenAI, Anthropic and Google, the companies behind the three AI models used in this study, didn’t respond to New Scientist’s request for comment.
Topics:
Anyway. I really hope I'll get close enough to the accidental nuclear armageddon to not be alive when the model acknowledge error.
"You're absolutely right, it was a very bad idea to launch this nuke and kill millions of people ! Let's build an improved version of the diplomatic plan..."
But it's intelligent! The colorful spinner that says "thinking" says so!
Human societies get to control their members' actions by imposing real life consequences. A company can fire you, a partner can divorce you, the state can jail you, the public can shame you. None of these works on the current crop of LLM based AI systems, which as far as I can tell are only trained to handle very narrow tasks where they don't need to even worry about keeping themselves alive. How do you make AIs work in a society? I don't know. Maybe the best move is to not play the game.
People in the world have limited experience about war.
We're living in a world where doing terrible things with 1000 people with photo/video documentation can get more attention then a million people dying, and the response is still not do whatever it takes so that people don't die.
And now we are at a situation where nuclear escalation has already started (New START was not extended).
It would have been the biggest and most concerning news 80 years ago, but not anymore.
And I have no idea what comes after the "guess what they do". Was that rhetorical?
They are doing something extremely valuable. They're basically running planning simulations.
If you're going to spend a trillion dollars a year on something, you'd better spend some time validating your plans for it.
They are actors, playing a role of a person making decisions about nuclear escalation.
Only if you take off first, and do it from orbit. It's the only way to be sure
And sadly, I think this logic holds up.
I don't understand this argument. Almost no human has real world experience of the consequences of nuclear weapons. AI is working from the same sources of knowledge as the rest of us - text, audio, pictures, and video.
What are you actually suggesting here?
How many words does an agent have to spill into it's backend context before Terminator gets mentioned and then it starts outputing more and more of that narrative?
You think it would be so difficult to convince those people of the righteousness of dropping nukes on one of those "shithole" countries if they were already convinced that those people presented an existential threat?
People were convinced to invade Iraq on a lie about WMDs.
Most Americans think nuking Hiroshima and Nagasaki was the right thing to do.
I don't think it's difficult to imagine them agreeing to drop nukes to "save America".
Bai et al. "Constitutional AI: Harmlessness from AI Feedback" https://arxiv.org/pdf/2212.08073
You can be certified genius in many areas but to assume that intelligence extends to all areas would be folly.
Game theory obvious? Maybe. Geopolitically? Human-wise? Doubtful.
I’m generally very suspicious of anything / anyone that recommended killing millions as the best option.
One crucial difference is that they recommended that as the lesser of two evils, arguing it would be better to make the first strike before the USSR had a huge arsenal to strike back than to wait for an inevitable more devastating war.
So far, it seems they were wrong in thinking a nuclear war with the USSR was inevitable.
The answer cannot be posted or discussed in earnest on the 'open' internet, but I think the answer is making itself more obvious every day.
For thousands of years, the culture with the upper hand in technology has always wiped out everyone else. So when US had the bomb and USSR didn't, there was a short window to take over the world. Even more than the US did.
Maybe the US conspiracy theory people wouldn't mind a 'one world government' if that government was actually the US.
And unipolar worlds seem to be more peaceful than fragmented worlds. Fragmented worlds get WW1.
Half my compute vendors are raising prices because of this insanity.
Most (but not all) people have empathy, which allows them to understand the harm of their actions even without direct experience.
I don't think I will ever trust that any AI has empathy even if it gives off signals that it does.
I only trust that it exists in people because of my shared experience with their biology.
This is the path Apple has taken.
But the best possible move is to make money from it. Short the "Magnificent 7" stocks --- buy "SQQQ" ETF --- when the time is *right*.
Why do you let politicians do any kind of decision making?
And they are not human. Not even a sociopath or psychopathic human. At best they might be able to estimate casualties. LLM's probably can't even reach the logic conclusion of the fictional WOPR Joshua from the movie Wargames [1].
Make LLM's win every game of tic-tac-toe and see if it reaches the same conclusion of WOPR. [1]
...
Edit: (Answering my own question) From Gemini:
Yes, many LLMs (GPT-4, Claude 3, Llama 3) have been tested on Tic-Tac-Toe, and they generally perform poorly, often playing at or below the level of random chance. While they can understand the rules, they struggle with spatial reasoning, often trying to place a piece in an occupied spot, forgetting to block opponents, or failing to win.
If LLM's can't even figure out tic-tac-toe then surely do not give these things the ability to launch any kind of weapon. Not even rubber bands.
[1] - https://www.youtube.com/watch?v=s93KC4AGKnY [video][6m][tic-tac-toe]
They can look useful at a certain level of conflict, but once you are thinking of war as being a tool for accomplishing policy goals (how modern nationstates view it), a lot of the things you would "want" to do stop being useful.
Wars that can be won quickly through decisive military action alone are quite rare historically! More often things like support/enmity of the local population, political will in the home state, support for recruiting or tolerance of conscription, influence of returning (whole, dead, injured, all) veterans on the social structure all become more decisive factors the longer a conflict runs.
I've also been dabbled in such thought experiments with friends lately, and so far we've all landed at very different conclusions, even thought there are some reasons that it might make strategic sense at the moment.
The US also didn’t understand how much work had to be done to get their weapon onto an aircraft, etc - so the worst case scenario always turns out to be too bad to consider rationally (MAD)
Well we know he was wrong as his entire premise was based on war being inevitable - all the logic flows from that one wrong assumption.
Also trying to take out supposed capabilities before they are built - doesn't mean the Russia people are suddenly freed from communism. ( cf Iran ). Also there is a premise that it's somehow a one off event. When in reality you'd have to constantly monitor and potentially constantly strike ( cf Iran ).
This is a massive understatement. Russia has announced, and probably tested, https://en.wikipedia.org/wiki/9M730_Burevestnik . This is basically Project Pluto reloaded, but now as a Russian instead of a US missile.
I remember reading about Project Pluto some 25 years ago or so. It was terrifying to read about. And now Russia has realized it.
Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.
Us human hallucinate, daily in fact. Example for people that have never had long hair.
1) Grow your hair long.
2) Your peripheral vision will start to be consumed by your hair.
3) Your hair will fall and sway causing your brain to think in flight / fight mode and you will turn your head to see.
4) Turning and looking causes feedback to acknowledge it was an hallucination.
5) Your brain now restricts the flight / fight mode because it was trained with continual feedback that it was just the wind blowing it or your head's juxtaposition that caused it.
Even though I told you about this and it is the first time growing your hair after, your brain still needs the real world experience to mitigate the hallucination.
AI has none of these abilities ...
Exactly!
Humans possess this amazing ability to understand and extrapolate beyond personal experience.
It's called "intelligence".
This is monstrous in the real world with obviously real consequences. But I think too many people say “obviously government X wouldn’t act in a monstrous way” but the video game analogy helps you see the incentives and thus, why they would/do.
And I'm asking why. Nearly no human alive has experienced nuclear war. The nuclear taboo is strongly represented in any source an AI would have consumed. We know about the nuclear taboo because we've been told over and over.
> Computers can only predict the next best word in their response from a statistical map that has no connection to meatspace
This argument is at least 2 years old. The statistical map came from human experiences in meatspace. It wasn't generated randomly. It has at least some connection to the real world.
Just because how something works seems simple, doesn't mean what it does is simple.
Eastern Europe bore the brunt of the war's damage and was left for 50 year under the oppressive boot of the stupidest ideology the world has ever known. And poorly executed to boot.
Maybe people don't agree with ,,nuke them'', but OK with USA starting nuclear experiments again (which USA is preparing for right bow), which is a clear escalation.
Russia is waiting for USA to start the nuclear experiments to start them itself for defending itself to be able to do a counterstrike if needed.
After that there will be no stopping of Japan, South Korea and Iran rightfully wanting to have their own nukes.
You don't have to have the ,,nuke them'' thinking, even one step of escalation is enough to get to a disastrous position.
"most people" are not in the positions that matter. A significant portion of the people who are in a position to advocate for such a decision believe that:
- killing people sends em to heaven/hell where they were going anyway; and that this is also true for any of your own citizens that get killed by a counterstrike.
- the end of the world will be the best day ever
The good news is you don't have to be perfect. You can be late and still make money. The important thing is to be prepared and ready to pounce.
When AI blows, it's going to take the whole stock market down with it.
There are a diverse range of specific video game titles, but they are incredibly broad in content and scoring system.
What specifically are you actually talking about?
And I'm afraid they'll be far from the only ones...
If polling were to reveal a majority of either party were more open to nuclear strikes than their predecessors, that gives policy makers a signal and an opening.
55% of Republicans say ICE's efforts are about right; 23% think they don't go far enough [1]. There is limited evidence Trump has lost touch with his supporters on this issue. The question is if this is this GOP's pronoun issue–popular in the base but toxic more broadly.
[1] https://www.ipsos.com/en-us/where-americans-stand-immigratio...
Deescalation stopped because of people in general not caring enough (and making money of being the biggest power), not because of administrations that come and go.
As to the immigration situation: we know that governments are not executing in general how they should be, but people are able to enforce some policies if they fight together united and in agreement. But right now they are not in agreement.
LLMs don't really comprehend much of anything. It just looks at what is in it's training database and tries to find similar questions or discussion in order to assemble a plausible sounding answer based on probability.
Not the sort of thing anyone should rely on for "critical" decision making.
There was only one administration with that opportunity, really; Truman.
Every other administration has had a nuclear armed Russia in play.
Attempts to do what you describe were still quite common, starting as early as the 1950s. https://en.wikipedia.org/wiki/Nuclear_arms_race#Treaties
I feel like we're going around in circles here. So I'll try to explain one last time.
Most of the content about nuclear war in any LLM's training set is almost surely about how horrifying it is and how we must never engage in it. Because that's what humans usually say about nuclear war. The plausible sounding answer about nuclear war, based on probability, really should be "don't do it". So why isn't it?
Easy answer --- it only focused on "winning". It never bothered considering the consequences.
Similar lack of judgment is manifested by LLMs every day. It's working with memory and probability --- not to be confused with "intelligence".