So DoW did get the “all lawful purposes” language they were after, with reference to existing (inadequate, in my view) regulations around autonomous weapons and mass surveillance.
But I imagine that being honest about your corporate identity is suboptimal. It’s probably an important cognitive dissonance tool for the employees? It’s like when autocracies repeat big obvious lies endlessly. Gives those who want to opt out of reality an option.
Posted here: https://news.ycombinator.com/item?id=47195085
Can anyone explain this constraint?
Why do fully autonomous weapons require edge deployment?
Does "fully autonomous" in this context mean "disconnected from the Internet"?
If so, can a drone with Internet connectivity use OpenAI?
Or maybe it's about on-premise requirements: the military doesn't want to depend on OpenAI's DCs for weaponry, and instead wants OpenAI in their own DCs for that?
From what I can tell the Anthropic issue was triggered by something Palantir was doing as a contractor for DoW, not anything related to direct contracts between DoW and Anthropic, and DoW was annoyed that Anthropic interfered with what Palantir was up to.
In other words will OpenAI enforce these "red lines" against use by a third-party government contractor?
If not, this seems pretty meaningless if they are essentially playing PR while hiding behind Palantir.
Weak. You reached a deal that Anthropic could not because you demanded more safeguards than Anthropic?? (Based on what you know, of course).
Makes total sense!
This feels like IBM in the 1930s selling tabulating machines to the Germans and downplaying their knowledge of their use. They seem to want us to naively believe they won't use it for exactly what the military has always wanted, autonomous weapons and mass surveillance. Further more there are much more mundane use they might make of the technology that is perfectly legal yet morally in gray areas.
This is weak.
I’m going to be left with scrap PCs and Debian at this rate.
But Sam pretending that he wanted the same restrictions as Anthropic *and* seeing how quickly they swooped in and made a deal with the DoD really skeeves me out. (But Sam always gave me the heebie jeebies).
Anyway, I've always preferred Claude, so I'm going to happily stay a paying customer there. This may end up being a big "branding" differentiator.
In my mind, the government would be fully happy to use this to surveil citizens (and indeed anyone) with or without any legal basis, but the issue was that Anthropic has a safety stack / training and inference protocols that it follows. Refusals, abuse models, and manual guardrails. They didn’t want to shut those off. Likely there were some very basic technical reasons, some being that the team’s safety posture is fully ingrained in the model itself and thus difficult to remove.
In this document, OpenAI admits that while they are not “turning off” their safety stack, they are completely willing to provide the government with a different model, different guardrails, etc. That should be incredibly concerning. Anthropic was unwilling to do this, cited their ToS, and ultimately had to walk away from the deal. Given that the government (DoW really) framed this in terms of a hilariously stupid position (surveillance and autonomous weapons), Anthropic felt that this was something they could voice to the public and therefore the entire guardrails discussion turned into a “we want the language changed”. Also the government can’t actually compel Anthropic to create new guardrails so they had no choice but to raise the stakes, make this a moral thing, and basically accuse Anthropic of being woke.
IMO this is really sad for OpenAI employees. Yet again Sam Altman proves that he wants to weasel his way around public perception. Folks at the company have to grapple with working for someone of that disposition.
> We want AI to be aligned with all of humanity.
One of many contradictions. Liars.
Our contract explicitly references the surveillance and autonomous weapons laws and policies as they exist today, so that even if those laws or policies change in the future, use of our systems must still remain aligned with the current standards reflected in the agreement.
So, this apply only if they changes the law, not if they break the law.
"What happens if the government violates the terms of the contract?"
As with any contract, we could terminate it if the counterparty violates the terms. We don’t expect that to happen.
WE COULD [...]. Yeah, I believe
The emphasized language is the delta between what OpenAI agreed and what Anthropic wanted.
OpenAI acceded to demands that the US Government can do whatever it wants that is legal. Anthropic wanted to impose its own morals into the use of its products.
I personally can agree with both, and I do believe that the Administration's behavior towards Anthropic was abhorrant, bad-faith and ultimately damaging to US interests.
My reading of this is that OpenAI's contract with the Pentagon only prohibits mass surveillance of US citizens to the extent that that surveillance is already prohibited by law. For example, I believe this implies that the DoW can procure data on US citizens en masse from private companies - including, e.g., granular location and financial transaction data - and apply OpenAI's tools to that data to surveil and otherwise target US citizens at scale. As I understand it, this was not the case with Anthropic's contract.
If I'm right, this is abhorrent. However, I've already jumped to a lot of incorrect conclusions in the last few days, so I'm doing my best to withhold judgment for now, and holding out hope for a plausible competing explanation.
(Disclosure, I'm a former OpenAI employee and current shareholder.)
More of the same here. Not a wonder why the DoD signed with OpenAI and instead of Anthropic. Delegating morality to the law when you know the law is not adequate seems like "not a good thing".
"For intelligence activities, any handling of private information will comply with the Fourth Amendment, the National Security Act of 1947 and the Foreign Intelligence and Surveillance Act of 1978, Executive Order 12333, and applicable DoD directives requiring a defined foreign intelligence purpose. The AI System shall not be used for unconstrained monitoring of U.S. persons’ private information as consistent with these authorities. The system shall also not be used for domestic law-enforcement activities except as permitted by the Posse Comitatus Act and other applicable law."
> “ The Department of War may use the AI System for all lawful purposes… [proceeds to describe current law, with clear openings if the law changes]”
Thus, OAI is relying on the Trump administration’s interpretation of current law. Which, I will remind readers, suggests that it is legal to kill civilians on boats, kidnap foreign leaders, deploy troops in American cities, shoot American citizens protesting ICE.
Yeah I’ve cancelled my OAI sub.
I wonder how many will do so, and how many will simply accept Sam’s AI written rationalization as this own and keep collecting their obscene pay packages…
Goodbye Sam.
Edit: Also, referring to the DOD as the Department of War is cringe.
But I do think my cancelling ChatGPT so I can try Claude, at this time, sends the message I want to send, which is why I did it.
Even Google and Microsoft should be worried. This is like 1936 germany, we have ways to go. Look at the tune this administration is singing, if they get their way these CEOs aren't looking at law suits and federal investigations, the current order of things will be long gone by the time people start asking who's responsible for all the blood on the streets.
It begins “The Department of War may use the AI System for all lawful purposes…” and at no point does it limit that. Rather, it describes what the DOW considers lawful today, and allows them to change the regulations.
As Dario said, it’s weasel legal language, and this administration is the master of taking liberties with legalese, like killing civilians on boats, sending troops to cities, seizing state ballots, deporting immigrants for speech, etc etc etc.
Sam Altman is either a fool, or he thinks the rest of us are.
Excuse me, but what a fucked up perspective. "Impose its own morals into the use of its products"? What happened to "We give each other the freedom to hold beliefs and act accordingly unless it does harm"? How on earth did it come to something where the framing is that anyone is "imposing" anything on another simply by not providing services or a product that fits somebody else's need? That sounds like you're buying into the reversed victim and offender narrative.
And this is not about whether one agrees with their beliefs. It is about giving others the right to have their own.
This is key because it's the textbook example of a war crime. It's also something that the current administration has bragged doing dozens of times.
More succinctly: who decides what is legal here? OpenAI, the Secretary of Defense, or a judge?
The other says you may build the Terminator if the DOD lawyers say it’s okay.
This is a major distinction.
> No use of OpenAI technology to direct autonomous weapons systems
Aiding someone while you know they're trying to break the law is conspiracy to break the law. OpenAI is culpable. You can't sue the government in many cases, but you can with OpenAI.
He calls this exact scenario out in last night's interview: https://youtu.be/MPTNHrq_4LU
Even on a personal level: OpenAI has changed it's privacy policy twice to let them gather data on me they weren't before. A lot of steps to disable it each time, tons of dark patterns. And the data checkout just bugs out too, it's a fake feature to hide how much they are using everything you type to them
Employees often have the power to oust the owner and take over the company; and more often than that have the power to have business grind to a halt. It does take a strong union and a culture of solidarity and sticking together of course, which I doubt we would find in a place like OpenAI.
I sort of agree and think that over a long horizon, Open weights models are going to be the best / are the best
I do think only a fraction of companies might do what Anthropic did here. There must have been quite a significant pressure on them to fold but they didn't. So to me, I'd rather try to do atleast something to show companies that people do care about such things and its best if we have at the very least some unconditional morals which are not for sale no matter the price.
I think that we can still have disagreements with Anthropic on matters and I certainly still have some disagreements about their thoughts on Open Models for example but in all regards I would trust them as more trustworthy than OpenAI imho.
That being said, I do think that its worth telling that given that I don't have good GPU, I am gonna stop using Chatgpt as well and will use either Claude/(Kimi?) as well like many people are doing too. I do think that it might be the path going forward.
What if Anthropic's morals are "we won't sell someone a product for something that it's not realistically capable of doing with a high degree of success? The government can't do what something if it's literally impossible (e.g. "safe" backdoors in encryption), but it's legal for them to attempt even when failure is predetermined. We don't know that's what's going on here, but you haven't provided any evidence that's sufficient to differentiate between those scenarios, so it's fairly misleading to phrase it as fact rather than conjecture.
I guess you can consider it a moral stance that if the government constantly does illegal things you wouldn't trust them to follow the law.
I know that's not what Anthropic said but that's the gist I'm getting.
Mass surveillance doesn't require a warrant, that's why they want it, that's why it's "mass". warrants mean judicial overview. Anthropic didn't disagree with surveillance where a court (even a FISA court!!) issued a warrant. Trump just doesn't want to go through even a FISA court.
This is pure evil from Sam Altman.
Is anyone listing these peoples names somewhere for posterity's sake? I'd hate to think this would all be forgotten. From Altman to Zuckerberg, if justice prevails they'll be on the receiving end of retribution.
The "human approval" will be someone clicking a YES button all the time, like Israeli officers did in the Gaza bombing.
This is an incredible power when exercised en-masse.
I can see the logic if we were talking about dumb weapons--the old debate about guns don't kill people, people kill people. Except now we are in fact talking about guns that kill people.
> More succinctly: who decides what is legal here? OpenAI, the Secretary of Defense, or a judge?
Yeah, there's a pretty strong case that anyone claiming to trust that the administration cares about operating in good faith with respect to the law is either delusional or lying.
Not surprised to see a guy like Altman adopt the strategy
>How on earth did it come to something where the framing is that anyone is "imposing" anything on another simply by not providing services or a product that fits somebody else's need?
The department of defense in particular has a law on the books allowing them to force a company to sell them something. They generally are more than willing to pay a pretty penny for something so it hardly needs used, but I'd be shocked if any country with a serious military didn't have similar laws.
So your right when it comes to private citizens, but the DoD literally has a special carve out on the books.
A lawsuit challenging it would have actually been insane from anthropic because they would have had to argue "we're not that special you can just use someone else" in court.
A more clear example would be, what would you expect to happen if Intel and amd said our chips can't be used in computers that are used in war.
If Anthropic can survive on open source contributors shelling out $200/mo and private sector companies doing the same, the government wishes them well. But surely you agree the government has a right to determine how its budget is appropriated?
This is just incoherent. You can't have US companies fix an unhinged US government.
If the government runs wild, there are some serious questions to be asked at a state level, about how that could happen, how to fix it quickly and how to prevent it in the future – but I should hope none of them concern themselves with the ideas of individual company owners, because if the government can de fact do what it wants regardless of legality the next thing that this government does could simply be pointing increasingly non-metaphorical guns at individual AI company functionaries.
> Per DoD Directive 3000.09 (dtd 25 January 2023), any use of AI in autonomous and semi-autonomous systems must undergo rigorous verification, validation, and testing to ensure they perform as intended in realistic environments before deployment. The emphasized language is the delta between what OpenAI agreed and what Anthropic wanted.
> OpenAI acceded to demands that the US Government can do whatever it wants that is legal. Anthropic wanted to impose its own morals into the use of its products.
So first off, regarding that first paragraph, didn't any of these idiots watch WarGames, or heck, Terminator? This is not just "oh, why are you quoting Hollywood hyperbole" - a hallmark of today's AI is we can't really control it except for some "pretty please we really really mean it be nice" in the system prompt, and even experts in the field have shown how that can fail miserably: https://www.tomshardware.com/tech-industry/artificial-intell...
Second, yes, I am relieved Anthropic wanted to "impose" their morals because, if anything, the current administration has been loud and clear that the law basically means whatever they says it does and will absolutely push it to absurd limits, so I now value "legal limits" as absolutely meaningless - what is needed are hard, non-bullshit statements about red lines, and Anthropic stood by the those, and Altman showed what a weasel he is and acceded to their demands.
for many decades, the DoD has used a carrot to get what they want. this is a stick.
I think the government doesn't have rights, it is my elected representative. And I do not agree with it trying to punish a company for not agreeing to contract terms.
Consider this, the bill of rights stipulates that a soldier cannot be stationed on your property in times of peace, but in times of war it will be allowed. It makes exceptions for times of war. but even in times of war, 4th amendment's search and seizure protection don't have an exception. Even in times of insurrection and rebellion. To deliberately violate that for personal and political reasons, that in itself is treason. With that intent alone, even without action, it invalidates all legitimacy that government has. If a clause in a contract is broken, the contract is broken. The bill of rights is the contract between the people and their government that gives the government its powers to rule, in exchange for those rights. With the contract explicitly, deliberately and with provable malicious intent broken, the whole agreement is invalidated.
I'll even say this, the US military itself is on the hook if they stand by and let this happen.
This has been a huge talent advertisement for Anthropic. Their recruiting just got easier for the next 6 months.
But, as I've said, I tend to agree with both Anthropic and the Administration's positions. What was wrong here is that rather than just terminating the contract, the Administration went nuclear.
This is obviously subjective, and the only subject that matters in this case is the leadership at the DoD.
> We have no choice but to pay taxes and make the federal government 20 percent of our economy. There is no single company or any other entity that is close. And extending it to everyone who has a government contract probably makes it the majority of the economy.
I, too, hate big government and the all-powerful executive branch. Welcome to my tent. Let’s invent a time machine together so we can elect Ron Paul in 2008 and nip this in the bud.
Until then, this is what we’re stuck with.
The current US government has a fundamentally different ontology for the derivation of human rights.
Wheras you and I likely agree that human rights are inalienable due to them being derived from the universe nature of human experience, the administration believes that human rights begin and end with them, the state. When they're the one able to affect the world with violence, it doesn't matter who's on the hook. The US electorate thought they could heal a status wound by authoritarianism instead of therapy and everyone else is paying the price.
That would most definitely not be the Constitutional recourse. Or a sensible approach. If that happens, the Constitution is past tense.
Congress and the Supreme Court are the recourse. If they don't hold up the Constitution then violence or even a non-violent military coup, however well intended, are not going to put the splattered egg back together again.
The last two and a half decades have seen all four presidents, congress, the Supreme Court and both parties allow blatantly unconstitutional surveillance become the norm (evolving an adaptive fig leaf of intermediaries), and presidential military actions entirely blur out the required Congressional oversight. That the weakening of loyalty to the Constitution has been pervasive on those serious counts, is one of the reasons it has been so easy to undermine further.
When governing bodies become familiar with the convenient practice of "deciding" what the constitution means, without repercussions, that lost respect becomes very hard to reinstate.
"Impose" makes it sound like Anthropic is being hostile here. And also, I don't think this is a situation that calls for moral relativism.
The US government can update its laws and come back to Anthropic, or do what they just did
> The US government can update its laws and come back to Anthropic
No, this I do take issue with. It's the people who update the U.S. government's laws.