There's a lot of hand wringing about how far wrong LLMs can go, but can we be serious for a second, if you're running <whatever the name is now>, you're tech savvy and bear the consequences. This isn't simple child abuse like teenage girls on facebook.
There is a reason people are buying mac minis for this and it's cool. We really need to be more excited by opportunity, not threatened.
Thanks to the reports, hopefully, with time, some additional security measures will also be added to the product.
I wonder how many people have inadvertently enabled access to some auto-pay or donate function buried in some other service their bot has access to.
[0] https://garymarcus.substack.com/p/openclaw-aka-moltbot-is-ev...
Ah, so a bit more useful than my teenage son? Where do I sign up??
> With direct access to the Internet, the ability to write source code and increased powers of automation, this may well have drastic and difficult to predict security consequences.
AutoGPT was a failure, but Claude Code / Codex CLI / the whole category of coding agents fit the above description almost exactly and are effectively AutoGPT done right, and they've been a huge success over the past 12 months.
AutoGPT was way too early - the models weren't ready for it.
The answer is, no, because people will take the AIs out the box for a bit of light entertainment.
Let alone any serious promise of gain.
Maybe we should take the same approach to bridge design! Think of the efficiency! Slap a disclaimer on that bad boy and see how many people choose to use the bridge at their own risk. I’m sure we can just assume people aren’t doing irresponsible things like driving school buses over it, and even if they were, it’s their own responsibility.
It’s really not so bad if you focus your messaging on how many people won’t die… and’s they’ll all lean from the mistakes of the dead and choose a more reliable bridge. And it would be so much cheaper and faster to build bridges so you’d have a fraction of the downtime. I think it’s a winner!
Sure there would be larger consequences for the local job market and such when they get disrupted, but hey… if you’re going to make an omelet…
It's all lighthearted hypotheticals until someone you love or you yourself in a moment of inattention make a catastrophic mistake.
In theory, we don't need guardrails on roads. Just stay on the fucking road and if you swerve off it, you'll get a lesson in why that's a bad idea.
In practice, we are primates whose cognitive systems are made of squishy grey goop and we make mistakes all the time. Building systems that turn predictable mistakes into catastrophic consequences is what we used to call "poor engineering".
He "started out" a lot earlier, he wrote a book in 2001 and his written 8 books in total and has publications in academic journals like Cognitive Psychology dating back to 1995.
The world didn't start when LLMs got popular.
Yeah, this guy is... something. The text form equivalent to Youtube Shorts.
By what I've seen so far it is great for exposing (sensitive) data.
I’m glad I’m not the only one. As a parent, the “teenage son” is a bewildering sight to behold.
He’s vocal and perhaps sometimes annoying, but who cares. A number of his articles have made great points at times when people are loosing themselves with hype for AI. Reality is somewhere in the middle, and it’s good to have more people critiquing what AI companies are doing. Who cares if a lot of the blog posts are short and not that interesting.
LLM people defend these tools/companies as if it were their girlfriend..
Unfortunately, that might be way more of a reality than fiction.
They lose billions of dollars annually.
In what universe is that a business success?
The conclusion that seems readily apparent to me, as it has always been, is that these "agents" are completely incapable of creating production-grade software suitable for shipping, or even meaningfully modifying existing software for a task like a port. Like the one-shot game they demo'd, they can make impressive proof-of-concepts, but nothing any user would use, nor with a suitable foundation for developers to actually build upon.
Essentially, the author has deliberately misconfigured an openclaw installation so it is as insecure as possible, changing the defaults and ignoring the docs to do so. Lied about what they've done and what the defaults are. Then "hacked" it using the vulnerability they created.
That said, there are definite risks to using something like openclaw and people who don't understand those risks are going to get compromised, but that doesn't justify blatant lying.
When he was a young botnet!
But we don't want that! We want blind faith in the promises of SV AI companies. We want positivity!
I base this on my own experience with them plus conversations with many other peers who I respect.
You can argue that OpenAI Codex using Electron disproves this if you like. I think it demonstrates a team making the safer choice in a highly competitive race against Anthropic and Google.
If you're wondering why we aren't seeing seismic results from these new tools yet, I'll point out that November was just over 2 months ago and we had the December holiday period in the middle of that.
1. wasn't economical to write in the first place previously, and
2. doesn't need to be sold to anyone else or maintained over time
So, Brad in logistics previously had to collate scanned manifests with purchase requests once a month, but now he can tell Claw to do it for him.
Which is interesting given the talk of The End of Software Development or whatever because "software that nobody was willing to pay for previously" kind of by definition isn't going to displace a lof of people who make software.
The organizations that provide them lose money because of the R&D costs involved in staying competitive in the model training arms race.
I wonder why that is, and what it portends regarding the future of that "tribe"
On one side you have people who know how to build deep nn saying one thing, and on the other there seems to be people who don’t even know what tanh is and are very sure of their “strong” opinions.
Do you have an example of someone who actually knows how LLMs work who has a tribalistic view?
I will note that you specifically said the agents have shown huge success over "the past 12 months", so it feels like the goalposts are growing legs when you say "actually, only for the last two months with Opus 4.5" now.
Checking whether Claude Code by itself is profitable or not is probably impossible. It doesn't make a lot of sense divorcing R&D from the product. And obviously the running costs are not insignificant.
The company as a whole loses money.
OpenAI Codex CLI and Gemini CLI followed a few months afterwards
It took a little while for the right set of coding agent features to be developed and for the models to get good enough to use those features effectively.
I think this stuff went from interesting to useful around Sonnet 4, and from useful to "let it write most of my code" with the upgrades in November.
If they make money on each customer they have a credible business - they could become profitable even with their existing R&D losses provided they can sign up enough new paying customers.
If they lose money on every customer - such that signing a $1m new enterprise account costs them $1.1m in server costs - then their entire "business" is a sham.
I currently believe that Anthropic make money on almost every customer, such that their business is legit.
I guess we'll have to wait for the IPO paperwork to find out if I'm right about that.
Most of the investors and companies that built the rail network went bust. The iron remained.
Most of the investors and companies that built the telecom network went bust. The fiber remained.
Most of the investors and companies that are building models will go bust. The files (open weight or transfered to new owners for pennies) will remain, and yield economic benefits for as long as we flow current through them.