Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.

Trying to use markdown files to limit access should never be treated as a security guarantee at all.

This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.

The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.

You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.

But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.

Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.

Install with

pi install npm:pi-supergrok

Reality is, I have seen agents read .env, bash history, keychain (if you let them), etc.

There is quite literally no way you are going to save off just your little secret somewhere it won't be able to read it, all software needs to read it ~eventually~

So it's best to sandbox and reset credentials frequently.

> We're a stupid industry if we're not warning everybody

Hmm. Annoy everybody just to warn the stupid few?

I prefer the current solution. Leave the targetting to the chatbots.

I don't use AI at all in my daily life.

Work however will demand you use it.

AI is not here to help people.

    sudo -u restricteduser myagent

I didn't really need a second clue.

If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.

And the user even paid $99/month or more for having their data leaked.

This sort of thing is why I'm hopeful I'll continue to have employment going forward. Some expertise is hard won and there's just no replacing learning through experience.

It needs to be baked into the OS.

At that point, HN users start screeching about it, so it's lose/lose, really.

https://cursor.com/docs/agent/tools/search

I don't think it is reasonable to expect every user (including those just starting out with the tools - maybe experimenting, maybe younger/less experienced in general) to think that the tool they're running for the very first time is going to automatically exfiltrate all of their data.

It's a pretty serious fuck-up. This guy tweeted about it, who knows how many didn't even notice. It should have been opt-in, it should give user an indication that it's about to do this, etc.

The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.

what is rocks and what is gars?

I've heard it said that piping curl into your shell is no different to running any other program you've downloaded from the Internet (binary, or otherwise): the maximum possible damage that `example.com/script.sh` could do is exactly the same as `githubusercontent.com/someone/releases/myprogram.exe`. At least with `script.sh` you can easily inspect what the script actually does instead of busting out Ghidra.

It comes down to trust: do you trust Example.com to not serve-up a malicious program (shell script or executable binary)?

Now we take that principle and apply it to Mr. Musk's "MechaHitler" LLM vendor xAI: they have a well-documented history of unnecessary risk-taking - and outright criminal behaviour (child-porn generators are a good thing that everyone should have, apparently?). Would I trust Grok with anything? Absolutely not.

> Until I'm shown otherwise it seems like a good compromise

Agreed. In my case I went a long way running the Pi harness in a (simple, rootfull) docker container. As the project I worked on relied on a standardized docker compose stack for local dev and testing, I realized I could automate more if only my agent could use docker. Ultimately, the need for docker for my agent grew when testcontainer [2] was introduced in the project. That's when I finally took the time to setup a VM with incus [3], and now I can let the agent go wild with docker inside the VM.

This is at least one example where more isolation is required. Otherwise, the dedicated Linux user, if it works for you, is by far the easiest and most pragmatic solution IMO.

[1] https://docs.docker.com/engine/install/linux-postinstall/#ma... ; https://wiki.debian.org/Docker

[2] https://testcontainers.com/getting-started/

[3] https://linuxcontainers.org/incus/docs/main/

It brings some extra sandboxing features compared to just a docker container.

For instance it also maps the secrets that the agent harness has access to. And has an allow list for outgoing http connections. And there's a way for agents to request extra access. And once approved by the user from an external cli the policy is updated and hot reloaded.

It's pretty recent. So time will tell how robust it is / will become. What its longevity will be. After all; in a time of LLMs one off experiments are cheap. Longtime nurturing not so much.

But it turns out even in the container, there are footguns that have occasionally made the news by being fired: few projects don't have any external resources and when credentials with any form of write access happen to make it into the container (even if it's just a session cookie) agents might jump at the opportunity.

[0] https://pubs.aeaweb.org/doi/pdfplus/10.1257/0002828067772121...

Some/maybe most of the HN crowd, sure, but as a rule ... definitely not. People are generally happy to take whatever the path of least resistance is.

I know quite a few civilians (even C-level people) who "heard openclaw was cool" (or whatever) and downloaded the first installer they found and hit enter/run without thinking twice.

If I recall correctly, I did a full system reset before setting it up this way. It's certainly not logged into iCloud etc.

Putting it in ALL CAPS!

Instead of generating patches, this exposes the agent's checkout as a git remote though.

Most similar tools (and I believe your tooling as well?) bind-mount the repository checkout from the host into the container. This was always a source of user and permission errors for me, since you have to align the user ids inside and outside the container. Also, some build tools don't like it when the repo is on a different filesystem (bind mount vs container root) than the rest of the system. So I made rumpelpod just bake the repo checkout into the base image that the container is launched from, and since then I haven't had any issues like this.

For giving the agent access to a docker daemon I found sysbox to work very well. Usually the advice for nested containers is to pass in the host's docker socket into the container. But that would break the outer container isolation completely, since access to the docker daemon is equivalent to root access on the host. With sysbox it's trivial to run a nested docker instance inside the outer container. I haven't tried it with podman yet though. In theory sysbox is just another OCI runtime, but there's probably some tinkering required to get it to work.

    docker run --rm -it -v $(pwd):/src -w /src alpine sh

Note entirely perfect, but will be enough against anyone not actively exploiting kernel privilege escalation bugs.

... that's guaranteed vuln-free, right?

It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.

For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.

This is a new pattern for me, I'm curious what others are doing.

This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.

...it was quite the sting because I bought a Tesla car only 2 weeks prior to that.

This is probably most important for anyone operating agents on corporate systems. I would suggest corporate security should take interest in this idea so they have a good answer for auditors, insurance companies, investors and customers. A full audit trail of what every agent had access to and ingested.

True, but it isn't here to not help people, either.

It's a spanner. Who wields the spanner, makes all the difference.

We've spent the last couple of decades, cultivating a huge crop of ultimate scumbag billionaires, with comically exaggerated sociopathy, and that has filtered down to almost every level of society. They are treated as gods, these days (they certainly think of themselves that way).

It still shocks me (but really shouldn't), on a daily basis, to encounter regular folks, interacting in stores and restaurants, or driving on roads, that mirror the values systems exemplified by our billionaires. Our politicians act that way, and one of their biggest selling points, is normalizing sociopathy (not just the US, either).

LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time. I've come to terms with the impact slop coding will have on most software jobs in the future, but seeing seemingly intelligent people fall for lies and fantasies concocted by an LLM is making me more and more uncomfortable with the direction we're all heading in.

I don't really have a good mental model of how ports/files/etc get exposed/permissioned across users. Containers and sandboxing seem much more common than agent-user accounts. Networking seems a little more complicated in the general case.

I roughly went the route of running apple's container-cli (separate vm/kernel per instance I believe) and mount the relevant home directory/projects and bind ports. Seatbelt/linux sandboxing seems simpler sometimes, but has its complications too.

Not sure which weirdness happened here

My assumption when asked "do you trust this directory?" is that I am being asked if I am certain I understand what is in the directory and that it won't include some sort of prompt injecting attack. I would never dream that I was consenting to the complete exfiltration of that directory.

We need to make it easier (and faster and cheaper) to get a secure cloud agent computer than it is to YOLO it on your own computer.

Personally I'm not worried about sharing business code with the model providers. The value of code and IP is decreasing to zero, the only thing that matters is execution on the business front. If they want to steal your business they can do so by rebuilding the idea from scratch.

Of course not.

To me it's on a server, in a VM. And they're not seeing the real data/databases from the actual projects: they're seeing fake infos used only while in the dev environment. There's no way I'm dumping, even for tests, the real or part of the real DB somewhere an AI can see it.

To find bugs (for example), AIs are useful but honestly for code generated by LLMs, I'm thinking about going back to the early copy/paste from the ChatGPT days: because I see so many horrors in the code output by the latest SOTA LLMs that every single line of code they spew has to be checked by someone who does know better.

It's not just an issue of protecting confidential data / preventing spying: we're all discovering that we've got serious sloppy-pasta code problems now.

Even if there is a misunderstanding who is really uploading the directory, the TUI/CLI itself by actual code, or if the model decided to do so in the session, if you apply the recommendations from the replies to parent, and it no longer matter who did it, neither the software nor the model will be able to upload all your ssh keys.

Whose fault is it if someone drives a car without learning how to and injures themselves? On the other hand if the manufacturer has promoted it as one you can drive without learning how to, then whose fault is it?

A lot of users are fine with everything being uploaded. Most people's primary computing device is now a phone that backs up everything to cloud and using apps that are thin front ends over cloud services.

My two guesses would be one the LLM decided it needed these files for the task or two the user simple asked grok to do it so they could post the tool calls on twitter.

[0] https://github.com/superagent-ai/grok-cli

So, Google "backs up" a 128Gb worth of photos on the phone onto 15Gb free storage combined with Gmail and who knows what, completely clogs it (as if it couldn't be predicted) and then has audacity to suggest paying for "extra storage". There is no way in online UI to just delete the whole "backup". And the cherry on top: when you finally get to delete some there is a fine-print - "the selected photos will be deleted from all synced devices". Well, I guess I must be thankful that they at least show this warning. This is what passes as "backup" in Google's parlance these days.

It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"

Claude.md is an equally effective defensive tool.

But sorry if you lost some files from reading that.

I explain a bit more over here: https://news.ycombinator.com/item?id=48893874

It copies the current Git repo into the sandboxes dir, mounts that copy at /workspace in the container. The original repo is never mounted writable, so I don't care if the agent goes to town/wild in there (peace of mind).

It also builds cached Debian/mise/Elixir/Phoenix images, can start a private Postgres container, publishes selected localhost ports, reuses dependency/build caches, and prints commands on exit for reviewing diffs, exporting patches, applying them back to the real repo, or reopening the same sandbox later. Pi, and OpenCode are configured with proper LLM access keys (derived from my own).

So spinning a new sandbox is a matter of cding into a project directory and run something like: `ai-sandbox --port 4000 --postgres somedbname` or `ai-sandbox --port 4001` if I don't need DB support. Then when running the server in the container I can access it from the host machine to review in my browser.

https://www.thetimes.com/world/article/google-bans-father-ov...

> Mark, from San Francisco, had noticed swelling in his son’s groin and used his phone to photograph the problem to get an emergency appointment in February last year. He shared the pictures with a nurse so that a doctor could review them.

> However, Google’s artificial intelligence system used to detect child abuse flagged the image to the police and Mark, a software engineer who asked to be identified by only his first name, was investigated and lost access to his Google accounts. He was exonerated by the police in San Francisco but his Google account has not been reinstated.

Or, the internal sandboxing.

But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.

That said, I do hope these become the norm if LLMs are here to stay.

I wonder what will happen after our benevolent prophets St.Sam and St.Dario will succeed in re-training humanity and break this collective expectation of program correctness. I guess they didn't even think about that.

It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.

https://en.wikipedia.org/wiki/Automation_bias

Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?

And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."

And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.

In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.

For example: seat belts are here to help people (by reducing the severity of injuries & the likelihood of fatalities).

I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).

We don’t have to head in this direction of using LLMs for most development at all.

It's annoying, and often rather infuriating, but I understand that one of the motivations for people buying Apple stuff, is for that very reason, so I'm really sawing off the branch that I'm sitting on, by trying to work around it.

I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.

P.S. the conflicting information was Keith Richard's age. One search said he was 37 Dec 18 1981, the other said he was 37 in 1980.

For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.

It's the manufacturer's fault. Because that's not a reasonable thing for a car to do.

I understand that one should think carefully about how they work with a non-deterministic tool, but this if different completely. This is xAI just choosing to upload and store everyone’s directories - with full git history.

We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.

(Addendum: But indeed, the failure to formalize our assumptions into enforcable rules isn't specific to agents / LLM-based applications. For any desktop app, we have implicit assumptions like "please only read and write your own config files", that we never care to enforce via filesystem permissions, running them in a vm or similar. But with these almighty agents that are supposed to guess our will from just a couple of words, the risk of them violating our unwritten assumptions gets so much higher...)

The tool analogy is intentionally minimizing, and doesn't capture just how different rented tools with constant surveillance are.

   > cat /etc/passwd|wc -l
   50

System-level ACLs; mandatory or discretionary access control; secure-by-default application and network configurations are all for naught if you take an LLM, run it with all the privileges you'd have an accountable, judgemental operator, and then tell it to act based on arbitrary untrusted input which might include prompt injection attacks, something which cannot generally be sanitized.

Well-defined, well-enforced security policies can mitigate disasters, but many in the wild right now just don't account for this kind of threat model.

Sure, for IT crowd containers are a no-brainer and due to existence of validation software like compilers, linters, interpreters, analyzers etc. using LLMs is efficient, safe and rational.

But the problem is that they are also advertised to the remaining 99% of population too. You can't sandbox a bank, a broker or an exchange. You can't sandbox your email, calendar and other such tools (you can, but not in any way that will be used by non-IT people in any real work conditions). You can't sandbox messengers, social networks etc. And the list goes on. These people will either use so called "agents" in production under root, so to speak, or they won't use "agents" at all. They have no safety option, and they aren't properly informed about that by the peddlers of the LLM rapture.

I never claimed WSL2 is a security sandbox. I am saying running it in a container or WSL2 allows you to severly limit the blast radius. I am not expecting the agent to be malicious, but I am expecting it to do unexpected things.

Beyond that, I just keep advocating for more safe and secure approaches any opportunity I get.

Replacing people is what C-suiters think of it (and many of them aren't billionaires).

In the README it mentions that it puts the dev environment in a filesystem jail, but how are you able to use your hosts bins without leaking access to the rest of the system? Or is that just an assumed liability?

It's a common misconfiguration and one of the footguns available through containers, which I don't say a wholesale condemnation of the technology, but certainly as a UX facet that could use reevaluation.

So file contents are uploaded for embedding/indexing, but supposedly none of those contents are stored at Cursor after embedding.

It took me less than 5 minutes to completely disable... nobody cared, they just kept going - check the box and move on.

We live in an age, where the mere thought of personal Responsibility is terrifying.

The way I look at it is similar to how I'd look at any hypothetical employee. How do I ensure the agent only has access to the minimum possible they need to get their job done?

That means no access to git repositories (no pushes on my behalf means it can't accidentally nuke git history, something there is anecdotal evidence of agents doing). It can make local changes in git only and I will take responsibility for pushing them. No access to the wider internet beyond what I deem acceptable. No permissions to access any internal APIs except what I provide (and not using my credentials).

In one case, I have a tool that has a set of dangerous commands alongside a large number of safe ones. I don't even have it installed in the agent's VM. I run an MCP that is a simple python wrapper around the tool on the host side, and expose it to the agent in the VM, so that it can only possibly run a strict safe subset that I can trust it with access.

There currently is no path towards profitability, it's just a question of whether you can grab funding before it dries up.

Sorry if it's a dumb question - I'm just getting into sandboxing for the first time myself and ran into this same thought before.

Software "engineering" in particular has always been more than 50% cargo culting. Good engineering practices never matter when the alternative is just going through the motions of whatever rituals are in vogue.

A user account is a sandbox.

It’s kind of wild to watch in real time, instead of over the decades it took society for SA.

Viable AI companies will one get to profit. Firms layoff qualified people because AI is much cheaper. Now there's no easy turning back, so if AI service increase prices by 50%, well you don't have option.

> I knew two guys with sub 2.5 GPAs in college,

Gah! The horror! They should have been sent to the labor camps.

Do Anthropic users consider themselves clever enough to not make the same mistakes as Microsoft?

Edit: it’s about the attack surface