Things Unix can do atomically (2010)

You can use `ln` atomicity for a simple, portable(ish) locking system: https://gist.github.com/pwillis-els/b01b22f1b967a228c31db3cf...

Missing (probably because of the date of the article): `mv --exchange` aka renameat2+RENAME_EXCHANGE. It atomically swaps 2 file paths.

Unless they can be guaranteed by the POSIX specification, they are implementation specific and should not be relied upon for portable code.

I use several of these to implement alternative SQLite locking protocols.

POSIX file locking semantics really are broken beyond repair: https://news.ycombinator.com/item?id=46542247

The mmap/msync one is incorrect I believe? (Correct me if I am wrong).

msync() sync content in memory back to _disk_. But multiple processes mapping the same file always see the same content (barring memory consistency, caching, etc.) already. Unless the file is mapped with MAP_PRIVATE.

>fcntl(fd, F_GETLK, &lock), fcntl(fd, F_SETLK, &lock), and fcntl(fd, F_SETLKW, &lock)

There's also `flock`, the CLI utility in util-linux, that allows using flocks in shell scripts.

Nice catalog. One subtle thing I’ve found in building deterministic, stateless systems is that atomic filesystem and memory operations are the only way to safely compute or persist secrets without locks. Combining rename/link/O_EXCL patterns with ephemeral in-memory buffers ensures that sensitive data is never partially written to disk, which reduces race conditions and side-channel exposure in multi-process workflows.

Even though it can do some things atomically, it only does with one file at a time, and race conditions are still possible because it only does one operation at a time (even if you are only need one file). Some of these are helpful anyways, such as O_EXCL, but it is still only one thing at a time which can cause problems in some cases.

What else it does not do is a transaction with multiple objects. That is why, I would design a operating system, that you can do a transaction with multiple objects.

rename() is certainly the easiest to use for any sort of file-system based synchronization.

Not much apparently, although I didn't know about changing symlinks, that could be very useful.

I wonder why the author left out atomic writes with O_APPEND.

Anywhere there is atomic capability you can build a queuing application.

Not technically related to atomicity, but I was looking for a way to do arbitrary filesystem operations based on some condition (like adding a file to a directory, and having some operation be performed on it). The usual recommendation for this is to use inotify/watchman, but something about it seems clunky to me. I want to write a virtual filesystem, where you pass it a trigger condition and a function, and it applies the function to all files based on the trigger condition. Does something like this exist?

This document being from 2010 is, of course, missing the C11/C++11 atomics that replaced the need for compiler intrinsics or non portable inline asm when "operating on virtual memory".

With that said, at least for C and C++, the behavior of (std::)atomic when dealing with interprocess interactions is slightly outside the scope of the standard, but in practice (and at least recommended by the C++ standard) (atomic_)is_lock_free() atomics are generally usable between processes.

Sorry, there is zero chance I will ever deploy new code by changing a symlink to point to the new directory.

I use several of these to implement alternative SQLite locking protocols.

POSIX file locking semantics really are broken beyond repair: https://news.ycombinator.com/item?id=46542247

Missing (probably because of the date of the article): `mv --exchange` aka renameat2+RENAME_EXCHANGE. It atomically swaps 2 file paths.

I tried using this a while back and found it was not widely available. You need coreutils version 9.1 or later for this, many distros do not ship this.

I made https://github.com/rubenvannieuwpoort/atomic-exchange for my usecase.

Title says Unix, renameat2 is Linux-only.

You can use `ln` atomicity for a simple, portable(ish) locking system: https://gist.github.com/pwillis-els/b01b22f1b967a228c31db3cf...

Really nice explanation of a useful pattern. I was surprised to discover that even the famously broken NFS honours atomicity of hardlink creation.

Unless they can be guaranteed by the POSIX specification, they are implementation specific and should not be relied upon for portable code.

Which of these are not guaranteed by the POSIX specification? It’s been a while since I studied it, but if I recall correctly the ones mentioned in the article are guaranteed.

The mmap/msync one is incorrect I believe? (Correct me if I am wrong).

Yeah I agree that one isn't very clear, perhaps the idea is to use `msync()` as a barrier to achieve consistent ordering of the writes without having to handle that yourself with more complex primitives. But then, they do mention some of those primitives at the bottom of the article, so it's hard to say what exactly the idea is.

mmap/msync is behavior is also very platform specific. On some systems (like AIX, at least older versions), even without msync, memory mapped data is synced back to disk periodically.

I worked on a code base that was portable between Linux, AIX, and some other Unix flavors. mmap/msync was a source of bugs. Just imagine your system running for days, never syncing any data to disk... then someone pulls the plug. Where'd my data go? Even worse, it happened "in production" at a beta site. Fortunately we had a way to recover data from a log.

I tried using this a while back and found it was not widely available. You need coreutils version 9.1 or later for this, many distros do not ship this.

I made https://github.com/rubenvannieuwpoort/atomic-exchange for my usecase.

Which of these are not guaranteed by the POSIX specification? It’s been a while since I studied it, but if I recall correctly the ones mentioned in the article are guaranteed.

Really nice explanation of a useful pattern. I was surprised to discover that even the famously broken NFS honours atomicity of hardlink creation.

mmap/msync is behavior is also very platform specific. On some systems (like AIX, at least older versions), even without msync, memory mapped data is synced back to disk periodically.

>fcntl(fd, F_GETLK, &lock), fcntl(fd, F_SETLK, &lock), and fcntl(fd, F_SETLKW, &lock)

There's also `flock`, the CLI utility in util-linux, that allows using flocks in shell scripts.

What are flocks in this context? Surely not a number of sheep...

In UNIX/POSIX file locks are advisory, not enforced, it only works if all processes play ball.

Aren’t flock and POSIX locks backed by totally different systems?

Title says Unix, renameat2 is Linux-only.

What else it does not do is a transaction with multiple objects. That is why, I would design a operating system, that you can do a transaction with multiple objects.

>Title says Unix,

You're misinterpreting the title. The author didn't intend "Unix" to literally mean only the official AT&T/TheOpenGroup UNIX® System to the exclusion of Linux.

The first sentence of "UNIX-like" makes that clear : >This is a catalog of things UNIX-like/POSIX-compliant operating systems can do atomically,

Further down, he then mentions some Linux specifics : >fcntl(fd, F_GETLK, &lock), fcntl(fd, F_SETLK, &lock), and fcntl(fd, F_SETLKW, &lock) . [...] There is a “mandatory locking” mode but Linux’s implementation is unreliable as it’s subject to a race condition.

Windows had APIs for this sort of thing added in Vista, but they're now deprecating it "due to its complexity and various nuances which developers need to consider":

https://learn.microsoft.com/en-us/windows/win32/fileio/about...

In some cases, you can start by using the "at" functions (openat...) to work on a directory tree. If you have your logical "locking" done at the top-level of the tree, it might be a fine option.

In some other cases, I've used a pattern where I used a symlink to folders. The symlink is created, resolved or updated atomically, and all I need is eventual consistency.

That last case was to manage several APT repository indices. The indices were constantly updated to publish new testing or unstable releases of software and machines in the fleet were regularly fetching the repository index. The APT protocol and structure being a bit "dumb" (for better or worse) requires you to fetch files (many of them) in the reverse order they are created, which leads to obvious issues like the signature is updated only after the list of files is updated, or the list of files is created only after the list of packages is created.

Long story short, each update would create a new folder that's consistent, and a symlink points to the last created folder (to atomically replace the folder as it was not possible to swap them), and a small HTTP server would initiate a server side session when the first file is fetched and only return files from the same index list, and everything is eventually consistent, and we never get APT complaining about having signature or hash mismatches. The pivotal component was indeed the atomicity of having a symlink to deal with it, as the Java implementation didn't have access to a more modern "openat" syscall, relative to a specific folder.

I don't follow, sorry. Are you saying that if we run:

    mv a b
    mv c d

We could observe a state where a and d exist? I would find such "out of order execution" shocking.

If that's not what you're saying, could you give an example of something you want to be able to do but can't?

Not much apparently, although I didn't know about changing symlinks, that could be very useful.

Anywhere there is atomic capability you can build a queuing application.

The challenge with that approach is memory: trigger conditions, if added irresponsibly, can result in unbounded memory and (depending on implementation) potentially linear performance degradation of filesystem operations as well. Unbounded kernel memory growth leads to stability or security risks.

That tradeoff is at the root of why most notify APIs are either approximate (events can be dropped) or rigidly bounded by kernel settings that prevent truly arbitrary numbers of watches. fanotify and some implementations of kqueue are better at efficiently triggering large recursive watches, but that’s still just a mitigation on the underlying memory/performance tradeoffs, not a full solution.

Sounds half baked. What context does this function run in? Is it an interpreted language or an executable that you provide?

Inotify is the way to shovel these events out of the kernel, then userspace process rules apply. It's maybe not elegant from your pov, but it's simple.

I've used FUSE for something similar.

There are sample "drivers" in easily-modified python that are fast enough for casual use.

are you asking for if statements?

if(condition) {do the thing;}

I wonder why the author left out atomic writes with O_APPEND.

rename() is certainly the easiest to use for any sort of file-system based synchronization.

Sorry, there is zero chance I will ever deploy new code by changing a symlink to point to the new directory.

Aren’t flock and POSIX locks backed by totally different systems?

In some cases, you can start by using the "at" functions (openat...) to work on a directory tree. If you have your logical "locking" done at the top-level of the tree, it might be a fine option.

In some other cases, I've used a pattern where I used a symlink to folders. The symlink is created, resolved or updated atomically, and all I need is eventual consistency.

This requires O_SYNC and O_DIRECT afaik.

Even then it is only some file systems that guarantee it and even then file size updating isn’t atomic afaik.

Not so sure about file size update being atomic in this case but fairly sure about the rest.

Matklad had some writing or video about this.

Also there is a tool called ALICE and authors of that tool have a white paper about this subject.

Also there was a blog post about how badger database fixed some issues around this problem.

Unsure. Aren’t there filesystems which make O_APPEND less durable than it’s specified to be, which might be interpreted to adversely affect atomicity? Could that be it?

As long as you don't run into or want freedom from possible path races, for that you need the missing:

  frenameat2(srcdirfd, srcfd, srcname, dstdirfd, dstfd, dstname)

I don't do devops/sysadmin anymore, so this would have been before the age of k8s for everything. But I once interviewed for a company hiring specifically because their deployment process lasted hours, and rollbacks even longer.

In the interview when they were describing this problem, I asked why the didn't just put all of the new release in a new dir, and use symlinks to roll forward and backwards as needed. They kind of froze and looked at each other and all had the same 'aha' moment. I ended up not being interested in taking the job, but they still made sure to thank me for the idea which I thought was nice.

Not that I'm a genius or anything, it's something I'd done previously for years, and I'm sure I learned it from someone else who'd been doing it for years. It's a very valid deployment mechanism IMO, of course depending on your architecture.

Why? What do you prefer to do instead?

why? it works and its super clever. Simple command instead some shit written in JS with docker trash

Works pretty well for Nix

Then you are locking yourself out of a pretty much ironclad (and extremely cost-effective) way of managing such things.

Isn't that the standard way to do that? Why wouldn't you?

that's how some phone OSes update the system (by having 2 read only fs)

that's how Chrome updates itself, but without the symlink part

Nobody's saying you should deploy code with this, but symlinks are a very common filesystem locking method.

This document being from 2010 is, of course, missing the C11/C++11 atomics that replaced the need for compiler intrinsics or non portable inline asm when "operating on virtual memory".

That's right, atomic operations work just fine for memory shared between processes. I have worked on a commercial product that used this everywhere.

What are flocks in this context? Surely not a number of sheep...

In UNIX/POSIX file locks are advisory, not enforced, it only works if all processes play ball.

I don't follow, sorry. Are you saying that if we run:

    mv a b
    mv c d

We could observe a state where a and d exist? I would find such "out of order execution" shocking.

If that's not what you're saying, could you give an example of something you want to be able to do but can't?

Windows had APIs for this sort of thing added in Vista, but they're now deprecating it "due to its complexity and various nuances which developers need to consider":

https://learn.microsoft.com/en-us/windows/win32/fileio/about...

Sounds half baked. What context does this function run in? Is it an interpreted language or an executable that you provide?

Inotify is the way to shovel these events out of the kernel, then userspace process rules apply. It's maybe not elegant from your pov, but it's simple.

I've used FUSE for something similar.

There are sample "drivers" in easily-modified python that are fast enough for casual use.

As long as you don't run into or want freedom from possible path races, for that you need the missing:

  frenameat2(srcdirfd, srcfd, srcname, dstdirfd, dstfd, dstname)

Unsure. Aren’t there filesystems which make O_APPEND less durable than it’s specified to be, which might be interpreted to adversely affect atomicity? Could that be it?

Then you are locking yourself out of a pretty much ironclad (and extremely cost-effective) way of managing such things.

Isn't that the standard way to do that? Why wouldn't you?

Nobody's saying you should deploy code with this, but symlinks are a very common filesystem locking method.

Sure, but the discussion is around whether they’re atomic, not whether they’re advisory.

Depending on metadata cache behavior configuration, if the system is powered off immediately after the first command, then that could indeed happen I think.

As to whether it’s technically possible for it to happen on a system that stays on, I’m not sure, but it’s certainly vanishingly rare and likely requires very specific circumstances—not just a random race condition.

I don't think that's happening in practice, but 1) it may not be specified and 2) What you say could well be the persisted state after a machine crash or power loss. In particular if those files live in different directories.

You can remedy 2) by doing fsync() on the parent directory in between. I just asked ChatGPT which directory you need to fsync. It says it's both, the source and the target directory. Which "makes sense" and simplifies implementations, but it means the rename operation is atomic only at runtime, not if there's a crash in between. It think you might end up with 0 or 2 entries after a crash if you're unlucky.

If that's true, then for safety maybe one should never rename across directories, but instead do a coordinated link(source, target), fsync(target_dir), unlink(source), fsync(source_dir)

All you need for this to occur is the window where both renames occurs overlap. A system polling to check if a, b, c, and d exist while the renames are happening might find all four of them.

I'm almost certain what the OP meant was if the commands were run synchronously (ie: from 2 different shells or as `mv a b &; mv c d`) yes there is a possibility that a and d exist (eg: On a busy system where neither of the 2 commands can be immediately scheduled and eventually the second one ends up being scheduled before the first)

Or to go a level deeper, if you have 2 occurrences of rename(2) from the stdlibc ...

rename('a', 'b'); rename('c', 'd');

...and the compiler decides on out of order execution or optimizing by scheduling on different cpus, you can get a and d existing at the same time.

The reason it won't happen in the example you posted is the shell ensures the atomicity (by not forking the second mv until the wait() on the first returns)

>Title says Unix,

You're misinterpreting the title. The author didn't intend "Unix" to literally mean only the official AT&T/TheOpenGroup UNIX® System to the exclusion of Linux.

The first sentence of "UNIX-like" makes that clear : >This is a catalog of things UNIX-like/POSIX-compliant operating systems can do atomically,

Bit rot alert: Linux doesn't even have mandatory file locks these days.

Linux-specific open file description locks could be brought up in a modern version of TFA though.

They aren’t misinterpreting the title, the title is incorrect.

But I also don't think the auther meant Things you can do in Linux but not Unix

Except POSIX doesn't specify some of them as happening atomically.

Many people write UNIX/POSIX without ever reading what it says.

Sounds like the key term then is probably this:

> POSIX-compliant

Which, FWIW, doesn't mean Linux. AFAIK there is no Linux distro that's fully compliant, even before you worry about the specifics of whether it's certified as compliant.

are you asking for if statements?

if(condition) {do the thing;}

I know this is trivial to do programmatically, but I was looking for a way this will be handled by the filesystem. For instance, if I have some processes generating log files, and I have a script that converts them to html, I wanted the script to be called every time a log file is updated, without having a daemon running in the background to monitor the directory, just some filesystem mount. This would have made some deployments easier.

This requires O_SYNC and O_DIRECT afaik.

Even then it is only some file systems that guarantee it and even then file size updating isn’t atomic afaik.

Not so sure about file size update being atomic in this case but fairly sure about the rest.

Matklad had some writing or video about this.

Also there is a tool called ALICE and authors of that tool have a white paper about this subject.

Also there was a blog post about how badger database fixed some issues around this problem.

Why? What do you prefer to do instead?

Works pretty well for Nix

why? it works and its super clever. Simple command instead some shit written in JS with docker trash

that's how some phone OSes update the system (by having 2 read only fs)

that's how Chrome updates itself, but without the symlink part

That's right, atomic operations work just fine for memory shared between processes. I have worked on a commercial product that used this everywhere.

Thanks, I didn't find this when I was looking for a solution for my problem. This is pretty much the exact solution for my usecase, though for some reason inotify feels more complicated than some kind of filesystem mount solution for me.

I don't think any part of your post is right. Aside from NFS, there should not be filesystems where this doesn't work. If there are, those are just bugs. The flags you mentioned are not required or relevant. Setting the fd offset to the end of the file atomically is the entire purpose of O_APPEND.

Anything less than an entire new k8s cluster and switching over is just amateur hour obviously

Worked pretty well in production systems, serving huge amount of RPS (like ~5-10k/s) running on a LAMP stack monolith in five different geographical regions.

Just git branch (one branch per region because of compliance requirements) -> branch creates "tar.gz" with predefined name -> automated system downloads the new "tar.gz", checks release date, revision, etc. -> new symlink & php (serverles!!!) graceful restart and ka-b00m.

Rollbacks worked by pointing back to the old dir & restart.

Worked like a charm :-)

And for Stow[1] before it, and for its inspiration Depot[2] before even that. It’s an old idea.

[1] https://www.gnu.org/software/stow/

[2] http://ftp.gregor.com/download/dgregor/depot.pdf

Ah, the memories of capistrano, complete with zero-downtime unicorn handover

https://github.com/capistrano/capistrano/

No snapshotting at all? Thinking about it.. The filesystem does not support it I suppose.

not surprised about the chrome part, but pretty shocked at the phone OS part. I know APFS migration was done in this way, but wouldn't storage considerations for this be massive?

Sure, but the discussion is around whether they’re atomic, not whether they’re advisory.

Or to go a level deeper, if you have 2 occurrences of rename(2) from the stdlibc ...

rename('a', 'b'); rename('c', 'd');

...and the compiler decides on out of order execution or optimizing by scheduling on different cpus, you can get a and d existing at the same time.

The reason it won't happen in the example you posted is the shell ensures the atomicity (by not forking the second mv until the wait() on the first returns)

nitpick, it should be `touch a c & mv a b & mv c d` as `&;` returns `bash: syntax error near unexpected token `;'`. I always find this oddly weird, but that would not be the first pattern in BASH that is.

`inotifywait` actually sees them in order, but nothing ensure that it's that way.

  $ inotifywait -m /tmp
  /tmp/ MOVED_FROM a
  /tmp/ MOVED_TO b
  /tmp/ MOVED_FROM c
  /tmp/ MOVED_TO d

`stat` tells us that the timestamps are equal as well.

  $ stat b d | grep '^Change'
  Change: 2026-02-06 12:22:55.394932841 +0100
  Change: 2026-02-06 12:22:55.394932841 +0100

However, speeding things up changes it a bit.

Given

  $ (
    set -eo pipefail
    for i in {1..10000}
    do
      printf '%d ' "$i"
      touch a c
      mv a b &
      mv c d &
      wait
      rm b d
    done
  )
  1 2 3 4 5 6 .....

And with `inotifywait` I saw this when running it for a while.

  $ inotifywait -m -e MOVED_FROM,MOVED_TO /tmp > /tmp/output
  cat /tmp/output | xargs -l4 | sort | uniq -c
  9104 /tmp/ MOVED_FROM a /tmp/ MOVED_TO b /tmp/ MOVED_FROM c /tmp/ MOVED_TO d
  896 /tmp/ MOVED_FROM c /tmp/ MOVED_TO d /tmp/ MOVED_FROM a /tmp/ MOVED_TO b

Depending on metadata cache behavior configuration, if the system is powered off immediately after the first command, then that could indeed happen I think.

Uhh, if the system powers off immediately after the first command (mv a b), the second command (mv c d) would never run. So where would d come from if the command that created it never executed?

If that's true, then for safety maybe one should never rename across directories, but instead do a coordinated link(source, target), fsync(target_dir), unlink(source), fsync(source_dir)

why is this being downvoted? If there's something wrong, explain?

All you need for this to occur is the window where both renames occurs overlap. A system polling to check if a, b, c, and d exist while the renames are happening might find all four of them.

Assuming that the two `mv` commands are run in sequence, there shouldn't be any possibility for a and d to be observed "at once" (i.e. first d and then afterwards still a, by a single process).

They aren’t misinterpreting the title, the title is incorrect.

Bit rot alert: Linux doesn't even have mandatory file locks these days.

Linux-specific open file description locks could be brought up in a modern version of TFA though.

Except POSIX doesn't specify some of them as happening atomically.

Many people write UNIX/POSIX without ever reading what it says.

>, the title is incorrect.

Differing philosophies of how to interpret titles. Prescriptive vs Descriptive language.[0]

There can be different usages of the word "Unix":

#1: Unix is a UNIX(tm) System V descendent. More emphasis that the kernel needs to be UNIX. In this strict definition, you get the common reminder that "Linux is not a Unix!"

#2: "Unix" as a loose generic term for a family of o/s that looks/feels like Unix. This perspective includes using an o/s that has userland Unix utilities like cat/grep/awk. Sometimes deliberately styled as asterisk "*nix" or a suffix-qualifier "Unix-like" but often just written as a naked "Unix".

A Prescriptivist says the author's title is "incorrect". On the other hand, a Descriptivist looks at the whole content of the article -- notices the text has a lot of Linux specific info such as fcntl(,F_GETLEASE/F_SETLEASE), and every hyperlink to a man page url points to https://linux.die.net/man/ , etc -- and thus determines that the author is using "Unix"(#2) in the looser way that can include some Linux idiosyncrasies.

"Unix" instead of "*nix" as a generic term for Linux is not uncommon. Another example article where the authors use the so-called incorrect "Unix" in the title even though it's mostly discussing Linux CUPS instead of Solaris : https://www.evilsocket.net/2024/09/26/Attacking-UNIX-systems...

[0] https://en.wikipedia.org/wiki/Linguistic_prescription

But I also don't think the auther meant Things you can do in Linux but not Unix

>But I also don't think the auther meant Things you can do in Linux but not Unix

I wasn't claiming that. I just thought the ggp had a useful comment about renameat2() which led to gp's "correction" which wasn't 100% accurate.

IBM z/OS UNIX also has renameat2(). It doesn't have the Linux specific flag RENAME_EXCHANGE.

https://www.ibm.com/docs/en/zos/3.1.0?topic=functions-rename...

Sounds like the key term then is probably this:

> POSIX-compliant

Which, FWIW, doesn't mean Linux. AFAIK there is no Linux distro that's fully compliant, even before you worry about the specifics of whether it's certified as compliant.

>POSIX-compliant Which, FWIW, doesn't mean Linux. AFAIK there is no Linux distro that's fully compliant

I read author's use of "POSIX-compliant" as a loose and fuzzy family category rather than an exhaustive and authoritative reference on 100% strict compliance. Therefore, the author mentioning non-100%-compliant Linux is ok.

There seems to be 2 different expectations and interpretations of what the article is about.

- (1) article is attempting to be a strict intersection of all Unix-like systems that conform to official UNIX POSIX API. I didn't think this was a reasonable interpretation since we can't be sure the author actually verified/tested other POSIX-like systems such as FreeBSD, HP-UX, IBM AIX, etc.

- (2) article is a looser union of operating systems and can also include idiosyncracies of certain systems like Linux that the author is familiar with that don't apply to all other UNIX systems. I think some readers don't realize that all the author's citations to man pages point to Linux specific urls at : https://linux.die.net/man/

The ggp's (amstan) additional comment about renameat2(,,,,RENAME_EXCHANGE) is useful info and is consistent with interpretation (2).

If the author really didn't want Linux to be lumped in with "POSIX-like", it seems he would avoid linux.die.net and instead point to something more of a UNIX standard such as: https://unix.org/apis.html

[0] Intersection vs Union: https://en.wikipedia.org/wiki/Set_(mathematics)#Intersection

AFAIK you don't even want to be POSIX-compliant unless having a sticker means more to you than being reasonable. Most projects knowingly steer away from compliance (and certifying compliance is probably also expensive)

The slash is read as "OR" in this case.

As in: Unix-like OR POSIX-compliant

In that light, it's probably fine to not nitpick over certifications here.

EulerOS was certified UNIX some years ago.

Anything less than an entire new k8s cluster and switching over is just amateur hour obviously

Worked pretty well in production systems, serving huge amount of RPS (like ~5-10k/s) running on a LAMP stack monolith in five different geographical regions.

Rollbacks worked by pointing back to the old dir & restart.

Worked like a charm :-)

why is this being downvoted? If there's something wrong, explain?

`inotifywait` actually sees them in order, but nothing ensure that it's that way.

  $ inotifywait -m /tmp
  /tmp/ MOVED_FROM a
  /tmp/ MOVED_TO b
  /tmp/ MOVED_FROM c
  /tmp/ MOVED_TO d

`stat` tells us that the timestamps are equal as well.

  $ stat b d | grep '^Change'
  Change: 2026-02-06 12:22:55.394932841 +0100
  Change: 2026-02-06 12:22:55.394932841 +0100

However, speeding things up changes it a bit.

Given

  $ (
    set -eo pipefail
    for i in {1..10000}
    do
      printf '%d ' "$i"
      touch a c
      mv a b &
      mv c d &
      wait
      rm b d
    done
  )
  1 2 3 4 5 6 .....

And with `inotifywait` I saw this when running it for a while.

  $ inotifywait -m -e MOVED_FROM,MOVED_TO /tmp > /tmp/output
  cat /tmp/output | xargs -l4 | sort | uniq -c
  9104 /tmp/ MOVED_FROM a /tmp/ MOVED_TO b /tmp/ MOVED_FROM c /tmp/ MOVED_TO d
  896 /tmp/ MOVED_FROM c /tmp/ MOVED_TO d /tmp/ MOVED_FROM a /tmp/ MOVED_TO b

Assuming that the two `mv` commands are run in sequence, there shouldn't be any possibility for a and d to be observed "at once" (i.e. first d and then afterwards still a, by a single process).

>, the title is incorrect.

Differing philosophies of how to interpret titles. Prescriptive vs Descriptive language.[0]

There can be different usages of the word "Unix":

#1: Unix is a UNIX(tm) System V descendent. More emphasis that the kernel needs to be UNIX. In this strict definition, you get the common reminder that "Linux is not a Unix!"

[0] https://en.wikipedia.org/wiki/Linguistic_prescription

And for Stow[1] before it, and for its inspiration Depot[2] before even that. It’s an old idea.

[1] https://www.gnu.org/software/stow/

[2] http://ftp.gregor.com/download/dgregor/depot.pdf

I really liked stow. My toy distro back in the day was based on it.

Ah, the memories of capistrano, complete with zero-downtime unicorn handover

https://github.com/capistrano/capistrano/

Still use php deployer each day and works with symlinks as well. https://deployer.org/

It depends on what you mean by atomic. If it is only writing to page cache and you are writing a small amount then yes?

If there is a failure like a crash or power outage etc. then it doesn’t work like that.

You might as well be pushing into an in-memory data structure and writing to disk at program exit in terms of reliability

No snapshotting at all? Thinking about it.. The filesystem does not support it I suppose.

not surprised about the chrome part, but pretty shocked at the phone OS part. I know APFS migration was done in this way, but wouldn't storage considerations for this be massive?

Uhh, if the system powers off immediately after the first command (mv a b), the second command (mv c d) would never run. So where would d come from if the command that created it never executed?

>But I also don't think the auther meant Things you can do in Linux but not Unix

I wasn't claiming that. I just thought the ggp had a useful comment about renameat2() which led to gp's "correction" which wasn't 100% accurate.

IBM z/OS UNIX also has renameat2(). It doesn't have the Linux specific flag RENAME_EXCHANGE.

https://www.ibm.com/docs/en/zos/3.1.0?topic=functions-rename...

what would be more massive would be phones not booting up because of a botched update. this way you can just switch back to the old partition

Not really, because only the OS core is swapped in this way. Apps and data live in their own partitions/subvolumes, which are mutable and shared between OS versions.

The OS core is deployed as a single unit and is a few GB in size, pretty small when internal storage is into the hundreds of GB.

Er, sorry: I meant: if the first command runs, the plug is pulled, system starts again, second command runs.

In recent versions, z/OS has been copying lots of Linux-specific APIs (e.g. unshare [0]) in order to support the z/OS port of Kubernetes.

If Kubernetes starts using renameat2(RENAME_EXCHANGE), they could very plausibly add it.

[0] https://www.ibm.com/docs/en/zos/3.2.0?topic=csd-unshare-bpx1...

pedantic but z/OS isn't a unix, it can just pretend to be one enough for the open group to call it one. IBM has a unix still, AIX.

>POSIX-compliant Which, FWIW, doesn't mean Linux. AFAIK there is no Linux distro that's fully compliant

There seems to be 2 different expectations and interpretations of what the article is about.

The ggp's (amstan) additional comment about renameat2(,,,,RENAME_EXCHANGE) is useful info and is consistent with interpretation (2).

[0] Intersection vs Union: https://en.wikipedia.org/wiki/Set_(mathematics)#Intersection

The slash is read as "OR" in this case.

As in: Unix-like OR POSIX-compliant

In that light, it's probably fine to not nitpick over certifications here.

EulerOS was certified UNIX some years ago.

I really liked stow. My toy distro back in the day was based on it.

Still use php deployer each day and works with symlinks as well. https://deployer.org/

It depends on what you mean by atomic. If it is only writing to page cache and you are writing a small amount then yes?

If there is a failure like a crash or power outage etc. then it doesn’t work like that.

You might as well be pushing into an in-memory data structure and writing to disk at program exit in terms of reliability

You are projecting imaginary features onto O_APPEND and then hypothesizing that your imaginary features might not work.

POSIX says that for a file opened with O_APPEND "the file offset shall be set to the end of the file prior to each write." That's it. That's all it does.

Oh cool. I was a bit confused about not using snapshots and relying on symlinks but it couldn't be so simple. I guess it's just a simple userspace cow mount. https://source.android.com/docs/core/ota/virtual_ab#compress...

what would be more massive would be phones not booting up because of a botched update. this way you can just switch back to the old partition

Not really, because only the OS core is swapped in this way. Apps and data live in their own partitions/subvolumes, which are mutable and shared between OS versions.

The OS core is deployed as a single unit and is a few GB in size, pretty small when internal storage is into the hundreds of GB.

pedantic but z/OS isn't a unix, it can just pretend to be one enough for the open group to call it one. IBM has a unix still, AIX.

Er, sorry: I meant: if the first command runs, the plug is pulled, system starts again, second command runs.

Sure, but splitting "atomic" operations across a reboot is an interesting design choice. Surely upon reboot you would re-try the first `mv a b` before doing other things.

In recent versions, z/OS has been copying lots of Linux-specific APIs (e.g. unshare [0]) in order to support the z/OS port of Kubernetes.

If Kubernetes starts using renameat2(RENAME_EXCHANGE), they could very plausibly add it.

[0] https://www.ibm.com/docs/en/zos/3.2.0?topic=csd-unshare-bpx1...

You are projecting imaginary features onto O_APPEND and then hypothesizing that your imaginary features might not work.

POSIX says that for a file opened with O_APPEND "the file offset shall be set to the end of the file prior to each write." That's it. That's all it does.

Sure, but splitting "atomic" operations across a reboot is an interesting design choice. Surely upon reboot you would re-try the first `mv a b` before doing other things.

Things UNIX can do atomically 2010/01/06

This is a catalog of things UNIX-like/POSIX-compliant operating systems can do atomically, making them useful as building blocks for thread-safe and multi-process-safe programs without mutexes or read/write locks. The list is by no means exhaustive and I expect it to be updated frequently for the foreseeable future.

The philosophy here is to let the kernel do as much work as possible. At my most pessimistic, I trust the kernel developers more than a trust myself. More practically, it’s stupid to spend CPU time locking around an operation that’s already atomic. Added 2010-01-07.

Operating on a pathname

The operations below are best left to local filesystems. More than a few people have written in crying foul if any of these techniques are used on an NFS mount. True. When there are multiple kernels involved, the kernel can’t very well take care of all the locking for us. Added 2010-01-06.

mv -T <oldsymlink> <newsymlink> atomically changes the target of <newsymlink> to the directory pointed to by <oldsymlink> and is indispensable when deploying new code. Updated 2010-01-06: both operands are symlinks. (So this isn’t a system call, it’s still useful.) A reader pointed out that ln -Tfs <directory> <symlink> accomplishes the same thing without the second symlink. Added 2010-01-06. Deleted 2010-01-06: strace(1) shows that ln -Tfs <directory> <symlink> actually calls symlink(2), unlink(2), and symlink(2) once more, disqualifying it from this page. mv -T <oldsymlink> <newsymlink> ends up calling rename(2) which can atomically replace <newsymlink>. Caveat 2013-01-07: this does not apply to Mac OS X, whose mv(1) doesn’t call rename(2). mv(1).
link(oldpath, newpath) creates a new hard link called newpath pointing to the same inode as oldpath and increases the link count by one. This will fail with the error code EEXIST if newpath already exists, making this a useful mechanism for locking a file amongst threads or processes that can all agree upon the name newpath. I prefer this technique for whole-file locking because the lock is visible to ls(1). link(2).
symlink(oldpath, newpath) operates very much like link(2) but creates a symbolic link at a new inode rather than a hard link to the same inode. Symbolic links can point to directories, which hard links cannot, making them a perfect analogy to link(2) when locking entire directories. This will fail with the error code EEXIST if newpath already exists, making this a perfect analogy to link(2) that works for directories, too. Be careful of symbolic links whose target inode has been removed ("dangling" symbolic links) — open(2) will fail with the error code ENOENT. It should be mentioned that inodes are a finite resource (this particular machine has 1,245,184 inodes). symlink(2). Added 2010-01-07
rename(oldpath, newpath) can change a pathname atomically, provided oldpath and newpath are on the same filesystem. This will fail with the error code ENOENT if oldpath does not exist, enabling interprocess locking much like link(oldpath, newpath) above. I find this technique more natural when the files in question will be unlinked later. rename(2).
open(pathname, O_CREAT | O_EXCL, 0644) creates and opens a new file. (Don’t forget to set the mode in the third argument!) O_EXCL instructs this to fail with the error code EEXIST if pathname exists. This is a useful way to decide which process should handle a task: whoever successfully creates the file. open(2).
mkdir(dirname, 0755) creates a new directory but fails with the error code EEXIST if dirname exists. This provides for directories the same mechanism ~~link(2)~~ open(2) with O_EXCL provides for files. mkdir(2). Added 2010-01-06; edited 2013-01-07.

Operating on a file descriptor

fcntl(fd, F_GETLK, &lock), fcntl(fd, F_SETLK, &lock), and fcntl(fd, F_SETLKW, &lock) allow cooperating processes to lock regions of a file to serialize their access. lock is of type struct flock and describes the type of lock and the region being locked. F_SETLKW is particularly useful as it blocks the calling process until the lock is acquired. There is a “mandatory locking” mode but Linux’s implementation is unreliable as it’s subject to a race condition. fcntl(2).
fcntl(fd, F_GETLEASE) and fcntl(fd, F_SETLEASE, lease) ask the kernel to notify the calling process with SIGIO when another process opens or truncates the file referred to by fd. When that signals arrives, the lease needs to be removed by fcntl(fd, F_SETLEASE, F_UNLCK). fcntl(fd, F_NOTIFY, arg) is similar but doesn’t block other processes, so it isn’t useful for synchronization. fcntl(2).
mmap(0, length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0) returns a pointer from which a file’s contents can be read and written by normal memory operations. By making frequent use of msync(addr, length, MS_INVALIDATE), data written in this manner can be shared between processes that both map the same file. mmap(2), msync(2).

Operating on virtual memory

__sync_fetch_and_add, __sync_add_and_fetch, __sync_val_compare_and_swap, and friends provide a full barrier so “no memory operand will be moved across the operation, either forward or backward.” These operations are the basis for most (all?) lock-free algorithms. GCC Atomic Builtins.

Something I should add to my repertoire? Race condition? Let me know at r@rcrowley.org or @rcrowley and I’ll fix it.

Hacker Times