Rootless is definitely the way to go. You can forward ports manually on the host if you really need to use privileged ports. I generally expose my containers through a reverse proxy, running bare metal on the host, and that completely bypasses the privileged port issue.

I get the savings of Distroless, but when you have an issue it's a pain to debug.

just curious why you'd bind to a priv port inside of a container.

Many such things have been built. Problem afaik is that the underlying hypervisor setup to support VMs isn't present, or isn't enabled by default, or is hard to figure out, on many machines. Plus you need nested virtualization when the thing you're using is a VM itself (also comes with possible pain). Docker runs because it has "ROAC" properties (Runs On Any Computer). That's not today true of VMs (of which micro-VMs, firecracker etc are a subset).

Maybe this here helps (I have not tried yet): https://github.com/virtkit-dev/virtkit

this has already been done.

runsc (gvisor)

Kata Containers (virtualization)

both drop in replacements for runc (default OCI runtime)

there are others too. one based on libkrun : https://github.com/containers/crun/blob/main/krun.1.md

and these are just runc replacements.

with some minor scripting you can roll a CLI solution using any of gvisor, cloud-hypervisor, firecracker, muvm (libkrun).

BETA: I have been following https://github.com/superradcompany/microsandbox a bit, very simple to get started with at least.

Containers and security are oxymorons. The flood of page cache cves (which can always be escalated/weaponized to an escape) from the other month is making deploying containers to prod untenable.

As for orchestration - a lot of folks think you need a completely new orchestration system for dealing with vms but we just simply re-use the existing infrastructure that already exists - the public clouds. Those companies have tens of thousands of engineers that are much better than the average engineer at this stuff, custom hardware, custom protocols and close to several decades of existing deployment.

I can build and ship a vm from my laptop/ci to prod on AWS/GCP in ~tens of second. Granted I come from the camp that thinks deploying full blown general purpose operating systems to prod is an increasingly incredibly risky practice.

I've been doing this for a while now and it is by far much easier to work with and just as safe if configured properly. I think people just cargo cult thier way into wanting rootless containers for their long running services with no real look into if it's actually needed.

Rootless containers have their place, actual userspace workloads. Things like dev containers or normal users running containers but if one is creating users to run system level services then they are just adding extra steps imo.

This whole privileged port thing is an outdated convention from the time when Linux ran on mainframes. Depending on your use case, it can be perfectly fine to lower it. I have set to `net.ipvX.ip_unprivileged_port_start = 80` on my server so that I can run rootless containers without extra privileges and have them bind to ports 80 and up.

lol the before times when Linux ran on mainframes.

It does run on IBM 370 type mainframes, but that port was done years after it first ran on the 386, and isn't in any way relevant to low port numbers.

Nits: It's a Unix convention (not just Linux), and Unix traditionally ran on minicomputers (not mainframes). Mainframes had their own specialized OSes, although some could run Unix in a partition or virtual machine.

I think that "Linux ran on mainframes" was said in jest. Even though Linux runs on mainframes pretty well even now.