My home network observes bedtime with OpenBSD and pf

> However, I ran into trouble with the RealTek ethernet hardware support in OpenBSD, which had been running fine with Linux for years.

I've run into problems with realtek gigE nics on Linux, FreeBSD, and Windows. I'm convinced their hardware/firmware has a timing issue where if the wrong things happen, the descriptor indexes get unsyncronized. This can lead to network stalls, but also wild writes. IIRC, reset behavior is weird too; vague because it's been a while since I looked, but I think if you get a network stall and do a reset, the card may receive and DMA a packet into RAM in the process ... something like that anyway.

I have systems where the FreeBSD base driver consistently stalls, but the realtek provided driver works mostly ok; but the realtek driver is full of undocumented flag setting, so who knows what it's doing... it also sets the NIC to emit pause frames when it runs out of RX buffers which I never want; things will be much better if packets are dropped when RX buffers are full.

I would love to have the equipment and time to figure out what's going on, but a) realtek probably should be the ones to do it, b) switching drivers usually works at no cost, and swapping to intel almost always works but you need slots and cards (ebay gets you multiport 1g for $10, 10g for $20-$30 though). I've heard realtek is good at 2.5g and intel isn't; but I haven't run enough realtek 2.5g to know.

Cool post, I love a good firewall story.

One suggestion though: rather than doing this all on a single LAN network and having to deal with adding exceptions for devices that still need access to the Internet during 'bedtime' periods, I suggest creating a separate VLAN for devices that need 'bedtime' enforcement and put those devices there, while leaving your 'always online' devices in your main VLAN where access to the Internet is always available. This way all you have to do is simply change your firewall rules for that VLAN to enforce bedtime, which removes the extra rules needed for exceptions.

Love your watercolors! What a fun addition to a technical article :)

The anchor-based approach for time-dependent rules is elegant. Most people would reach for a cron job that rewrites firewall rules on a schedule, but using pf anchors keeps the state management inside the packet filter where it belongs. The key advantage of pf over iptables for this kind of use case is that rule evaluation is deterministic and the syntax stays readable enough to audit six months later without documentation archaeology. Nice to see OpenBSD used for practical home network management instead of just theoretical security posturing.

Only allowing TCP will break a lot of stuff. I was wondering why even bother with the transport layer, instead of just focusing on IP directly

Fun article! I like your watercolors too, especially the one of them going into the pufferfish's mouth :D

Thank you for sharing! What are your thoughts on intentionally degrading service over the course of an hour instead of a hard cutoff? Like implementing an increasingly restrictive cap on download speeds/intentionally dropping a % of packets over the hour. Might be a little less jarring than a hard stop.

I tried something like this. my kids just turned off wifi and I can't control the cell signals. Phone parent controls are not powerful enough for what I need

Waiting for the “pf is garbage, use Linux” comments.

Yet another “I cant’t parent so I’ll show my kids what a surveillance state looks like” post.

Fun article! I like your watercolors too, especially the one of them going into the pufferfish's mouth :D

> However, I ran into trouble with the RealTek ethernet hardware support in OpenBSD, which had been running fine with Linux for years.

Only allowing TCP will break a lot of stuff. I was wondering why even bother with the transport layer, instead of just focusing on IP directly

Would be trivial to block new connections first, and kill existing ones a while later.

I tried something like this. my kids just turned off wifi and I can't control the cell signals. Phone parent controls are not powerful enough for what I need

I am just asking mines after dinner to leave the phone after dinner to my office where all electronic devices charge during the night.

THIS. Incredibly frustrating that iOS (I’m not sure about Android) doesn’t provide a way to disable mobile data or Personal Hotspot as part of Screen Time settings.

Redirect their DNS through pihole

Love your watercolors! What a fun addition to a technical article :)

Cool post, I love a good firewall story.

Yet another “I cant’t parent so I’ll show my kids what a surveillance state looks like” post.

I had to make something like this at work once (allow access during some hours deny it at other times. and they wanted it enforced at the packet level) and I was a new sys-admin and all I had was an old linux box with iptables. It was ending up as the normal iptables mess until I asked myself "self: how would pf do it?", "anchors. I said, pf would do it with anchors" and I did it that way. well as much as possible. It was by far my cleanest work with iptables.

It has been a while since I have had to mess with iptables but if I remember correctly(quickly reads the iptables man page) the equivalent to pf anchors in iptables is to use named chains then you can faf about loading and unloading the dynamic rules from the chains without messing with the static rules.

The whole thing really made me appreciate the design of pf. I think, strictly speaking, iptables is more capable than pf, but pf and openbsd in general, is far more ergonomic.

> The key advantage of pf over iptables for this kind of use case is that rule evaluation is deterministic and the syntax stays readable enough to audit six months later without documentation archaeology.

Is iptables not deterministic? Don't the packets look at each rule in numerical order until something matches? If you have two rules with the same number, shame on you.

Re archaeology, OpenBSD changed the rules syntax for some reason and the other platforms with pf kept the existing syntax, so that's always a fun game to play.

Not digging the desired outcome but yes the watercolors are great addition to an interesting technical article.

Me too! It was a fantastic addition that I would not have expected. I wish I was artistic enough to do something like that. It had the interesting technical content, with the coziness of a children's book. Really a great piece that the author should be proud of

This is also the approach I would have used - I was surprised the author didn't end up here. I used a separate VLAN to achieve same thing as author to shutdown internet access on the VLAN my kids devices use at bedtime, as well as another VLAN with no internet access at all for IoT devices, security cameras etc.

Blocking all UDP traffic by default is something I would never have even attempted for a domestic setup either. As the author discovers with Discord and Roblox, a great many common applications and games rely upon it. A UDP block on my kid's VLAN would last about 5 seconds before they attacked me for breaking their online Minecraft games.

The next(I think? It's in -CURRENT now anyway.) version of OpenBSD will be adding VLAN awareness to veb(4). Should make my OpenBSD home router experience much easier.

Waiting for the “pf is garbage, use Linux” comments.

THIS. Incredibly frustrating that iOS (I’m not sure about Android) doesn’t provide a way to disable mobile data or Personal Hotspot as part of Screen Time settings.

Would be trivial to block new connections first, and kill existing ones a while later.

I am just asking mines after dinner to leave the phone after dinner to my office where all electronic devices charge during the night.

I usualy do that but I sometimes forget.

Redirect their DNS through pihole

That will not fix changing networks which will change your dns provider.

Move to a house with exterior stucco. Faraday cage built in!

Not digging the desired outcome but yes the watercolors are great addition to an interesting technical article.

The whole thing really made me appreciate the design of pf. I think, strictly speaking, iptables is more capable than pf, but pf and openbsd in general, is far more ergonomic.

Is iptables not deterministic? Don't the packets look at each rule in numerical order until something matches? If you have two rules with the same number, shame on you.

Re archaeology, OpenBSD changed the rules syntax for some reason and the other platforms with pf kept the existing syntax, so that's always a fun game to play.

I think the issue is if you are dynamically updating the rules then you might have an intermediate state where some packages are processed according to some set of rules which is neither the set of rules at the start nor at the end. Wheras with anchors you can flip between different sets of rules atomically. (though I suspect you can do the same with iptables but it'll called something different)

The next(I think? It's in -CURRENT now anyway.) version of OpenBSD will be adding VLAN awareness to veb(4). Should make my OpenBSD home router experience much easier.

I usualy do that but I sometimes forget.

Move to a house with exterior stucco. Faraday cage built in!

That will not fix changing networks which will change your dns provider.

watercolor of puffy the fearsome openbsd mascot guarding a pathway that enters its open mouth

(Sketchbook ink and watercolor by the author: A fearsome Puffy determines which packets shall pass.)

The centerpiece of my setup is the pf packet filter, which is built into the OpenBSD kernel and originated, like many good things, from OpenBSD.

The bulk of pf configuration is done through /etc/pf.conf.

I constructed mine from scratch while reading The Book of PF, 4th Ed. (see the references section at the bottom of this page).

You can view my full conf in the repo here: pf.conf.

(I like to thoroughly document things I won’t be touching frequently, so there are a lot of comments in that file, including instructions on updating pf after I make changes.)

Anyway, I set this up in the recommended fashion: block all traffic and then let only selected traffic through.

When it’s daytime, I use the rule:

pass proto tcp from <leased_ips>

When it’s bedtime, that rule changes to:

pass proto tcp from <bedtime_exempt>

There are two IP address tables being used:

<leased_ips> is maintained by dhcpd when it leases addresses to clients on the local network.
<bedtime_exempt> is maintained manually by me. I store the addresses in a text file and load them into the table with a script whenever I make a change.

When it’s bedtime, I only explicitly allow traffic to the exempt computers. This blocks traffic to everything else because, as you may recall, the default is block all!

You’ll notice that I’m only doing this for TCP traffic. I’m handling ICMP and UDP packets in a strict fashion in accordance with the wisdom of the book. We’ll see if I end up needing to make any exceptions.

(Update: Sure enough, I’m going to need to experiment with the daytime rule - the above doesn’t allow Discord voice chat or Roblox to function, which…was not appreciated by certain members of this house.)

Updating tables

Since this is all predicated on the two address tables, how do these tables get updated?

The <leased_ips> table is initially created in pf.conf with this placeholder:

table <leased_ips> persist counters

It is populated automatically by dhcpd from this command line option set in /etc/rc.conf.local (also in the repo):

dhcpd_flags="-L leased_ips"

I think it’s great how tables are built right into the OpenBSD kernel and all the tooling understands them. It feels very cohesive and, dare I say it, planned and thought-out?

I store the <bedtime_exempt> addresses in a text file and update the table from the file contents with pfctl:

pfctl -t bedtime_exempt -T replace -f no_bedtime.txt

The text file is a simple list with one address per line. It can also have standard Unix-style comments (line starts with '#'). Again, all of this feels very cohesive and flexible to me. It’s the good parts of the Unix Philosophy.

When you or a program update a table, the changes take place immediately in the running kernel’s tables and you don’t have to tell pf about them.

Anchors

The crux of bedtime enforcement is the ability to schedule a change to the rules that allow traffic from local computers.

Anchors are a grouping for rules in pf.conf. There are a couple different uses for them, but in my case, I’m using an anchor as a named chunk of rules which I can change from the command line without having to reload anything else.

Initializing an anchor can be as simple as giving it a name:

anchor foo;

In my case, I’m pre-populating my 'bedtime' anchor with the unrestricted Internet access rule so access works when pf starts up with the assumption that it’s currently "daytime":

anchor bedtime { # the default "awake" rule, bedtime not enforced pass proto tcp from <leased_ips> }

Once you have an anchor, you can swap out its rules on the fly (they’ll be parsed and added to the ruleset) from a file or even STDIN at the command line. Here’s an example that uses echo to replace the rules of a 'foo' anchor:

echo "block all" | pfctl -a foo -f -

As soon as you replace an anchor’s rules, pf will start using them immediately.

By the way, if you made a mistake and have a syntax error in the rules you’re trying to load into an anchor, you’ll just get a syntax error and the rule won’t be loaded. Nothing bad happens and everything keeps running.

Kill active connections in the state table

One of the hardest problems I ran into was the fact that by default, pf keeps track of active connections and stores them in a state table. It keeps these connections alive even if a new rule would have forbidden it.

This is normally desirable for two reasons:

pf doesn’t have to expend cycles examining each packet that follows in the same connection.
It keeps existing connections (like SSH sessions!) alive even when rules change or address tables update.

However, in my case, I want bedtime to cut off traffic, especially long-lived connections like YouTube streams!

So when bedtime starts and I change the anchor rules, I’m also killing connections on the local network with pfctl:

pfctl -k 10.0.0.0/24

(Ideally, I would be able to kill connection states for only the entries in <leased_ips> minus the entries in <bedtime_exempt>. But I’ll admit, after reading the man pages front-and-back, searching the wasteland that is the modern Web, and even diving into the source code, I don’t see an easy way to do that. One possible avenue would be to use the ability to label pf rules and then kill connections by label, so maybe I can figure out how to craft a 'match' rule that uses the tables to get the correct list of connections to kill? It’s that or figure out the difference of the two tables in my script, which…no thanks.)

Lastly, I put all the above together in a shell script.