I’m fed up with my Ubiquiti routers. UnifiOS has a bunch of bugs related to WAN failover that have been there for a year+. Also a lot of mysterious behavior and some odd engineering problems. And still no official solution for Wireguard last I looked. So I want to switch (back) to OpenWRT for routing. I want three ethernet ports in the end; LAN, WAN, and backup WAN. I’m not using the WiFi on the RPi4 at all. I could, but I have hardware with better antennas.
These notes are a work in progress and are not a fully followable cookbook. It took me like 5 hours of tinkering in different orders to get this going. Don’t worry, it’s not normally that hard. Here are some other sources I found helpful for doing this: one, two, three.
Step 1: Hardware and device drivers
OpenWRT runs on all sorts of hardware but I’m going to do this with a Raspberry Pi 4. It’s way more powerful CPUs and RAM than a typical router. The networking is a bit weaker though. The RPi4 gives you one ethernet port on the PCIe bus and the rest have to be USB ports. The TP-Link UE300 is a commonly recommended USB adapter, it has an RTL-8053 which is well supported. My LAN will be on the on-board ethernet, the WAN links on USB. I think this setup is good enough to support gigabit throughput and is certainly good enough for Starlink’s max 200 Mbps. I’m told you can do traffic shaping and VPN stuff with very good speeds using the RPI4 CPU, but I haven’t tested it. (Few consumer routers can do gigabit Internet with these fancies.)
Another interesting hardware option is an RPi CM4 with a router board. That gives you two full Gigabit ethernet ports connected to the PCIe bus. A third option is to only have a single ethernet port and use VLAN tagging so the router can work in a “router on a stick” mode.
Step 2: Basic install
OpenWRT has pretty good docs for running on an RPi 4, I just followed them to get a basic system going. Note that the RPi 4 is way more powerful hardware than OpenWRT is normally aimed at, so some of the extreme things that OpenWRT does to save resources may not be necessary. Some folks recommend using ext4 as the filesystem instead of the compressed SquashFS. There’s also something about resizing the partition to use all the SD card storage. I didn’t bother.
Flash your SD card, then boot the router without the USB ethernet devices plugged in. (Or with, it doesn’t matter, but don’t plug in any WAN cables yet.) Plug a laptop ethernet into the on-board ethernet port. OpenWRT should come up and be serving DHCP on the ethernet. Point your laptop to http://192.168.1.1/ and you’re logged in to LuCI, the OpenWRT web interface. Note that OpenWRT will call this ethernet port eth0 and it will be bridged into br-lan.
Step 3: enabling your USB ethernet device
There’s one hard thing setting this up; OpenWRT doesn’t have the kernel drivers for USB ethernet installed by default. You need the kmod-mii, kmod-usb-net, and kmod-usb-net-rtl8152 opkg packages (IPK files) installed. No configuration needed, but you do have to install the packages. The problem is you’re not connected to the Internet to just install them.
You have several options for getting them installed. My choice was to manually download the needed packages from the OpenWRT repository to my laptop, then upload them via LuCI to the router. (You have to install them one at a time and click “Dismiss” manually on the install popup). It’s also possible to plug a WAN cable into the one working ethernet and get access to the router somehow (keyboard and screen? routing tricks?) to install them. I like the “2nd way” in the linked Reddit post; you make a custom OpenWRT image that included the necessary drivers.
Step 4: configuring the WAN
Now that there’s an ethernet device for the WAN you have to configure it. Do this in LuCI; go to Network / Interfaces and add an interface for eth1. Call it WAN and then go to firewall settings and put it in the WAN zone. This is a little mysterious but I think that’s all you have to do. I rebooted somewhere in here just to be sure that it was all working right.
Step 5: Improve OpenWRT
The blessing (and curse) of OpenWRT is it’s a very flexible Linux system and you can do a lot with it. A search for packages matching “luci-app-*” is a good way to see what’s available. Here’s some of what I’ve done.
Update the software lists and manually update each package one at a time. The updates aren’t exactly recommended but before I did that some new LuCI stuff I installed broke the old LuCI install until I updated it.
Install some quality of life packages: less, bash, curl, nano, mg (for me).
Install luci-app-statistics. You also have to enable this by going to Statistics / Setup and saving the config.
TODO / options
Some things I have not yet done but may get around to (and update this blog post if so)
See if I can export monitoring stats somehow to my InfluxDB instance for plotting in Grafana.
Install luci-app-sqm for traffic shaping
My own notes
Some notes of interest to me only.
The network interfaces are:
LAN eth0/br-lan E4:5F:01:5F:AB:D8
WAN1 eth1 5C:A6:E6:AA:BC:9A
Wireless not enabled
System / logging / external system log server to my Linux server on the LAN
DHCP range: 192.168.3.2 – 192.168.3.199
Override DNS servers and use 22.214.171.124 and 126.96.36.199
Set a static route for Starlink for 192.168.100.1 / 255.255.255.255 on eth1.
Some static leases (all have a lease time of 86400)
gvl 94:c6:91:1e:c8:38: .75
printer 84:25:19:0e:7b:0b: .67
pvspi0 e4:5f:01:78:d7:f9: .140