aion: IPv4 proxy VPS

{
  networking.hostName = "aion";
  networking.domain = "midna.dev";
  nixpkgs.hostPlatform = "x86_64-linux";
  system.stateVersion = "23.11";

  <<config>>

  _class = "nixos";
}

aion is a Hetzner VPS that serves a bridge to expose my services on a stable public IPv4 address. My Caddy VMs are accessible directly from the outside world via IPv6, but since I have a pretty typical home internet connection, I can't rely on having a stable IPv4 address.

For some time, I tried exposing things via IPv6 exclusively, but I found there were too many situations where that made things unusable for me or others. So I set up this basic VPS to proxy the traffic to the Caddy VMs. For publicly exposed services, my A records point to aion's public IP, and the AAAA records point to the Caddy VMs directly.

Originally I was doing this via haproxy, which was working well until I realized it wouldn't be a good solution if I wanted to support HTTP/3. Since that works via QUIC which in turn works on top of UDP, a TCP proxy wasn't going to cut it. haproxy can support QUIC, but it seems to expect to terminate TLS, which I want Caddy to handle instead.

A friend helped me figure out how to use a combination of ipxlat (hopefully a future addition to the mainline Linux kernel) and nftables to be able to forward all traffic on the relevant ports to the Caddy VMs, translating the incoming IPv4 packets to IPv6 packets. So far, this seems to be working well, though I will probably iterate on it a bit more to make it a little less janky.

boot.initrd.availableKernelModules = [
  "ahci"
  "xhci_pci"
  "virtio_gpu"
  "virtio_net"
  "virtio_pci"
  "virtio_scsi"
  "sd_mod"
  "sr_mod"
];
boot.initrd.kernelModules = [ "virtio_gpu" ];

This mostly comes from nixos-generate-config based on detected hardware in the machine. One difference is that nixos-generate-config will import the qemu-guest profile, and I've instead opted to just inline the modules from it that I believe I need in stage 1.

fileSystems."/" = {
  device = "/dev/disk/by-label/nixos";
  fsType = "ext4";
};

aion uses a pretty simple ext4 root filesystem. Perhaps I'll switch this to be an ephemeral root in the future.

fileSystems."/boot" = {
  device = "/dev/disk/by-label/boot";
  fsType = "vfat";
  options = [
    "fmask=0077"
    "dmask=0077"
  ];
};

boot.loader.limine = {
  enable = true;
  biosDevice = "/dev/sda";
  biosSupport = true;
  efiSupport = false;
};

aion is the only machine in the fleet that uses legacy boot instead of UEFI, because that's how Hetzner's x86 VPS machines work. Grub is terrible, so instead I'm using Limine as the bootloader. One nice thing about it is it uses a FAT partition to hold configuration, kernels, and initrds, just like a system booting with UEFI would. This at least makes it very familiar to how my other machines work, and it simplifies the bootloader because it doesn't have to understand all kinds of different filesystems.

swapDevices = [ { device = "/dev/disk/by-label/swap"; } ];
boot.zswap.enable = true;

aion has a swap partition, so it also gets to use zswap.

mjm.deploy.targetHost = "5.78.46.61";

My VPSes don't have local DNS records from my router, so I end up just targeting them explicitly by IP address.

systemd.network = {
  networks."10-primary-lan".enable = false;
  networks."10-wan" = {
    matchConfig.Name = "lan0";
    networkConfig.DHCP = "ipv4";
    address = [ "2a01:4ff:1f0:879b::1/64" ];
    routes = [ { Gateway = "fe80::1"; } ];
  };
};

It's very important that aion have working IPv6, since it exists to proxy IPv4 connections to machines that are only reachable via IPv6. Hetzner provides a /64 prefix with the machine, so I can just pick a /128 from within that and statically assign it to the machine. More of the space will be used for ipxlat shenanigans.

mjm.server.enable = true;
mjm.server.isLocal = false;

This machine is running as a server, so it needs the common server infrastructure for my lab. However, it is not running on my home network, so some things are disabled because of that.

mjm.ipv4-proxy.enable = true;

Finally, the ipv4-proxy service is the thing aion is there to run, so that is enabled.