demeter: Minimal microVM host

{
  imports = [
    ./blastoise.nix
    ./squirtle.nix
    ./wartortle.nix
  ];

  networking.hostName = "demeter";
  networking.hostId = "044faaed";
  nixpkgs.hostPlatform = "x86_64-linux";
  system.stateVersion = "25.05";

  <<config>>

  _class = "nixos";
}

demeter is a Dell OptiPlex 7050 tower with 16GiB of RAM. It exclusively runs VMs that are redundant with VMs running elsewhere for high availability.

demeter used to serve as the NAS for my network, but once apollo was rebuilt with modern hardware and a larger rack-mounted case, it was clear that would be the better machine to hold multiple HDDs.

boot.initrd.availableKernelModules = [
  "xhci_pci"
  "ahci"
  "nvme"
  "usbhid"
  "sd_mod"
];
boot.kernelModules = [ "kvm-intel" ];
hardware.enableRedistributableFirmware = true;

This comes from nixos-generate-config based on detected hardware in the machine.

fileSystems."/nix" = {
  device = "fast/crypt/nix";
  fsType = "zfs";
};
fileSystems."/var" = {
  device = "fast/crypt/var";
  fsType = "zfs";
};

mjm.zfs.enable = true;
mjm.zfs.arcMax = 4 * 1024;
mjm.zfs.encryption.rootPool = "fast";

demeter uses ZFS. For important filesystems that need to be mounted in stage 1, I set mountpoint=legacy and mount them with fstab. I set a lower maximum for the ARC cache because demeter only has 16G of RAM. demeter is using ZFS native encryption with support from my ZFS module.

fileSystems."/persist" = {
  device = "fast/crypt/persist";
  fsType = "zfs";
  neededForBoot = true;
};

mjm.state.persistDir = "/persist";
system.etc.overlay.mutable = false;

demeter is using an ephemeral root filesystem, so / is a tmpfs and things that must be persisted are bind-mounted from /persist. The use of an ephemeral root also makes it easier to adopt the immutable etc overlay.

boot.zfs.extraPools = [ "slow" ];

demeter also has a spinning disk in it, which is used for local backups of VM data.

fileSystems."/boot" = {
  device = "/dev/disk/by-partlabel/boot";
  fsType = "vfat";
  options = [
    "fmask=0077"
    "dmask=0077"
  ];
};

mjm.secureboot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
boot.lanzaboote.measuredBoot.enable = false;

demeter uses my secureboot module to set up Lanzaboote for SecureBoot. It can't use measured boot, because the TPM in this machine doesn't support the necessary operation.

Lanzaboote is built on systemd-boot, so it needs the EFI system partition mounted at /boot.

swapDevices = [
  {
    device = "/dev/disk/by-partlabel/swap";
    randomEncryption = true;
  }
];
boot.zswap.enable = true;

demeter also has a swap partition, so it can use zswap to compress memory. Since hibernation doesn't need to be supported, it's pretty free to use the randomEncryption option in NixOS, which creates an emphemeral LUKS volume for the swap device, allowing swapped memory to be encrypted.

mjm.server.enable = true;

This machine is running as a server, so it needs the common server infrastructure for my lab.

mjm.spire.agent.tpm.publicKeyHash =
  "e6da25378860a50835cf2acf8a2dedb922d7e8fdcb199b4219377e7d71f53702";

Any of my machines that has a TPM uses it to attest to the SPIRE server and get its node identity. The public key hash here is specified only to create a registration entry that aliases this public key identity to the hostname of the machine.

mjm.microvm-host.enable = true;
mjm.microvm-host.zfsPrefix = "fast/crypt";
mjm.networkd.secondaryLinkName = "lan1";
mjm.networkd.macvlan.enable = true;

demeter's sole purpose is to host microVMs, with their data living in the fast/crypt/microvms dataset. All microVM hosts use MACVLAN to provide network devices to the VMs, and they all do so with a dedicated secondary NIC named lan1 (as opposed to lan0, which is reserved for the host).

mjm.remote-builder.enable = true;

My microVM hosts also do double duty as remote builders for CI, specifically for running NixOS VM tests.

mjm.nut = {
  enable = true;
  connectedUPSName = "smart500";
};

All physical hosts in my homelab run NUT to be able to be shutdown properly in the case of power loss. Since I have two UPSes in the rack, each machine is configured with the one it is connected to, so that will be the one that it monitors.

microVMs

=> blastoise
=> squirtle
=> wartortle