networkd configuration

#| file: modules/nixos/networkd.nix
{ config, lib, ... }:
let
  cfg = config.mjm.networkd;
in
{
  options.mjm.networkd = {
    <<options>>
  };

  config = lib.mkIf cfg.enable {
    <<config>>
  };

  _class = "nixos";
}

I use networkd to set up networking on any NixOS systems that don't have WiFi (in which case I use NetworkManager to be able to freely change networks more easily). I prefer to set my networkd configuration up explicitly, without using NixOS's networking.useNetworkd option that translates the various networking.* options to be backed by networkd.

My networkd configuration is a bit complex, primarily because it needs to optionally support using MACVLAN to provide networking for microVMs. I think the end result is pretty nice though.

There are three different network config structures I'm trying to support:

• Basic single ethernet connection.
• Two ethernet connections. One is used normally to access the machine, and the other is used exclusively as a MACVLAN bridge. This structure is used for VM hosts. Each microVM gets its own MACVTAP interface using this second link.
• Single ethernet connection with MACVLAN. This structure is sometimes used for my desktop, uranus, when I want to test VMs using MACVLAN without a second NIC. To allow communication between the host machine and the VMs, the host must create its own MACVLAN device to use as its main network connection, rather than using the link directly.

The complexity of this module is to be able to support all three of these.

Options

enable = lib.mkOption {
  type = lib.types.bool;
  default = !config.networking.networkmanager.enable;
};

networkd is enabled by default on any machine that doesn't have NetworkManager enabled. This should include all non-portable machines.

primaryLinkName = lib.mkOption {
  type = lib.types.str;
  default = "lan0";
};

The name of the primary link used to reach the machine. This defaults to "lan0" and generally isn't changed from that. As I'll show later, I have link rules set up in networkd that rename the various NICs in my machines so that the one intended to be the primary is always named lan0.

secondaryLinkName = lib.mkOption {
  type = with lib.types; nullOr str;
  default = null;
};

My VM hosts have a second NIC installed that is shared via a MACVLAN bridge with the VMs. To configure it, this option must be set to the name of an interface. Usually, this will be "lan1".

macvlan.enable = lib.mkOption {
  type = lib.types.bool;
  default = false;
};

Enables configuring a MACVLAN bridge. If secondaryLinkName is non-null, then that interface will be dedicated to MACVLAN, and the primary link will be used for this machine otherwise. If secondaryLinkName is null, then the primary link will be used, and an extra MACVLAN device will need to be created for the host to use as its primary interface.

primaryIface = lib.mkOption {
  type = lib.types.str;
  internal = true;
  readOnly = true;
  default = if cfg.secondaryLinkName == null && cfg.macvlan.enable then "mac0" else cfg.primaryLinkName;
};

The interface that this machine actually has an IP address on. The rest of the module will use this, as do some other unrelated modules; it is not expected to be set manually.

While there's three possible network shapes to consider, there's really only two cases here. If MACVLAN is enabled but there's only a single NIC for it, then we will have to create a "mac0" interface and use that as the primary. Otherwise, it's just the primary (or only) NIC.

bridgeParentName = lib.mkOption {
  type = lib.types.str;
  default = if cfg.secondaryLinkName == null then cfg.primaryLinkName else cfg.secondaryLinkName;
  readOnly = true;
};

When MACVLAN is enabled, this is the link that is being used for the bridge. If there's a secondary link defined, then it's that, otherwise it's the primary link.

Config

systemd.network.enable = true;
networking.useDHCP = false;

To use networkd without using NixOS's networking.useNetworkd option, you have to disable the default DHCP option. There's an assertion for it, so evaluation will fail.

systemd.network.networks."10-primary-lan" = {
  name = cfg.primaryIface;
  networkConfig = {
    DHCP = "ipv4";
    IPv6AcceptRA = true;
    IPv6PrivacyExtensions = false;
  };
  dhcpV4Config = {
    UseDomains = true;
  };
  dhcpV6Config = {
    UseDNS = false;
  };
};

The primary interface needs to be configured to use DHCP to get its IPv4 address and SLAAC to get IPv6 addresses. I assign IPv4 addresses in my router's configuration, so even machines with static IPs get them through DHCP. IPv6 privacy extensions are disabled because they make it hard to configure Consul to detect the stable v6 address.

UseDomains is enabled to make accessing my servers more convenient. The UseDNS setting for IPv6 is old enough that I don't actually remember its purpose. It might not be needed for my setup.

systemd.network.netdevs.mac0 = lib.mkIf (cfg.primaryIface != cfg.primaryLinkName) {
  netdevConfig.Name = cfg.primaryIface;
  netdevConfig.Kind = "macvlan";
  macvlanConfig.Mode = "bridge";
};

If we're doing MACVLAN without a second NIC, then we'll need to create the MACVLAN interface that will be used as the primary interface for the host.

systemd.network.networks."10-bridge-lan" = lib.mkIf cfg.macvlan.enable {
  name = cfg.bridgeParentName;
  networkConfig = {
    MACVLAN = lib.mkIf (cfg.primaryIface != cfg.primaryLinkName) cfg.primaryIface;
    IPv6AcceptRA = false;
    LinkLocalAddressing = false;
  };
  linkConfig.RequiredForOnline = "carrier";
};

If MACVLAN is being used then the bridge interface for that needs to be configured. Router advertisements and link-local addressing are disabled, to avoid the device getting IP addresses that it isn't meant to use. Because it won't have an IP, "carrier" is the most online this interface is going to get, so letting networkd know that means its online status

is more correct.

The MACVLAN option is set if needed to connect this MACVLAN bridge to the mac0 primary interface, if it was created. The microVMs will handle creating their MACVTAP devices on this link when they start up, so that doesn't need to be configured here.

systemd.network.links = {
  "10-virtio" = {
    matchConfig.Driver = "virtio_net";
    linkConfig.Name = "lan0";
  };
  "10-rpi" = {
    matchConfig.Driver = "bcmgenet";
    linkConfig.Name = "lan0";
  };
  "10-vm-bridge" = {
    matchConfig.Property = "ID_VENDOR_ID=0x10ec ID_MODEL_ID=0x8125";
    linkConfig.Name = "lan1";
  };
  "15-other-ethernet" = {
    matchConfig.Driver = "e1000e r8169";
    linkConfig.Name = "lan0";
  };
};

Finally, I add a little configuration to rename my network interfaces to follow a consistent scheme. It pleases a weird and obsessive part of me, and it also makes configuration across machines easier. The main NIC will always be "lan0", and a second NIC if present will always be "lan1". I got the same PCIe NIC for all three VM hosts, so I can match on the exact vendor and model to match that device. Otherwise, I match by driver to fix the NICs for VMs, Raspberry Pis, and otherwise.

Note that the last rule has a higher number prefix, so that the "lan1" rule matches first. The secondary NICs use the same Realtek driver that is included in that last rule. This rule is only meant to match the built-in Ethernet port on the motherboard, which for my machines always uses either the Intel driver or the Realtek one.