{
pkgs,
config,
lib,
nodes,
...
}:
let
cfg = config.mjm.ingress;
in
{
options.mjm.ingress = {
enable = lib.mkEnableOption "ingress";
<<options>>
};
config = lib.mkIf cfg.enable {
<<config>>
};
}To expose services running in my lab to the outside world, I use Caddy as a web server and reverse proxy. In my opinion, Caddy is the best general-purpose HTTP server out there right now. I used to use nginx for this job, but now I would choose Caddy any day.
mjm.services.caddy = {
secrets.enable = true;
};Caddy is going to need access to a secret in order to update DNS records in deSEC for ACME challenges.
services.caddy = {
enable = true;
package = pkgs.caddy-with-plugins;
settings = {
<<caddy-settings>>
};
};{ caddy }:
caddy.withPlugins {
plugins = [
"github.com/caddy-dns/desec@v1.1.0"
"github.com/mholt/caddy-l4@v0.1.1"
];
hash = "sha256-maz4dtFExE7g4t3iHSQgzUq2paZh/DIn1VYNTYBBEqE=";
}I use a custom Caddy package that includes some additional plugins. Thankfully, Nixpkgs makes it easy to build these.
I'm using the settings option in the NixOS Caddy module, which uses the JSON config structure rather than the more succinct Caddyfile syntax. I think it makes more sense to use a structured data format when generating a config this complex programmatically, and once you see everything this module does, I think you'll agree.
logging.logs.acme_debugging = {
level = "DEBUG";
include = [
"tls.obtain"
"http.acme_client"
];
};Enabling debug logging for these loggers makes it easier to diagnose issues around ACME challenges.
apps.tls = {
certificates.automate = [
"midna.dev"
"*.midna.dev"
"*.mattmoriarity.com"
];
automation.policies = [
{
issuers = [
{
module = "acme";
email = "acme@matt.mattmoriarity.com";
profile = "shortlived";
challenges.dns = {
propagation_delay = "3m";
propagation_timeout = "6m";
provider = {
name = "desec";
token = "{file./run/credentials/caddy.service/caddy_desec_api_token}";
};
};
}
];
}
];
};This configuration sets up Caddy's certificate automation for all of my sites. The fact that this is integrated into the server itself is one of the things I really like about Caddy: it's much nicer than having to run sidecar automation to manage certificates.
Almost every service I self-host in this homelab lives as a subdomain under midna.dev. Since I have so many, I think it makes more sense to use a single wildcard certificate for all of them, rather than needing to request individual certs for each one. By default, Caddy would not request a wildcard certificate to share between multiple virtual hosts, but it will request and use it correctly if you ask for it.
Wildcard certificates prevent using HTTP challenges, so I have to configure DNS challenges instead. My DNS provider is deSEC, which requires a plugin for Caddy (see above) and a token from Vault. I've created separate _acme-challenge zones for my domains, and restricted the API token Caddy uses for deSEC to only be allowed to manage records in those zones.
I'm using the shortlived profile with Let's Encrypt, because shorter certificates are better! My TLS certificates last just under a week thanks to this profile, but Caddy handles renewing them as needed.
systemd.services.caddy.credentials = {
caddy.desec_api_token = { };
};The credential that Caddy is using above for deSEC needs to be explicitly requested as a systemd credential on the service.
apps.http.servers.metrics = {
listen = [ ":2020" ];
automatic_https.disable = true;
routes = [ { handle = [ { handler = "metrics"; } ]; } ];
};This sets up a plain HTTP listener on port 2020 that exposes Prometheus metrics for Caddy.
apps.http.servers.default =
let
<<vhosts>>
in
{
listen = [ ":443" ];
logs = { };
metrics = { };
automatic_https.disable_certificates = true;
<<routes>>
};This sets up the main listener for HTTPS connections. Access logs and metrics are enabled for this server. Automatic certificate management is disabled because it was already manually configured above.
The routes for this server will be covered with the reverse proxy setup.
networking.firewall.allowedTCPPorts = [ 80 443 1965 2020 ]; networking.firewall.allowedUDPPorts = [ 443 ];
Port 80 and 443 need to be open for HTTP and HTTPS respectively. 443 also needs to be open for UDP traffic to support HTTP/3, which works via QUIC. Port 1965 is open for Gemini traffic, which is covered later on. Port 2020 allows Prometheus to scrape Caddy's metrics.
mjm.consul.services.caddy = {
port = 443;
metrics.enable = true;
metrics.port = 2020;
blockDeploy = true;
checks.up = {
http.url = "http://localhost:2019/reverse_proxy/upstreams";
};
};The Caddy service isn't being discovered via Consul, but it's still useful to register it anyway to enable health checking and metrics scraping. The Caddy service is configured to block a deploy advancing. This ensures that a bad deploy doesn't bring down both Caddy instances.
routes =
let
<<make-vhost-route>>
<<make-https-transport>>
<<make-dynamic-upstreams>>
in
[
<<scraper-routes>>
<<auth-proxy-route>>
<<special-vhost-routes>>
]
++ map mkVhostRoute (lib.attrValues vhosts);The default server's routes are where the meat of the config is at. Thankfully, Caddy has pretty good defaults overall, so most of what needs to be configured here is the stuff that is actually interesting to my setup.
There are some routes added to the front of the list that will covered in their own subsections. Otherwise, generating the routes is just applying a function to each virtual host that's been configured.
The list of vhosts is assembled around the definition of the default server, since it's needed both for these normal routes and for the error routes:
vhosts = lib.pipe nodes [ lib.attrValues (lib.filter (node: !node.config.mjm.ingress.enable)) (map (node: node.config.mjm.ingress.vhosts)) lib.mergeAttrsList ];
This is just gathering the virtual host definitions from the mjm.ingress.vhosts options on every host except the ingress hosts themselves.
vhosts = lib.mkOption {
default = { };
type =
let
vhostType = { name, ... }: {
options = {
<<vhost-options>>
};
};
in
with lib.types;
attrsOf (submodule vhostType);
};The mjm.ingress module provides a vhosts option that is intended to be set by other hosts in the lab besides the ones running Caddy. This is how the hosts that run the services can configure how they should should be accessed from the outside. vhosts is an attribute set, where the keys are the subdomain of the virtual host (the ".midna.dev" will be added automatically).
name = lib.mkOption {
type = lib.types.str;
default = name;
};The name of the virtual host defaults to the key used in the attribute set. In theory, it could be set to a different value, but I don't think there's much reason to do so. It mostly exists so that the entries in vhosts remember their name without having to pass the key around as well.
upstream.service.name = lib.mkOption {
type = lib.types.str;
};This option configures the name of the Consul service that the virtual host will proxy to. This is the only required option for the upstream: there must always be a service to route to, and in most cases that's all that is needed.
upstream.service.port = lib.mkOption {
type = with lib.types; nullOr port;
default = null;
};In some cases, the port to use for the upstream is different from the one registered for the service in Consul. In that case, the port can be set here, and the one set in Consul will be ignored.
upstream.service.tag = lib.mkOption {
type = with lib.types; nullOr str;
default = null;
};If a tag is set, requests will only be proxied to service instances that are tagged in Consul with the corresponding tag. This currently has no effect if upstream.service.port is set, but it could easily be made to work. There just hasn't been a use-case for it yet.
enableAuthProxy = lib.mkOption {
type = lib.types.bool;
default = false;
};By default, services are expected to handle authentication themselves, usually via OIDC with Authelia but sometimes through other means. If a service can't do this, it should set enableAuthProxy to true, which will make Authelia intercept all requests to the service to ensure a user is logged in. More on how that is done later.
useIPv4Proxy = lib.mkOption {
type = lib.types.bool;
default = true;
};By default, services are accessible through the ingress via both IPv4 and IPv6. IPv6 connections are direct: the Caddy hosts listen directly on an address that is routable from the public Internet. That's not doable for IPv4, so a VPS with a public IPv4 address will handle IPv4 requests and forward them to the IPv6 address.
There are some services where I feel uncomfortable sending the traffic through that tunnel, so for those, I can disable using this proxy. Those services will only be accessible via IPv6: they will not have any A records, just AAAA.
mkVhostRoute = vhost: {
match = [ { host = [ "${vhost.name}.midna.dev" ]; } ];
handle = [
{
handler = "reverse_proxy";
transport = mkHttpsTransport vhost.upstream.service.name;
dynamic_upstreams = mkDynamicUpstreams vhost.upstream.service;
headers.request.set = {
Host = [ "{http.request.host}" ];
};
}
];
};Each virtual host gets a single route that matches its hostname. This is a pretty straightforward reverse proxy route. The details of resolving and communicating with the upstreams is handled by helper functions shown below. Because some services rely on it rather than X-Forwarded-* headers (which Caddy sets by default), I'm overriding the Host header to match what was in the incoming request.
mkHttpsTransport = name: {
protocol = "http";
tls = {
ca.provider = "file";
ca.pem_files = [ "/run/certs/caddy/bundle.pem" ];
server_name = "${name}.service.consul";
client_certificate_file = "/run/certs/caddy/cert.pem";
client_certificate_key_file = "/run/certs/caddy/key.pem";
};
};My services all use mutual TLS to secure communications, and that includes Caddy connecting to upstream services. Most of the time, that means a client runs a ghostunnel that handles the TLS and exposes a plain HTTP endpoint the client can use. For Caddy, which needs to connect to nearly every server running in the lab, that would be a lot of tunnels. It doesn't make a lot of sense to do this when Caddy can handle the client side of the TLS just fine on its own with a little configuration, so that's what this function does.
mjm.spire.certs.caddy = {
systemd.unit = "caddy.service";
systemd.action = "reload-or-restart";
user = "caddy";
};This sets up the spiffe-certs service that will create and populate the /run/certs/caddy directory. This will give Caddy a client certificate and key, as well as a CA, to use for communication with any upstream service. Those services will need to allow connections from Caddy's SPIFFE ID. In the transport config above, the server name is also configured to ensure Caddy is talking to the correct service, and not another one that managed to get registered in Consul.
mkDynamicUpstreams =
{
name,
port ? null,
tag ? null,
...
}:
{
refresh = "15s";
}
// (
if port == null then
{
source = "srv";
service = name;
proto = if tag != null then tag else "tcp";
name = "service.consul";
}
else
{
source = "a";
name = "${name}.service.consul";
port = toString port;
}
);Caddy will use DNS records provided by Consul to discover the upstreams for a particular service. The simple case here is that the upstream is just specified as a service name. Caddy will use SRV records to discover both the host and port for instances of that service. This is great because Caddy doesn't need to know the port ahead of time, and the port can be different across different instances of the service. A tag can optionally be specified, in which case it will only find instances that include that tag in Consul.
A less good alternative is necessary if the port to use is different than the one registered in Consul. This happens with Consul itself, as the port used for the web UI is not the one it advertises. In this case, A records are used instead, and the port is specified instead of being looked up from DNS.
My preference is to use OIDC for authentication when services support it. It usually integrates better, but more importantly, it only requires interaction with the auth provider, Authelia, at the time of login.
Not every service I run supports OIDC though. Some services don't even have a concept of authentication in the first place. When I still want to protect these, I use Authelia's support for forward auth. Basically, any request to the virtual host being protected will get forwarded to an Authelia endpoint. If the user is already signed in, based on the cookie in the request, then Authelia will return a 200 code and set response headers with information about the user. If they are not signed in, Authelia will redirect them to do so.
{
match = [ { host = protectedHosts; } ];
handle = [
{
handler = "reverse_proxy";
transport = mkHttpsTransport "authelia";
dynamic_upstreams = mkDynamicUpstreams { name = "authelia"; };
<<auth-proxy-handle>>
}
];
}The route matches any of the virtual hosts that are configured to use the auth proxy. The route handler is reverse_proxy, but instead of proxying the request to the backend for the service, it sends the request to Authelia.
protectedHosts = lib.pipe vhosts [
lib.attrValues
(lib.filter (vhost: vhost.enableAuthProxy))
(map (vhost: "${vhost.name}.midna.dev"))
];This list of hosts that need to be protected is generated higher up, so that it can be also be used later for error handling. Virtual hosts that want to be protected will set the enableAuthProxy option to true.
Now there is some more special handling needed for reverse proxy handler:
rewrite = {
method = "GET";
uri = "/api/authz/forward-auth";
};
headers.request.set = {
X-Forwarded-Method = [ "{http.request.method}" ];
X-Forwarded-Uri = [ "{http.request.uri}" ];
};The request is rewritten to go to Authelia's forward auth endpoint instead of the method and URL it had originally. These bits of information are preserved in X-Forwarded-* request headers, so Authelia can use them to make authorization decisions.
handle_response =
let
<<set-header>>
in
[
{
match.status_code = [ 2 ];
routes = [
{
handle = [ { handler = "vars"; } ];
}
]
++ map setHeader [
"Remote-User"
"Remote-Groups"
"Remote-Name"
"Remote-Email"
];
}
];The handle_response option for a reverse_proxy handler can intercept the response from the upstream and handle it in a special way. Here, I'm matching only successful responses: any other response will be sent as-is to the client, which is what I want.
A successful response will not write its body to the client. Instead, a handful of headers will be copied from the Authelia response into the request, so that they are passed along as request headers to the actual upstream of the service.
The mechanism for copying these headers is a bit clunky:
setHeader = name: {
handle = [
{
handler = "headers";
request.set.${name} = [ "{http.reverse_proxy.header.${name}}" ];
}
];
match = [
{
not = [
{ vars."{http.reverse_proxy.header.${name}}" = [ "" ]; }
];
}
];
};A quirk of Caddy means that each header being copied needs to be checked to see that it isn't empty before doing so.
With this route in place, only requests from already logged-in users will actually make it to the backends for the protected services. Other requests will be redirected to a login flow. A big downside of this approach is that it requires Authelia to be up and available any time a protected service needs to be accessed, rather than only during login. It's also very important to block direct access to these services, since they are not protecting themselves.
errors.routes = [ <<error-routes>> ];
Caddy lets you define routes for handling different HTTP errors, so you can show appropriate error pages.
{
match = [
{
vars."{http.error.status_code}" = [ "503" ];
vars."{http.error.message}" = [ "no upstreams available" ];
host = protectedHosts;
}
];
handle = [
{
handler = "static_response";
body = "Sorry, no proxy upstreams are available. This means either the service you're accessing or Authelia have no healthy backends right now.";
}
];
}A 503 error with the message "no upstreams available" is a particularly common error case for me: a service being down during a deploy or because of some problem is normal in the course of things. When that happens, I show a message explaining the situation.
This version of the route matches only hosts that are using Authelia as a forward auth proxy. For these hosts, this error could mean there service being accessed is down, but it could also mean that Authelia is down, since it needs to intercept incoming requests. The message is written to reflect that.
{
match = [
{
vars."{http.error.status_code}" = [ "503" ];
vars."{http.error.message}" = [ "no upstreams available" ];
}
];
handle = [
{
handler = "static_response";
body = "Sorry, no proxy upstreams are available. The service you're accessing has no healthy backends right now.";
}
];
}This is the same case as the previous route, but since protected hosts have already been handled above, this will only match unprotected virtual hosts. In this case, there's no need to mention Authelia, as it doesn't need to be healthy to access the service.
{
handle = [
{
handler = "static_response";
body = "Sorry, something went wrong: {http.error.message}\n\nThe error came from {http.error.trace}";
}
];
}Any other error is shown with a generic template that includes as much useful information about the error as possible.
www.midna.dev and midna.dev host my personal website.
{
match = [
{
host = [
"midna.dev"
"www.midna.dev"
];
}
];
handle = [
{
handler = "subroute";
routes = [
<<www-vhost-routes>>
];
}
];
}The midna.dev and www.midna.dev hosts are handled the same. Both get handled via a list of subroutes.
{
match = [ { path = [ "/.well-known/matrix/server" ]; } ];
handle = [
{
handler = "static_response";
body = builtins.toJSON { "m.server" = "chat.midna.dev:443"; };
}
];
}For my Matrix identity, I use midna.dev as the domain, but the actual Matrix server is hosted at chat.midna.dev. This well-known URL tells other Matrix servers how to resolve that.
{
match = [ { path = [ "/.well-known/matrix/client" ]; } ];
handle = [
{
handler = "headers";
response.set.Access-Control-Allow-Origin = [ "*" ];
}
{
handler = "static_response";
body = builtins.toJSON {
"m.homeserver".base_url = "https://chat.midna.dev/";
"org.matrix.msc3575.proxy".url = "https://chat.midna.dev";
};
}
];
}A similar URL exists for Matrix clients. The response here lets clients know that when I try to log in as @mjm:midna.dev, they should actually find my homeserver at chat.midna.dev. I'm not actually sure if the MSC 3575 proxy URL part is necessary for sliding sync anymore.
Any other requests for this host will be handled by September, which will proxy HTTP requests to Gemini. That will covered a bit later.
{
match = [
{
host = [
"mta-sts.midna.dev"
"mta-sts.mattmoriarity.com"
];
}
];
handle = [
{
handler = "file_server";
root = pkgs.writeTextFile {
name = "mta-sts-root";
destination = "/.well-known/mta-sts.txt";
text = ''
version: STSv1
mode: enforce
mx: in1-smtp.messagingengine.com
mx: in2-smtp.messagingengine.com
max_age: 2419200
'';
};
}
];
}MTA-STS lets you require TLS connections for email being delivered to your domain. Part of setting up MTA-STS is hosting a small text file at "mta-sts./.well-known/mta-sts.txt", so this route serves that file for my two email domains. I host my email at Fastmail, so those are the MX servers listed here.
The spiffe.midna.dev virtual host serves OIDC configuration information for the JWT tokens issued by SPIRE. In particular, it can be used by other services to get the currently valid public keys for those tokens. This is used for logging into Vault with a SPIFFE token.
{
match = [ { host = [ "spiffe.midna.dev" ]; } ];
handle = [
{
handler = "reverse_proxy";
upstreams = [
{ dial = "unix//run/oidc-discovery-provider/server.sock"; }
];
}
];
}The route is a simple reverse proxy to a UNIX socket. The OIDC discovery provider service will run on the same machine as Caddy.
systemd.services.oidc-discovery-provider =
let
configFile = hcl1.generate "oidc-discovery-provider.json" {
<<oidc-discovery-config>>
};
hcl1 = pkgs.formats.hcl1 { };
in
{
description = "SPIRE OIDC Discovery Provider";
wantedBy = [ "multi-user.target" ];
after = [ "spire-agent.service" ];
serviceConfig = {
ExecStart = "${pkgs.spire.oidc}/bin/oidc-discovery-provider -config ${configFile}";
DynamicUser = true;
Restart = "on-failure";
RuntimeDirectory = "oidc-discovery-provider";
<<oidc-discovery-hardening>>
};
};There's nothing super complex about the service running the OIDC discovery provider. The provider itself is provided by SPIRE and is meant to communicate with the SPIRE agent running on the system. It requires a small amount of configuration:
domains = [ "spiffe.midna.dev" ];
listen_socket_path = "/run/oidc-discovery-provider/server.sock";
workload_api.socket_path = "${config.mjm.spire.agent.socketPath}";
workload_api.trust_domain = "${config.mjm.spire.agent.trustDomain}";I think the configuration here is pretty straightforward. Info about the workload API is pulled directly from the options for the SPIRE agent, so there can't be a config mismatch there.
CapabilityBoundingSet = ""; DevicePolicy = "closed"; LockPersonality = true; MemoryDenyWriteExecute = true; PrivateDevices = true; PrivateIPC = true; PrivateUsers = "identity"; ProtectClock = true; ProtectControlGroups = true; ProtectHome = true; ProtectHostname = true; ProtectKernelLogs = true; ProtectKernelModules = true; ProtectKernelTunables = true; ProtectProc = "invisible"; RestrictAddressFamilies = [ "AF_UNIX" ]; RestrictNamespaces = true; RestrictRealtime = true; SystemCallArchitectures = "native"; SystemCallErrorNumber = "EPERM"; SystemCallFilter = [ "@system-service" "~@resources @privileged" ]; UMask = "0077";
The systemd service has various hardening options applied to lock it down.
mjm.services.oidc-discovery-provider = { };
mjm.spire.entries.oidc-discovery-provider = {
spiffe_id = "spiffe://${config.mjm.spire.agent.trustDomain}/svc/oidc-discovery-provider";
selectors = [
{
type = "systemd";
value = "id:oidc-discovery-provider.service";
}
];
};The systemd service needs to be associated with a SPIRE identity to talk to the workload API, so this entry does that.
apps.layer4.servers.default = {
listen = [ "tcp/:1965" ];
routes = [
{
handle = [
{
handler = "proxy";
upstreams = [ { dial = [ "agate.service.consul:1965" ]; } ];
}
];
}
];
};I'm using the layer4 plugin for Caddy to be able to proxy Gemini requests to the microVM that actually hosts the Gemini server. Since Gemini is not HTTP, the normal HTTP proxying abilities of Caddy can't be used for this, so a simple TCP proxy is used instead.
systemd.services.september = {
description = "September Gemini to HTTP Proxy";
wantedBy = [ "multi-user.target" ];
serviceConfig = {
ExecStart = "${pkgs.september}/bin/september";
DynamicUser = true;
Restart = "on-failure";
<<september-hardening>>
};
environment = {
ROOT = "gemini://midna.dev";
PORT = "8080";
};
};September is a web server that proxies HTTP requests to Gemini and renders the resulting pages as HTML. I'm using it to reproduce my Gemini capsule at gemini://midna.dev on my website.
CapabilityBoundingSet = ""; DevicePolicy = "closed"; LockPersonality = true; MemoryDenyWriteExecute = true; PrivateDevices = true; PrivateIPC = true; PrivateUsers = "identity"; ProtectClock = true; ProtectControlGroups = true; ProtectHome = true; ProtectHostname = true; ProtectKernelLogs = true; ProtectKernelModules = true; ProtectKernelTunables = true; ProtectProc = "invisible"; RestrictAddressFamilies = [ "AF_INET" "AF_INET6" ]; RestrictNamespaces = true; RestrictRealtime = true; SystemCallArchitectures = "native"; SystemCallErrorNumber = "EPERM"; SystemCallFilter = [ "@system-service" "~@resources @privileged" ]; UMask = "0077";
September has the usual systemd hardening. It shouldn't need much in the way of permissions, as it mostly just needs to make network requests.
{
handle = [
{
handler = "reverse_proxy";
upstreams = [
{ dial = "127.0.0.1:8080"; }
];
}
];
}Finally, the last route for the www.midna.dev virtual host is to reverse proxy requests to September.
At some point I noticed my Git forge was using an unusually high amount of CPU and was struggling to serve requests. I prefer to have my Git repos public in general, so I was falling victim to scrapers.
Well-behaved scrapers will respect robots.txt, but there are a lot out there that are not well-behaved. Rather than deploy extra software like Anubis, I prefer an approach to blocking them that uses the tools I already have. It turns out a single route in Caddy is all you really need to block scrapers enough to avoid problems.
{
match = [
{
host = [ "git.midna.dev" ];
path = [
"*/archive/*"
"*/commit/*"
"*/actions/runs/*"
];
protocol = "http/1.1";
}
];
handle = [
{
handler = "static_response";
status_code = 426;
headers.Connection = [ "upgrade" ];
headers.Upgrade = [ "HTTP/3.0, HTTP/2.0" ];
}
];
}Things that are exposed publicly on the internet are meant to be consumed by whoever, and in general, serving those requests should not be expensive enough to bring down most machines. Unfortunately, most Git forges aren't really written such that that's true. The reason my forge was being brought down was that the requests the scrapers were making would spawn git processes to read data out of the repositories they were scraping. Enough of that happening at once, without letting up, and they all start to slow things down.
My friend emilazy pushed for the particular approach I'm using here. The route is attempting to block as little as possible while still protecting my server. I've plucked out a few endpoints that seem expensive to me and that scrapers seem to actually be requesting. And I filter down to only HTTP/1.1 requests. The scrapers I'm getting are always making HTTP/1.1 requests, even though I support more modern HTTP versions. Most clients and all modern browsers will negotiate to use HTTP/2 or HTTP/3 first, so by restricting the route to only HTTP/1.1 requests, the risk of blocking something legitimate goes down to basically zero.
When a request matches the route, I serve a 426 status, which is an indication to upgrade the connection. Because of now HTTP versions are negotiated, this is unlikely to actually be used by any real clients: they would have used ALPN to make an HTTP/2 or HTTP/3 connection already. But it's still the most semantically correct response for the situation, so that's what I send. What matters is that these requests don't make it to my Git forge.
The ingress hosts run keepalived to be able to share an IP address between them. Previously, I would set up the AAAA records such that both ingress hosts were present in DNS. This can lead to downtime when one of the hosts is rebooted: the DNS mechanism doesn't allow for falling back to another host if one is down.
Instead, now both hosts have an extra IP address that they essentially share custody of. One host has the IP by default, but if it goes down, keepalived will notice and use it instead until the primary host comes back.
isMaster = lib.mkOption {
type = lib.types.bool;
default = false;
};The isMaster option determines which ingress host should hold the shared virtual IP address when both hosts are alive.
virtualIP = lib.mkOption {
type = lib.types.str;
default = "${config.mjm.ipv6Prefix}:7fa6:7739:2bd0:ef69";
readOnly = true;
};The virtualIP is essentially a constant that contains the IPv6 address that is shared between ingress hosts.
services.keepalived = {
enable = true;
openFirewall = true;
vrrpInstances.ingress = {
interface = "lan0";
state = if cfg.isMaster then "MASTER" else "BACKUP";
virtualRouterId = 42;
priority = if cfg.isMaster then 200 else 100;
virtualIps = [
{ addr = "${cfg.virtualIP}/64"; }
];
};
};I'm not an expert in keepalived or how it works, but as I understand, this is a pretty minimal config for doing what I'm doing with it.
networking.nftables.enable = false;
The openFirewall option for keepalived is only written to support the iptables backend of the NixOS firewall, so using that requires disabling nftables.
text/gemini;lang=en-USThis content has been proxied by September (UNKNO).