Calling out to Nix

type Nix interface {
	Evaluator
	Builder
	Copier
}

The Nix interface is implemented by a type that can perform all of the Nix operations that dippy needs to do its job. It is an interface so that a mock implementation can be subbed in for unit tests.

type MultiNix struct {
	Evaluator
	Builder
	Copier
}

MultiNix is a struct that embeds all three Nix interfaces. This produces a type that implements Nix where each individual piece can be swapped out as needed.

func New(useNom bool, keyPath string) Nix {
	var b Builder
	if useNom {
		b = NomBuilder{}
	} else {
		b = BasicBuilder{}
	}

	var c Copier
	if keyPath != "" {
		opts := []string{
			"-F",
			"none",
			"-T",
			"-o",
			"BatchMode=yes",
			"-o",
			"IdentityFile=" + keyPath,
		}

		slog.Debug("ssh options", "opts", opts)
		c = &RealCopier{SSHOpts: opts}
	}

	return &MultiNix{
		Evaluator: RealEvaluator{},
		Builder:   b,
		Copier:    c,
	}
}

New creates a new Nix instance. It takes a flag for whether to use nix-output-monitor for builds and a path to an SSH private key to use when copying derivation outputs. It uses MultiNix to return different implementations for the builder and copier based on the arguments.

The useNom flag determines which of the two builder implementations will be used. Their implementations are shown further below.

The keyPath argument can be an empty string if the Copy operation is not going to be used, in which case the resulting Nix value will have a nil copier and will error if Copy is called. SSH is configured appropriately for not running interactively. It also ignores configuration files so that the behavior is consistent.

//| file: packages/dippy/nix/nix.go
package nix

import (
	"log/slog"
)

<<Nix>>
<<MultiNix>>
<<New>>

Evaluating Nix code into derivations

type Evaluator interface {
	EvalJobs(ctx context.Context, opts EvalJobsOptions) (iter.Seq[EvalJobResult], error)
}

EvalJobs uses nix-eval-jobs to evaluate all of the derivations in the attribute set resulting from evaluating a particular Nix file.

type RealEvaluator struct{}

func (_ RealEvaluator) EvalJobs(ctx context.Context, opts EvalJobsOptions) (iter.Seq[EvalJobResult], error) {
	<<RealEvaluator.EvalJobs-args>>
	<<RealEvaluator.EvalJobs-start>>
	<<RealEvaluator.EvalJobs-iterator>>
}

The real implementation of EvalJobs works by first building the arguments for the nix-eval-jobs command, then starting it, and finally returning an iterator that will scan the resulting output and yielding the results to the caller.

type EvalJobsOptions struct {
	Path    string
	Expr    string
	Args    map[string]string
	Workers int
}

EvalJobsOptions allows passing in values that will determine the arguments passed to nix-eval-jobs.

//| id: RealEvaluator.EvalJobs-args
args := []string{"--max-memory-size", "6144"}
if opts.Workers != 0 {
	args = append(args, "--workers", strconv.Itoa(opts.Workers))
}
if opts.Expr != "" {
	args = append(args, "--expr", opts.Expr)
}
for k, v := range opts.Args {
	args = append(args, "--arg", k, v)
}
if opts.Path != "" {
	args = append(args, opts.Path)
}

The first step for EvalJobs is to translate the options passed in to a list of arguments. The maximum memory size for each worker is hardcoded here: I've played with the interaction between this parameter and the number of workers, and found this to be a happy place for now.

//| id: RealEvaluator.EvalJobs-start
slog.DebugContext(ctx, "running nix-eval-jobs", "args", args)
cmd := exec.CommandContext(ctx, "nix-eval-jobs", args...)
cmd.Stderr = os.Stderr

output, err := cmd.StdoutPipe()
if err != nil {
	return nil, fmt.Errorf("creating stdout pipe: %w", err)
}

if err := cmd.Start(); err != nil {
	return nil, fmt.Errorf("starting nix-eval-jobs: %w", err)
}

With the arguments in place, the nix-eval-jobs command can be started. Stderr is inherited from dippy, while stdout is captured with a pipe so that the iterator will be able to consume the lines of output.

//| id: RealEvaluator.EvalJobs-iterator
return func(yield func(EvalJobResult) bool) {
	s := bufio.NewScanner(output)
	var count int
	for s.Scan() {
		<<RealEvaluator.EvalJobs-process-line>>
	}

	if s.Err() != nil {
		panic(fmt.Errorf("scanning nix-eval-jobs output: %w", s.Err()))
	}

	if err := cmd.Wait(); err != nil {
		panic(fmt.Errorf("running nix-eval-jobs: %w", err))
	}

	slog.DebugContext(ctx, "finished nix-eval-jobs", "result_count", count)
}, nil

EvalJobs returns an iterator of EvalJobResult structs, each one representing an evaluated derivation. It does this by creating a Scanner, which is an easy way to process a reader's content line-by-line. The scan loop will handle yielding each result to the consumer of the iterator until the output stream closes. After that, there's some basic checks for errors in the scanner or with waiting for the nix-eval-jobs command to finish, and then the iterator is done.

//| id: RealEvaluator.EvalJobs-process-line
if len(s.Bytes()) == 0 {
	continue
}

var result EvalJobResult
if err := json.Unmarshal(s.Bytes(), &result); err != nil {
	panic(fmt.Errorf("parsing eval result as json: %w", err))
}

slog.DebugContext(ctx, "eval result", "attr", result.Attr, "error", result.Error, "drv_path", result.DrvPath, "out_path", result.OutPath())
count++
if !yield(result) {
	return
}

Each line is first checked to see if it's empty, and if so, it is skipped. Otherwise, the line is expected to be valid JSON representing the result of evaluating a derivation. This JSON is unmarshalled into an EvalJobResult struct. The Nix instance for the result is attached for reasons that will be shown later. Finally, the result is yielded to the consumer of the iterator.

If the consumer breaks the loop, then the iterator function stops. I'm not sure that this will clean things up properly, but I also don't believe I ever actually break the loop early this way in practice, so it hasn't really been put to the test.

type EvalJobResult struct {
	Attr     string            `json:"attr"`
	AttrPath []string          `json:"attrPath"`
	DrvPath  string            `json:"drvPath"`
	Outputs  map[string]string `json:"outputs"`
	Error    string            `json:"error"`
	System   string            `json:"system"`
}

EvalJobResult is the parsed JSON output for each result from nix-eval-jobs. It may produce more fields than this, but these are the ones that are of interest to dippy.

func (r EvalJobResult) OutPath() string {
	return r.Outputs["out"]
}

OutPath is a convenience for getting the path for the "out" output of a result. This is the default output for a Nix derivation.

//| file: packages/dippy/nix/eval.go
package nix

import (
	"bufio"
	"context"
	"encoding/json/v2"
	"fmt"
	"iter"
	"log/slog"
	"os"
	"os/exec"
	"strconv"
)

<<Evaluator>>
<<RealEvaluator>>
<<EvalJobsOptions>>
<<EvalJobResult>>
<<EvalJobResult.OutPath>>

Realising (building) derivations

type Builder interface {
	Realise(ctx context.Context, drvPaths []string) error
}

Realise uses the "nix-store --realise" command to build one or more derivations (drv files). It doesn't return the output paths for those derivations, just an error if the build failed. It expects the caller to already know the output paths, and since nix-eval-jobs produces those, that's not a problem for dippy.

type BasicBuilder struct{}

var basicArgs = []string{"--no-gc-warning", "--realise"}

func (_ BasicBuilder) Realise(ctx context.Context, drvPaths []string) error {
	args := append(slices.Clone(basicArgs), drvPaths...)
	cmd := exec.CommandContext(ctx, "nix-store", args...)
	cmd.Stderr = os.Stderr
	slog.DebugContext(ctx, "running nix-store", "args", args)
	if err := cmd.Run(); err != nil {
		return fmt.Errorf("running nix-store: %w", err)
	}

	slog.DebugContext(ctx, "finished nix-store")
	return nil
}

There are two real implementations of Builder. The first one is the simplest, as it just runs "nix-store --realise", inheriting stderr from dippy to present the build log to the user.

type NomBuilder struct{}

func (_ NomBuilder) Realise(ctx context.Context, drvPaths []string) error {
	<<NomBuilder.Realise-setup-commands>>
	<<NomBuilder.Realise-run>>
}

The second real implementation of Builder uses nix-output-monitor (nom) to show a nicer view of the progress of the build. The setup for that is a bit more complex, so I'm breaking it down into two parts.

//| id: NomBuilder.Realise-setup-commands
args := append(slices.Clone(basicArgs), drvPaths...)
args = append(args, "--log-format", "internal-json", "-v")

cmd := exec.CommandContext(ctx, "nix-store", args...)
nomCmd := exec.CommandContext(ctx, "nom", "--json")
nomCmd.Stdout = os.Stdout
nomCmd.Stderr = os.Stderr

nomIn, err := nomCmd.StdinPipe()
if err != nil {
	return fmt.Errorf("creating nom in pipe: %w", err)
}
cmd.Stdout = nomIn
cmd.Stderr = nomIn

First, instead of a single command, it runs two. The nix-store command is almost the same as with BasicBuilder, but it requests the JSON log format that nom prefers. The second command runs nom with both output streams inherited from dippy to display the output to the user. A pipe is created for the nom command's stdin, and that pipe is set as stdout and stderr for the nix-store command, connecting the two commands together. Now nom will receive the output from the build so it can transform it.

//| id: NomBuilder.Realise-run
slog.DebugContext(ctx, "running nom")
if err := nomCmd.Start(); err != nil {
	return fmt.Errorf("starting nom: %w", err)
}
slog.DebugContext(ctx, "running nix-store", "args", args)
if err := cmd.Run(); err != nil {
	return fmt.Errorf("running nix-store: %w", err)
}

slog.DebugContext(ctx, "finished nix-store")
nomIn.Close()
if err := nomCmd.Wait(); err != nil {
	return fmt.Errorf("running nom: %w", err)
}

slog.DebugContext(ctx, "finished nom")
return nil

Then both commands created above need to run to completion. The nom command will both start before and end after the nix-store command so it captures all of the output. Before waiting for nom to complete, the pipe between the commands is closed. Without this, nom would continue expecting output and never exit.

func RealiseJSON(ctx context.Context, b Builder, drvPath string, outPath string, out any) error {
	if err := b.Realise(ctx, []string{drvPath}); err != nil {
		return fmt.Errorf("realising %q: %w", drvPath, err)
	}

	f, err := os.Open(outPath)
	if err != nil {
		return fmt.Errorf("opening %q: %w", outPath, err)
	}
	defer f.Close()

	if err := json.UnmarshalRead(f, out); err != nil {
		return fmt.Errorf("decoding json: %w", err)
	}

	return nil
}

RealiseJSON is a convenience function for building a derivation that produces JSON and unmarshalling it into a Go struct. The given drv path is realised, which should produce the given out path. That path is opened for reading and unmarshalled into the out parameter.

func (r EvalJobResult) RealiseJSON(ctx context.Context, b Builder, out any) error {
	return RealiseJSON(ctx, b, r.DrvPath, r.OutPath(), out)
}

As a further convenience, a RealiseJSON method is included on EvalJobResult, which already knows the Nix instance, the drv path, and the out path. The JSON files that dippy produces for its plans are read using this function.

//| file: packages/dippy/nix/build.go
package nix

import (
	"context"
	"encoding/json/v2"
	"fmt"
	"log/slog"
	"os"
	"os/exec"
	"slices"
)

<<Builder>>
<<BasicBuilder>>
<<NomBuilder>>
<<RealiseJSON>>
<<EvalJobResult.RealiseJSON>>

Copying outputs to remote hosts

//| file: packages/dippy/nix/copy.go
package nix

import (
	"context"
	"fmt"
	"os"
	"os/exec"
	"strings"
)

type Copier interface {
	Copy(ctx context.Context, opts CopyOptions) error
}

type RealCopier struct {
	SSHOpts []string
}

type CopyOptions struct {
	Installables []string
	To           string
}

func (c *RealCopier) Copy(ctx context.Context, opts CopyOptions) error {
	args := []string{"copy", "--no-check-sigs"}
	args = append(args, "--to", opts.To)
	args = append(args, opts.Installables...)

	cmd := exec.CommandContext(ctx, "nix", args...)
	cmd.Stderr = os.Stderr
	cmd.Stdout = os.Stdout

	sshOptsStr := strings.Join(c.SSHOpts, " ")
	cmd.Env = append(os.Environ(), "NIX_SSHOPTS="+sshOptsStr)

	if err := cmd.Run(); err != nil {
		return fmt.Errorf("running nix copy: %w", err)
	}
	return nil
}

Copy uses Nix to copy content from the Nix store from the local machine to a remote one. This allows builds to be done on the machine running dippy and then have the results be copied to the host where they will be used.

The implementation is pretty simple as it just needs to run a single command. The SSHOpts of the Nix instance are used here to set the NIX_SSHOPTS environment variable, which then is used by nix when it needs to use SSH.