Skip to content

Add an optional Nix flake: builds, analysis, fuzzing and VM tests - #24

Draft
randomizedcoder wants to merge 11 commits into
mainfrom
nix-pr
Draft

Add an optional Nix flake: builds, analysis, fuzzing and VM tests#24
randomizedcoder wants to merge 11 commits into
mainfrom
nix-pr

Conversation

@randomizedcoder

@randomizedcoder randomizedcoder commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

An optional Nix layer for Homa: one command each to build it against many kernels and architectures, analyse it, fuzz it and test it in VMs.

It lives on its own. Everything is under nix/, plus flake.nix, flake.lock and two .gitignore lines. No Makefile, source or test file changes, nothing to maintain for anyone who doesn't use Nix, and it can be deleted with rm -r nix flake.*.

What it gives you

Command What it does
Kernels nix build .#homa-build-matrix homa.ko against every nixpkgs kernel (6.1–7.3) and kernel.org 7.0–7.2, with a summary
Architectures nix build .#homa-cross-matrix homa.ko and util/ cross-compiled for riscv64 and aarch64
Unit tests nix build .#homa-tests, .#homa-tests-coverage the unit tests under AddressSanitizer, against a pinned kernel; gcov coverage (87 % of lines, 82 % of branches)
netdev checks nix build .#analysis-netdev what netdev runs on a submission, on the make net-next files: checkpatch, kernel-doc, W=1 builds with gcc, clang and i386
Static analysis nix build .#analysis-deep ruff, shellcheck, clang-format, cppcheck, clang-tidy, sparse (endianness), smatch, with per-tool counts
Fuzzing nix build .#fuzz-run, .#fuzz-mock libFuzzer + ASan/UBSan on the wire format and on the real homa_softirq receive path
VMs nix run .#homa-vm-matrix microVMs on 6.12 and 7.3: smoke tests, timed lifecycles, two-VM pairs for every kernel pairing (wire compatibility), netem loss and delay, Homa's qdisc and pacer
Sanitizers nix run .#homa-pair-lifecycle-debug, -kcsan the smoke test and the full pair lifecycle on a KASAN/KFENCE/KMEMLEAK/lockdep/UBSAN kernel, and on a KCSAN kernel watching only Homa
Evidence nix build .#homa-evidence all of the above in one evidence.html (nix run .#homa-evidence-full adds the VMs)
Dev shell nix develop every tool above on PATH, KDIR set, homa-build/homa-test/homa-lint

Everything is pinned by flake.lock, so a result is the same on any machine. VMs need no root, tap or bridge.

What it has found

Details for each are in nix/STATUS.md.

Commits

  1. Builds and dev shell: the flake, kernels, cross builds, user-space tools, unit tests, OCI image and dev shell.
  2. Static analysis: the analysis tiers.
  3. Fuzzing: the fuzz harnesses.
  4. VM tests: the microVM tests, with Tcl/Expect drivers and their tcltest suites.
  5. Docs: nix/README.md, nix/docs/DESIGN.md, nix/STATUS.md.
  6. netdev checks (analysis-netdev).
  7. Coverage (homa-tests-coverage).
  8. A netem drop-rate tolerance fix in the Tcl drivers.
  9. Sanitizer VMs.
  10. The evidence report, and docs and status for all of it.

nix flake check passes at every commit; it runs analysis-quick, fuzz-selftest and the Tcl driver tests.

Try it

nix build .#homa-module
nix develop
nix run .#homa-microvm-test

Start with nix/README.md.

🤖 Generated with Claude Code

randomizedcoder and others added 11 commits October 30, 2026 18:20
@randomizedcoder @claude
nix: add a flake that builds Homa for many kernels and architectures
20b65c8 
An optional Nix layer, entirely under nix/ plus flake.nix and flake.lock: the Makefiles
and sources are untouched, and nothing changes for anyone who doesn't use Nix.

- nix build .#homa-module: homa.ko against a pinned kernel, and homa-module-<series> for
  every nixpkgs kernel (6.1 to 7.3) plus kernel.org 7.0-7.2 trees (compile only);
  homa-build-matrix builds them all and writes a summary
- homa-module-riscv64/-aarch64 and homa-cross-matrix: cross-compiled homa.ko and util/
- homa-utils (and an AddressSanitizer variant), homa-tests (the unit tests under ASan),
  homa-image (an OCI image of the tools)
- nix develop: a shell with the kernel build deps and analysis tools, KDIR pointing at
  the pinned kernel, and homa-build/homa-test/homa-checkpatch helpers
- nix/patches: a place for upstream-pending fixes the build needs (empty)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix: tiered static analysis
5e2648f 
Three tiers, each a build whose output holds every tool's report and a summary with
per-tool finding counts:

- analysis-quick: ruff, shellcheck + shfmt, clang-format against the kernel's style
  (no kernel needed; a `nix flake check` gate)
- analysis-standard: + cppcheck, + clang-tidy (bugprone, cert, clang-analyzer, misc)
  over a compile DB captured from a real kbuild
- analysis-deep: + sparse with endianness checks, + smatch

Each tool is also its own output (analysis-sparse, ...). homa-lint in the dev shell runs
analysis-quick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix: libFuzzer harnesses for the wire format and the receive path
2b056d4 
- fuzz: standalone harnesses (ASan + UBSan) for a decoder of Homa's wire format, with
  seed corpora and a dictionary; fuzz-run runs each for 20 s, fuzz-deep for as long as
  you like in fork mode
- fuzz-mock: harnesses over the real homa_softirq receive path, built on the unit
  tests' mock layer, one per packet type as well
- fuzz-cov: llvm-cov coverage of the decoder
- fuzz-selftest: a canary that fails if the harnesses stop finding a planted bug (a
  `nix flake check` gate)

Crashes are sorted by sanitizer signature, and known findings are listed in
nix/fuzz/known-findings.txt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix: microVM tests: single VMs, lifecycles, VM pairs and a matrix
1b4f183 
QEMU microVMs (microvm.nix) that boot a chosen kernel and load the homa.ko built for it,
with the host's /nix/store shared in, so no image is built. No root, tap or bridge: the
pair link is a QEMU UDP socket on 127.0.0.1.

- homa-microvm-test[-<series>]: insmod, echo RPCs over loopback and between two network
  namespaces (GRO + RPS), rmmod, dmesg gate; -debug runs it on a KASAN/KMEMLEAK kernel
- homa-lifecycle[-<series>]: the same steps as timed phases, driven over the serial and
  a virtio console, plus repeated module load/unload
- homa-pair-test[-<server>-<client>]: two VMs, echo and cp_node (Homa and TCP) across
  the link, for every pairing of kernels (wire compatibility between builds)
- homa-pair-lifecycle[-<series>]: the pair over its consoles, with tc netem loss and
  delay phases and Homa's qdisc and pacer
- homa-vm-matrix: all of them in parallel, with a summary table; homa-vm-compare
  compares two runs phase by phase

The host-side drivers are a Tcl/Expect package (nix/microvms/tcl) with table-driven
tcltest suites, nagelfar and tclint, run by the `tcl-tests` check. The dev shell gets
the Tcl tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix: README, design notes and status
e43b70c 
nix/README.md is the quick start and command reference; nix/docs/DESIGN.md explains
the layout and the choices behind it; nix/STATUS.md records what each target covers,
what it has found in Homa, and the measurements from the VM tests. Each directory under
nix/ also has its own README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/analysis: analysis-netdev, netdev's checks on the code as submitted
a92dd75 
The other tiers check the out-of-tree source. This one checks what would be
sent to netdev: the files `make net-next` writes (util/strip.py), placed in the
default kernel's tree as net/homa, with net/Kconfig and net/Makefile wired as
an upstream patch would. Five reports: checkpatch --strict --codespell,
kernel-doc -Wall, and in-tree W=1 builds of net/homa with gcc + sparse, clang
and ARCH=i386. A build that never compiles a net/homa file fails rather than
reporting zero.

First run: the stripped source does not compile in-tree. strip.py keeps both
branches of an #ifndef __UNIT_TEST__ in homa_tx_pool.c (homa_tx_pool_mem_nid
defined twice), and homa_rpc.c uses sk->sk_memcg, which only exists under
CONFIG_MEMCG. Also 8 kernel-doc warnings and 31 checkpatch findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/build: homa-tests-coverage, unit-test coverage with gcov
0876658 
The unit tests built with --coverage and run once; gcovr reports line, branch
and function coverage of Homa's own sources (not the tests or the mock layer):
summary.txt, lines.txt, coverage.json and an annotated html/. Currently 86.7 %
of lines and 81.6 % of branches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/microvms/tcl: netem's drop rate may read as low as a quarter of t…
2d2ee5c 
…he loss

tc counts sent packets per GSO segment but drops per skb, so the measured
rate reads low, and lower still on a slow kernel, whose skbs carry more
segments: on a KCSAN kernel, 0.1 % loss read as 0.04 % and failed the
"netem is really on" floor of half the loss. The floor is now a quarter of
the loss; no drops at all in 10000 packets still fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/microvms: sanitizer VMs (KASAN/lockdep/UBSAN and KCSAN), smoke an…
b3a3ab3 
…d pair lifecycle

- debug-kernels.nix: "kasan" (KASAN, KFENCE, KMEMLEAK, lockdep, UBSAN,
  DEBUG_LIST) and "kcsan" kernels; homa.ko is built against each.
- kcsan.nix: once homa.ko is loaded, KCSAN watches only Homa's functions,
  so races elsewhere cannot fail a Homa test.
- homa-microvm[-test]-{debug,kcsan} and homa-pair-lifecycle-{debug,kcsan}.
  Timeouts scale by sanitizerSlowdown; the boot gets bootTimeoutDebug.
- The smoke test checks that KCSAN is on and watching, and fails on any
  sanitizer or lockdep report before rmmod.
- Sanitizer guests wait up to 600 s for devices, and every lifecycle's
  virtio phase waits for the boot and udev to finish: a KASAN guest
  answers on hvc0 minutes before eth1 has its address.

Found F13 (lockdep: peertab->lock -> ht->lock) and F14 (KCSAN: lockless
read of a deferred RPC's queue); KASAN names F8 without its fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/evidence: homa-evidence, one report of every check; docs and status
7cac084 
- homa-evidence: builds, unit tests and coverage, netdev checks, static
  analysis and fuzzing in one evidence.html; homa-evidence-full adds the
  VM matrix (and, with HOMA_EVIDENCE_SANITIZERS=1, the sanitizer pairs).
- STATUS: F12 (net-next files do not build in-tree), F13, F14, the KASAN
  pair lifecycle, follow-ups 18-20.
- DESIGN: sanitizer variants, netdev checks, coverage, evidence report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@randomizedcoder @claude
nix/lib: drop a patch with no hunks in a source subset
463d4c5 
filterdiff leaves only the mail header of a util/-only patch in the
module's source, and patch rejects it ("Only garbage was found in the
patch input").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@randomizedcoder