Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions tools/renovate/bot-config.json5
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
// addon, so the workflow sets RENOVATE_X_IGNORE_RE2=true to take the RegExp path
// quietly.)
//
// Three entries, all load-bearing:
// Four entries, all load-bearing:
// 1. the toolchain-hash refresh, which re-prefetches the vendored-binary
// sha256 pins a tools/toolchain/versions/*.nix bump invalidates;
// 2. the catalog lockfile regeneration. Renovate's custom.regex manager exports
Expand All @@ -68,11 +68,20 @@
// 3. the devenv-nixpkgs lockstep, which re-locks devenv.lock at a new channel
// rev and rewrites the biome catalog pin to the version that rev bakes,
// keeping dev-shell baked==catalog parity.
// (1) and (3) are `bun <script>.ts` — the Renovate hooks are TypeScript run via
// bun (AGENTS.md "Scripts: TypeScript over bash"); bun is on the runner PATH.
// 4. the FOD-hash refresh, which recomputes the pinned Nix fixed-output-
// derivation hashes a dep bump invalidates — the Go `vendorHash`
// (guest-image/default.nix) on a gomod bump, the bun `outputHash`
// (agent-image/entrypoint.nix) on a bun/catalog bump. Left stale the image
// build fails `hash mismatch in fixed-output derivation` (PR #579). Wired at
// both the top-level (branch mode) and catalog-rule (update mode)
// postUpgradeTasks in config.json5 — one command string, so ONE allowlist
// entry covers both sites.
// (1), (3), and (4) are `bun <script>.ts` — the Renovate hooks are TypeScript run
// via bun (AGENTS.md "Scripts: TypeScript over bash"); bun is on the runner PATH.
allowedCommands: [
"^bun tools/renovate/refresh-toolchain-hashes\\.ts$",
"^bun install --lockfile-only$",
"^bun tools/renovate/refresh-devenv-nixpkgs\\.ts$",
"^bun tools/renovate/refresh-fod-hashes\\.ts$",
],
}
116 changes: 100 additions & 16 deletions tools/renovate/config.json5
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,22 @@
enabled: false,
},
{
// ── Go dependency rollup + guestd vendorHash lockstep ──
// A gomod bump rewrites go/go.mod|go.sum, moving the fetched Go module set —
// which invalidates the pinned `vendorHash` in guest-image/default.nix
// (buildGoModule compass-guestd, no in-repo vendor/ dir). Nothing regenerates
// that hash on its own, so the guest-image nix build fails
// `hash mismatch in fixed-output derivation …-go-modules.drv` and the bump PR
// goes red (PR #579's exact failure). The refresh task recomputes it in the
// same branch so the PR lands green.
//
// The command is NOT declared here — it rides the TOP-LEVEL postUpgradeTasks
// (branch mode) below, which every gomod branch inherits (a gomod branch
// never shares the single branch-task slot with the catalog rule, whose task
// is `executionMode: "update"`). refresh-fod-hashes.ts self-gates on
// go/go.mod|go.sum, so it refreshes the Go vendorHash alone here and no-ops
// the bun outputHash. Keeping the command in one place (top-level) avoids a
// second branch-mode task that would compete for that slot.
matchManagers: ["gomod"],
matchUpdateTypes: ["patch", "minor"],
groupName: "Go dependencies",
Expand Down Expand Up @@ -431,6 +447,20 @@
// would be a no-op here anyway. fileFilters cover exactly the files the script
// writes (compass has no committed inner-rev guard file, so no such entry).
//
// FOD-REFRESH IS INTENTIONALLY ABSENT here, though this branch rewrites
// bun.lock + a workspace manifest (the biome catalog pin) — the two inputs
// agent-image/entrypoint.nix names as moving its bun `outputHash`. It is
// safe because biome never enters the hashed tree: the FOD installs with
// `bun install --frozen-lockfile --ignore-scripts --filter '@compass/agent'`
// (entrypoint.nix), and biome is a ROOT-ONLY devDependency (package.json
// `devDependencies`, absent from @compass/agent's manifest). Verified: that
// exact filtered install yields biome in neither `node_modules` nor the
// `.bun` virtual store, so a channel/biome bump cannot move outputHash. If
// biome ever becomes a compass-agent dependency, this rule must append
// `bun tools/renovate/refresh-fod-hashes.ts` + `agent-image/entrypoint.nix`
// to its task (it runs after the bun.lock rewrite, so it would self-gate and
// refresh in the same branch). config.test.ts pins this exemption.
//
// Self-hosted bot config MUST allow the command:
// allowedCommands: [… , "^bun tools/renovate/refresh-devenv-nixpkgs\\.ts$"]
// see tools/renovate/bot-config.json5 (config.test.ts pins the two together).
Expand Down Expand Up @@ -468,15 +498,46 @@
// changing, so the evicted task would have been a no-op. It runs once per
// catalog upgrade, which is harmless: `bun install --lockfile-only` is
// idempotent and writes only bun.lock.
// The catalog-scoped task ALSO carries the FOD-hash refresh, for the same
// eviction reason it must carry the lockfile regen: a catalog bump changes
// compass-agent's installed node_modules tree (it consumes `catalog:` deps —
// effect, opentelemetry), which moves the pinned `outputHash` in
// agent-image/entrypoint.nix; left stale the agent/guest-image build fails
// `hash mismatch …-node-modules.drv`. The top-level branch-mode FOD task
// (below) is evicted on a catalog-FIRST rollup branch (the collapsed branch
// config spreads the alphabetically-first upgrade, and only ONE branch-mode
// task is built), so the refresh must ALSO ride this per-upgrade "update"
// pass to fire on every catalog branch regardless of sort order. Same command
// string as top-level → one allowlist entry; refresh-fod-hashes.ts self-gates
// on bun.lock so it refreshes the bun outputHash alone here and no-ops the Go
// vendorHash. Correctness is last-write-wins and order-independent.
//
// Self-hosted bot config MUST allow the command:
// allowedCommands: [… , "^bun install --lockfile-only$"]
// COST — unlike the lockfile leg, the FOD leg is idempotent but NOT free.
// `executionMode: "update"` runs it once PER catalog upgrade, and each run
// fakes the pin (a guaranteed FOD cache-miss) and realises the rootfs
// vehicle, forcing a full networked `bun install` of compass-agent's
// node_modules FOD. On a K-dep catalog rollup the outputHash is thus
// re-realised K times — including for catalog deps outside compass-agent's
// closure (astro/stylelint/starlight move bun.lock but not the hashed tree),
// which pay a realise to confirm no change. Accepted: last-write-wins is
// correct and a same-branch short-circuit is unsafe without relying on
// Renovate's grouped-update apply ordering (an early per-upgrade pass could
// see a partial tree and freeze a wrong hash). The double-fire on a
// bun-first branch (both this pass and the top-level task) is likewise
// correct, just a second realise.
//
// Self-hosted bot config MUST allow both commands:
// allowedCommands: [… , "^bun install --lockfile-only$",
// "^bun tools/renovate/refresh-fod-hashes\\.ts$"]
// see tools/renovate/bot-config.json5 (config.test.ts pins the two together).
matchManagers: ["custom.regex"],
matchDepTypes: ["workspaces.catalog"],
postUpgradeTasks: {
commands: ["bun install --lockfile-only"],
fileFilters: ["bun.lock"],
commands: [
"bun install --lockfile-only",
"bun tools/renovate/refresh-fod-hashes.ts",
],
fileFilters: ["bun.lock", "agent-image/entrypoint.nix"],
executionMode: "update",
},
},
Expand All @@ -496,24 +557,47 @@
},
],

// ── Toolchain hash coupling ──
// A versions/*.nix toolchain pin bump (bun/node/moon) leaves the vendored-binary
// sha256 pins beside each version stale → the dev-shell/CI build fails. This
// script re-prefetches and rewrites all three platform legs in each changed pin
// file in the same branch so the bump PR lands green. It self-gates (no-op unless
// a versions/*.nix pin changed), so it's cheap on every other branch — and it
// no-ops on go.nix, whose hashes come from go-overlay, not a per-file pin.
// `executionMode: branch` runs it once per branch after the update is applied.
// ── Toolchain hash + FOD hash coupling (top-level branch-mode tasks) ──
// Two refreshers ride the single per-branch task slot Renovate builds from the
// collapsed branch config; both self-gate on their trigger files, so each is a
// cheap no-op on a branch it doesn't apply to.
//
// Self-hosted bot config MUST allow the command:
// allowedCommands: ["^bun tools/renovate/refresh-toolchain-hashes\\.ts$"]
// and the runner needs `nix` (nix-command) on PATH.
// 1. refresh-toolchain-hashes.ts — a versions/*.nix toolchain pin bump
// (bun/node/moon) leaves the vendored-binary sha256 pins beside each version
// stale → the dev-shell/CI build fails. Re-prefetches and rewrites all three
// platform legs in each changed pin file in the same branch. Self-gates (no-op
// unless a versions/*.nix pin changed); no-ops on go.nix, whose hashes come
// from go-overlay.
// 2. refresh-fod-hashes.ts — a gomod bump (go/go.mod|go.sum) moves the pinned
// Go vendorHash in guest-image/default.nix; a bun bump (bun.lock) moves the
// pinned bun outputHash in agent-image/entrypoint.nix. Left stale the
// guest-image/agent-image build fails `hash mismatch in fixed-output
// derivation` (PR #579). Recomputes the invalidated hash by realising it.
// Self-gates per FOD on its trigger manifest. This is the branch-mode leg
// that covers gomod branches and bun/npm-first TypeScript-rollup branches;
// catalog-FIRST rollup branches evict this whole top-level task (their
// collapsed config carries the catalog rule's update-mode task instead), so
// the SAME fod command also rides that rule above — one command, one
// allowlist entry, idempotent on a double-fire.
//
// `executionMode: branch` runs each once per branch after the update is applied.
//
// Self-hosted bot config MUST allow both commands:
// allowedCommands: ["^bun tools/renovate/refresh-toolchain-hashes\\.ts$",
// "^bun tools/renovate/refresh-fod-hashes\\.ts$"]
// and the runner needs `nix` (nix-command) on PATH (refresh-fod-hashes realises
// a nix build; refresh-toolchain-hashes prefetches).
postUpgradeTasks: {
commands: ["bun tools/renovate/refresh-toolchain-hashes.ts"],
commands: [
"bun tools/renovate/refresh-toolchain-hashes.ts",
"bun tools/renovate/refresh-fod-hashes.ts",
],
fileFilters: [
"tools/toolchain/versions/bun.nix",
"tools/toolchain/versions/node.nix",
"tools/toolchain/versions/moon.nix",
"guest-image/default.nix",
"agent-image/entrypoint.nix",
],
executionMode: "branch",
},
Expand Down
99 changes: 91 additions & 8 deletions tools/renovate/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -188,15 +188,36 @@ describe("tools/renovate postUpgradeTasks ↔ allowedCommands (RIG-2432)", () =>
// postUpgradeTasks.commands are gated by the BOT config's global
// `allowedCommands` allowlist (a repo config cannot self-authorize a command),
// which Renovate matches UNANCHORED via regEx(pattern).test(cmd). So each
// entry's `^…$` IS the security property. Compass has exactly 3 commands and 3
// allowlist entries; every command must be permitted, every entry must be used,
// and no entry may be an unanchored substring rule.
// entry's `^…$` IS the security property. Compass declares four DISTINCT
// commands across the task sites (the FOD-hash refresh rides two sites — the
// top-level branch-mode task and the catalog rule's update-mode task — so it
// appears twice in the declared list but needs only one allowlist entry); every
// distinct command must be permitted, every entry must be used, and no entry may
// be an unanchored substring rule.
const commands = allDeclaredCommands();
const distinctCommands = [...new Set(commands)];
const allowed = bot.allowedCommands ?? [];

test("declares exactly three postUpgrade commands and three allowlist entries", () => {
expect(commands).toHaveLength(3);
expect(allowed).toHaveLength(3);
test("declares four DISTINCT postUpgrade commands and four allowlist entries", () => {
expect(distinctCommands).toHaveLength(4);
expect(allowed).toHaveLength(4);
});

test("the fod-hash refresh is declared at BOTH task sites (top-level + catalog)", () => {
// The command must ride two sites so it fires on every branch shape: the
// top-level branch-mode slot (gomod + bun/npm-first branches) AND the catalog
// rule's update-mode pass (catalog-first branches evict the top-level slot).
// If it collapses to one site, one of those branch shapes ships a stale FOD
// pin — PR #579's failure. So the raw (non-deduped) list carries it twice.
const fod = "bun tools/renovate/refresh-fod-hashes.ts";
expect(commands.filter((c) => c === fod)).toHaveLength(2);
const topLevel = cfg.postUpgradeTasks?.commands ?? [];
expect(topLevel).toContain(fod);
const catalogRule = cfg.packageRules.find(
(r) =>
r.matchDepTypes?.includes("workspaces.catalog") && r.postUpgradeTasks,
);
expect(catalogRule?.postUpgradeTasks?.commands).toContain(fod);
});

test("every declared command is permitted by an anchored allowlist entry", () => {
Expand All @@ -222,19 +243,67 @@ describe("tools/renovate postUpgradeTasks ↔ allowedCommands (RIG-2432)", () =>
new RegExp(a).test("bun install --lockfile-only; id"),
),
).toBe(false);
// …including on the FOD command (a `; rm -rf` tail must not slip through).
expect(
allowed.some((a) =>
new RegExp(a).test("bun tools/renovate/refresh-fod-hashes.ts; id"),
),
).toBe(false);
});

test("permits exactly the three RIG-2432 commands", () => {
expect(commands.sort()).toEqual(
test("permits exactly the four RIG-2432 commands", () => {
expect(distinctCommands.sort()).toEqual(
[
"bun install --lockfile-only",
"bun tools/renovate/refresh-devenv-nixpkgs.ts",
"bun tools/renovate/refresh-fod-hashes.ts",
"bun tools/renovate/refresh-toolchain-hashes.ts",
].sort(),
);
});
});

describe("tools/renovate FOD-hash refresh wiring (PR #579)", () => {
// A dep bump moves a pinned Nix fixed-output-derivation hash; left stale the
// image build fails `hash mismatch in fixed-output derivation`. refresh-fod-
// hashes.ts recomputes it, but Renovate only COMMITS files a task's fileFilters
// name — so a task that rewrites a FOD file without listing it silently drops
// the fix and the bump PR still goes red. Guard both sites' fileFilters.
const FOD = "bun tools/renovate/refresh-fod-hashes.ts";
const topLevel = cfg.postUpgradeTasks;
const catalogRule = cfg.packageRules.find(
(r) =>
r.matchDepTypes?.includes("workspaces.catalog") && r.postUpgradeTasks,
);

test("the top-level branch-mode task runs the fod refresh and is branch mode", () => {
expect(topLevel?.commands).toContain(FOD);
expect(topLevel?.executionMode).toBe("branch");
});

test("the top-level task commits BOTH FOD files (fileFilters cover them)", () => {
// gomod branches + bun/npm-first branches inherit this slot; it must be able
// to commit both the Go vendorHash file and the bun outputHash file.
expect(topLevel?.fileFilters).toContain("guest-image/default.nix");
expect(topLevel?.fileFilters).toContain("agent-image/entrypoint.nix");
});

test("the catalog rule runs the fod refresh in UPDATE mode (eviction-proof)", () => {
// A catalog-first rollup branch evicts the top-level branch task, so the
// refresh must also ride the catalog rule's per-upgrade update pass.
expect(catalogRule?.postUpgradeTasks?.commands).toContain(FOD);
expect(catalogRule?.postUpgradeTasks?.executionMode).toBe("update");
});

test("the catalog task commits the bun outputHash file it can move", () => {
// A catalog bump moves the bun outputHash (compass-agent consumes catalog:
// deps); it never touches the Go module set, so only entrypoint.nix is listed.
expect(catalogRule?.postUpgradeTasks?.fileFilters).toContain(
"agent-image/entrypoint.nix",
);
});
});

describe("tools/renovate OSV vuln source honors the fork fence", () => {
// Renovate owns security remediation. OSV is the config-driven vuln source
// that respects the forks/*/** disable packageRule, unlike a repo-wide toggle.
Expand Down Expand Up @@ -423,6 +492,15 @@ describe("tools/renovate devenv nixpkgs lockstep", () => {

// Branch-mode lockstep task over exactly the three files the script writes
// (compass has NO committed inner-rev guard file, unlike the internal monorepo's fourth entry).
//
// The `every(... refresh-devenv-nixpkgs ...)` assertion also PINS the verified
// FOD-refresh exemption: this branch rewrites bun.lock + the biome catalog pin,
// but biome is a root-only devDependency absent from the FOD's filtered
// `--filter '@compass/agent'` install (verified: not in node_modules nor .bun),
// so a channel bump cannot move agent-image/entrypoint.nix's outputHash and the
// FOD refresh is intentionally NOT wired here. If biome ever enters the
// compass-agent closure, adding the FOD command is what makes this test fail —
// forcing a conscious revisit of the exemption rather than a silent red build.
test("the lockstep postUpgradeTask is branch-mode over the written files", () => {
const task = devenvRule?.postUpgradeTasks;
expect(task?.executionMode).toBe("branch");
Expand All @@ -437,6 +515,11 @@ describe("tools/renovate devenv nixpkgs lockstep", () => {
/^bun tools\/renovate\/refresh-devenv-nixpkgs\.ts$/.test(c),
),
).toBe(true);
// Explicit: the FOD refresh is NOT on this rule (the verified exemption).
expect(task?.commands).not.toContain(
"bun tools/renovate/refresh-fod-hashes.ts",
);
expect(task?.fileFilters).not.toContain("agent-image/entrypoint.nix");
});

// The digest-excludes-rollup seam: the TS rollup ALSO matches custom.regex, so
Expand Down
Loading
Loading