feat(node)!: headless boot — bun supervisor, encryption at rest, LaunchDaemon - #26
Open
Demali-876 wants to merge 6 commits into
Open
feat(node)!: headless boot — bun supervisor, encryption at rest, LaunchDaemon#26Demali-876 wants to merge 6 commits into
Demali-876 wants to merge 6 commits into
Conversation
Carries an anonymous execution profile through the node's request path: the profile rides the data-plane proxy request, participates in the dedupe key, and its hash comes back on the response. Mirrors the client-side plan in consensus-client, with shared vectors so both sides agree on the wire format. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
src/supervise.ts runs the runtime server and the control tunnel as one unit and exits when either does, so `update_apply` and crashes alike cycle the unit from the refreshed `current` symlink. Same contract as the shell script it replaces, including exit 70 when no release is installed. Why: run-node.sh needed `wait -n` and therefore bash >= 4.3. Stock macOS ships bash 3.2, so it exited 78 on any Mac whose operator had not run `brew install bash` — the script carried that workaround as a documented caveat. Three deliberate differences from the shell version: - Children are spawned detached and signalled by process group. `bun run <script>` is a wrapper that forwards SIGTERM but cannot forward SIGKILL, so signalling the wrapper pid alone leaks its grandchild. The shell version had the same latent flaw; it never surfaced because `wait` blocked forever instead. - A 10s grace before SIGKILL (overridable via CONSENSUS_SUPERVISE_GRACE_MS for tests), so a wedged child cannot hold the unit open for the supervisor's full kill_timeout. - The interpreter is resolved from process.execPath rather than PATH, because a boot-time daemon's PATH does not include ~/.bun/bin. src/tests/supervise.test.ts covers exit-code propagation, the missing-release precondition, signalled shutdown, and forced kill — each asserting no child outlives the supervisor. Verified the orphan assertion fails when the process-group fix is reverted. BREAKING CHANGE: scripts/run-node.sh is removed. Deployments exec <install-dir>/current/src/supervise.ts instead; the bundled PM2 and systemd configs are updated, but a hand-rolled unit pointing at the old path will break. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
node.key and join-auth.json were plaintext on disk at mode 0600. They are now chacha20-poly1305 envelopes keyed by a 32-byte data key that lives OUTSIDE the state directory, with the file's slot name as AAD so ciphertext cannot be moved between slots. node.pub stays plaintext — it is public. Existing nodes migrate themselves: a plaintext file is sealed in place on first read and returns its original contents, so no re-registration and no operator action. A decryption failure throws rather than falling through to key generation — minting a fresh identity because the data key went missing would silently orphan the node's registration, which is worse than refusing to start. The data key lives in a 0600 file outside the state dir (~/Library/Application Support/consensus-node on macOS, ~/.config/consensus-node on Linux), reachable with no login session and no root. That is what makes pre-login boot possible. What this protects: a copied state directory, and backups scoped to it — the realistic leak, since ~/.consensus is what gets rsync'd or archived. What it does not: full-disk theft, or an attacker already holding the node's uid. That ceiling is forced by unattended boot, not chosen. The macOS System keychain was considered and rejected: it unlocks at boot from /var/db/SystemKey on the same disk, so it buys nothing here, and the login keychain needs a GUI login. The KeystoreAdapter interface is where a TPM adapter raises the ceiling on Linux. src/secrets-check.ts is a preflight the boot-unit installer runs AS THE DAEMON'S ACCOUNT, so an unreachable data key surfaces at install time rather than on the next reboot. Also isolates two tests that created an identity against the operator's real ~/.consensus/node; they now use a temp state dir and clean up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The node did not come back after a reboot until someone logged in. Cause: PM2's
own persistence (`pm2 startup`) emits a LaunchAgent on macOS, and agents load
only at user login. setup.ts merely printed that as a manual step.
Adds a LaunchDaemon in /Library/LaunchDaemons, which loads at boot with no login,
running pm2-runtime — PM2 in the foreground as one supervised process — against
the existing ecosystem config. scripts/install-launchd.sh renders the template
with absolute paths and bootstraps it.
Every path is substituted at install time because launchd performs NO variable
expansion: the plist this replaces used ${HOME} in EnvironmentVariables and both
log paths, which launchd would have taken literally. That plist was also
referenced by nothing. UserName drops the daemon to the operator so state files
keep their ownership, and HOME/PATH/PM2_HOME are set explicitly since a daemon
inherits none of them.
Before enabling the unit the installer runs secrets:check as the daemon's
account and refuses to install if the encryption data key is not readable
without a login — the failure would otherwise appear on the next reboot.
Documents two prerequisites that no configuration can substitute for: FileVault
must be off (it halts at a pre-boot unlock prompt, so nothing runs at all until a
human types the password), and `pmset -a autorestart 1` so the machine returns
after a power cut.
NOT YET VERIFIED IN SITU: install-launchd.sh requires sudo and a reboot on a real
node. The template renders and passes plutil -lint, and the preflight is tested,
but the boot path itself has not been exercised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 528af4e05e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+164
to
+171
| const cacheKey = profile?.cache_ttl | ||
| ? generateDedupeKey({ | ||
| target_url: request.target_url, | ||
| method, | ||
| headers: prepared?.headers ?? request.headers, | ||
| body: request.body, | ||
| profile_hash: profileHash, | ||
| }) |
There was a problem hiding this comment.
Profile-backed GETs that differ only in an ordinary forwarded header share this cache key, because generateDedupeKey includes only a small header allowlist while serveProxyRequest forwards other headers such as x-tenant-id or api-key. If an upstream varies its response by one of those headers, the second tenant can receive the first tenant's cached response; key the cache by all forwarded request headers or implement the upstream's Vary semantics.
Useful? React with 👍 / 👎.
Comment on lines
+228
to
+229
| if (cacheKey && profile?.cache_ttl && result.status >= 200 && result.status < 300) { | ||
| storeProfileResult(cacheKey, result, profile.cache_ttl); |
There was a problem hiding this comment.
When a profile permits POST, PUT, PATCH, or DELETE, any 2xx response is cached here and an identical later request is answered without contacting the upstream. That silently suppresses state-changing operations—for example, a repeated POST with the same body executes only once—so cache lookup and storage should be limited to GET/HEAD unless the protocol supplies explicit safe-method semantics.
Useful? React with 👍 / 👎.
Comment on lines
+262
to
+265
| try { | ||
| raw = await fs.readFile(file, "utf8"); | ||
| } catch { | ||
| return null; |
There was a problem hiding this comment.
When an existing node.key cannot be read because of permissions or a transient I/O error, this catch reports it as absent. loadOrCreateIdentity then falls through to key generation, and if the containing directory remains writable the atomic rename replaces the registered private key and rewrites node.pub, orphaning the node; only an actual ENOENT should return null, while other read errors must abort startup.
Useful? React with 👍 / 👎.
Comment on lines
+129
to
+134
| try { | ||
| const raw = await fs.readFile(keyFilePath(), "utf8"); | ||
| const key = Buffer.from(raw.trim(), "base64"); | ||
| return key.length === DEK_BYTES ? key : null; | ||
| } catch { | ||
| return null; |
There was a problem hiding this comment.
If the key file exists but is truncated, malformed, or otherwise decodes to a length other than 32 bytes, get() returns null, causing getOrCreateDataKey() to mint and overwrite it. All existing secret envelopes remain encrypted under the old key and become irrecoverable; distinguish ENOENT from invalid/unreadable key material and fail closed when a key file is already present.
Useful? React with 👍 / 👎.
Four P1s, all confirmed real by reading the code rather than taken on trust. secret-store: fail closed on damaged key material keyfileAdapter.get() returned null for ANY read failure, and for a file that decoded to the wrong length. getOrCreateDataKey treats null as "not provisioned" and mints a replacement, so a truncated or permission-denied key file was silently overwritten — making every secret sealed under the old key, including the node's identity, unrecoverable. Only ENOENT now returns null; a present but unreadable or malformed key throws and names the file. readSecretFile had the same shape: any read error read as absent, so loadOrCreateIdentity fell through to key generation and the atomic rename replaced a registered private key. Only ENOENT returns null now. The comment in identity.ts claimed this was already safe — it was true for decryption failures but not for I/O errors, and is now accurate. proxy-serve: restrict the profile cache to safe methods Any 2xx was cached regardless of method, so a repeated POST/PUT/PATCH/DELETE with the same body was answered locally and never reached the upstream — silently performing a state-changing operation once. Cache lookup and storage are both gated on the same key, which is now only computed for GET/HEAD. proxy-serve: key the cache on every forwarded header generateDedupeKey hashes a two-header allowlist (accept, content-type) because that is the locked cross-repo wire format, but serveProxyRequest forwards everything else the caller sent — authorization, x-tenant-id, api-key. Two callers differing only by such a header shared a cache entry, so one tenant could receive another's response. The node-local cache key now folds in all forwarded headers. generateDedupeKey and dedupe.vectors.json are untouched, so the orchestrator contract is unaffected; this value never leaves the node. Tests added for all four, each verified to fail when its fix is reverted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the node come back after a reboot without anyone logging in, and encrypts its secrets at rest on the way. Four new commits on top of the already-pushed
8176cc7.The problem
The node did not restart after a macOS reboot until someone logged in.
pm2 startup— whichsetup.tsprinted as a manual step — emits a LaunchAgent, and agents load only at user login. Linux was already fine:systemd/consensus-node.serviceis a system unit onmulti-user.target.What changed
refactor(node)!: replace run-node.sh with a bun supervisorsrc/supervise.tsruns the runtime server and control tunnel as one unit, exiting when either does. Same contract as the shell script, including exit 70 for a missing release.run-node.shneededwait -n, so bash ≥ 4.3 — but stock macOS ships 3.2, meaning it exited 78 on any Mac whose operator hadn't runbrew install bash. Three deliberate differences: children are spawned detached and signalled by process group (bun runforwards SIGTERM but cannot forward SIGKILL, so signalling the wrapper pid alone leaks its grandchild — the shell version had the same latent flaw, masked becausewaitblocked forever); a 10s grace before SIGKILL; and the interpreter comes fromprocess.execPath, since a daemon's PATH has no~/.bun/bin.feat(node): encrypt node secrets at restnode.keyandjoin-auth.jsonwere plaintext at 0600. They are now chacha20-poly1305 envelopes keyed by a 32-byte key held outside the state directory, with the slot name as AAD.node.pubstays plaintext. Existing nodes seal themselves on first read — no re-registration, no operator action.Honest scope: this covers a copied state directory and backups scoped to it — the realistic leak, since
~/.consensusis what gets rsync'd. It does not cover full-disk theft or an attacker already holding the node's uid. That ceiling is forced by unattended boot, not chosen. The macOS System keychain was considered and rejected: it unlocks at boot from/var/db/SystemKeyon the same disk, so it buys nothing, and the login keychain needs a GUI login.KeystoreAdapteris where a TPM adapter raises the ceiling on Linux.A decryption failure throws rather than generating a new key — minting a fresh identity because the data key vanished would silently orphan the node's registration.
feat(node): add a macOS LaunchDaemon for pre-login bootA LaunchDaemon running
pm2-runtimeagainst the existing ecosystem config. Every path is substituted at install time because launchd performs no variable expansion — the plist this replaces used${HOME}inEnvironmentVariablesand both log paths, which launchd would have taken literally. It was also referenced by nothing.install-launchd.shrunssecrets:checkas the daemon's account before enabling the unit, and refuses to install if the data key isn't readable without a login.feat(node): add profile-v1 execution plans— the anonymous execution profile threaded through the node's request path, mirroringconsensus-client.Two prerequisites no config can substitute for
sudo fdesetup authrestartboots once unattended; power loss still needs someone there.sudo pmset -a autorestart 1so the machine returns after a power cut at all.Verification
benchmarks,multi-core,sustained,network-eval)tsc --noEmitcleanplutil -lintNew suites:
test:supervise(exit-code propagation, missing release, signalled shutdown, forced kill — each asserting no child outlives the supervisor) andtest:secret-store(envelope primitives, AAD slot binding, tamper detection, migration, and the real keyfile adapter).Two latent breakages caught in passing
src/release.tsiterates a fixed directory list andfs.cpthrows ENOENT on a missing source — removinglaunchd/without updating it would have broken every release build.test:handshakeandtest:eval-clientcreated an identity against the operator's real~/.consensus/node. Now isolated to a temp state dir.🤖 Generated with Claude Code