Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -976,13 +976,22 @@ on the host:
unless it or a parent closes it, as Bun and Node do for the processes they
start. A cancelled gate waits until every process holding it has exited,
including one outside the group that the signal never reached.
- Windows owns only the command itself.

A nested invocation runs immediately inside the admitted command, and a gate
inside another Quest scope, such as `quest worktree run`, leaves ownership to
that scope: an owner of its own would take the command out of the enclosing
one's reach. Internal compiler or test worker parallelism remains the command's
responsibility.
- Windows runs the command under a small Bun stub in a Job Object that kills
every process left in it once the stub exits, and lets none break away. It
owns every descendant, including detached ones, and stops them once the
command exits or the gate is killed. Windows has no signals to forward:
Ctrl-C reaches every process on the console, and killing the gate
terminates the job.

A nested invocation runs immediately inside the admitted command. A gate inside
another Quest scope, such as `quest worktree run`, runs its command in a child
cgroup of its own within that scope, which systemd delegates to the scope's
processes, so the gate owns its own subtree and the enclosing scope still
reaches it; a cancelled gate there waits until that child cgroup is empty.
A scope started by an older Quest, without delegation, keeps
ownership instead: a scope of the gate's own would take the command out of
the enclosing one's reach. Internal compiler or test worker parallelism remains
the command's responsibility.

This repository's `just check`, `just test`, `just worker-check`, and
`just worker-test` run through the gate.
Expand Down Expand Up @@ -1283,6 +1292,8 @@ is empty, and a second signal kills everything at once. A run inside another
Quest scope, such as a gate's, binds its own scope to that one, so the
enclosing owner's kill stops it too, and leaves a repeated signal's kill to
that owner: it may be the copy of SIGTERM that Just forwards.
The scope is delegated, so a `quest gate` inside it owns its command through a
child cgroup of the scope.
Work handed to another service manager, such as `systemd-run` or a container
daemon, escapes the scope. If the supervisor itself is killed, its command keeps
running and its scratch is kept; the next `quest worktree` command drops the run
Expand Down
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,4 @@ additional login providers. This milestone does not gate the public release.
- [ChatGPT hosted runs](/quest/m2/chatgpt-hosted.md) - conditional on provider approval and current terms
- [ChatGPT approval](/quest/m2/chatgpt-waitlist.md) - resolve hosted-use approval before implementation
- [Worker dev setup](/quest/m2/worker-dev.md) - per-server cookies, seeded raw output, static assets, one fake GitHub, each test run alone
- [Gate ownership everywhere](/quest/m2/gate-ownership.md) - delegated sub-cgroups for nested gates and Windows Job Objects
- [Delete the legacy hold labels](/quest/m2/legacy-labels.md) - once every gated repository pins quest past #326, only `quest:active` and `quest:kix` are holds
19 changes: 0 additions & 19 deletions quest/m2/gate-ownership.md

This file was deleted.

7 changes: 4 additions & 3 deletions scripts/gate-check.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
#!/usr/bin/env bash
# Check that an installed CLI's gate admits a lone command, however loaded the
# host, and owns its command's descendants. Inside another gate or Quest scope
# the enclosing owner holds them instead, so CI runs this on its own rather than
# from `just test`.
# host, and owns its command's descendants. Inside another gate or Quest scope,
# the enclosing owner holds them or delivers their signals instead, so a signal
# sent to the gate alone never reaches its command: CI runs this on its own
# rather than from `just test`.
set -euo pipefail

binary=${1:?usage: gate-check.sh /absolute/path/to/quest}
Expand Down
46 changes: 31 additions & 15 deletions src/cli/gate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ import { dirname, join, relative, resolve, sep } from "node:path";
import type { Readable } from "node:stream";
import { setTimeout as sleep } from "node:timers/promises";
import { type Cancellation, cancelled, follow, forwards, GROUP, interrupts, received, requested } from "./interrupt";
import { enclosingScope, systemdScopes } from "./supervise";
import { stubbed } from "./job";
import { delegated, nest, questScope, systemdScopes } from "./supervise";

/**
* One admitted command: a unique id, the wall-clock time it was admitted, and the process ID of
Expand Down Expand Up @@ -491,6 +492,24 @@ export const inside: Owner = plain(true);
export const scope: Owner = (command, env, cancel) =>
systemdScopes.run(`quest-gate-${randomUUID()}.scope`, process.cwd(), command, { env, cancel, within: null });

/**
* Inside a delegated Quest scope, run the command in a new child of `parent`, this process's cgroup,
* which holds every descendant like a scope of its own and stays within the enclosing scope's reach.
* That scope's owner signals every process in it, so this leaves a received signal and the kill at
* its second one to that owner, as `inside` does, and lets the child's processes finish their own
* cleanup before it kills what is left.
*/
export const nested =
(parent: string): Owner =>
(command, env, cancel) =>
nest(parent, `quest-gate-${randomUUID()}`, process.cwd(), command, { env, cancel });

/**
* On Windows, run the command under a stub in a kill-on-close Job Object that forbids breakaway
* (see job.ts), so every descendant stops once the command exits or the gate stops the stub.
*/
export const job: Owner = (command, env, cancel) => direct(...stubbed(command, env), cancel);

/**
* Run the command as the leader of a new session, whose process group holds every descendant that
* does not move to another group or session. Once the command exits, the rest of its group is
Expand Down Expand Up @@ -559,23 +578,20 @@ export const group: Owner = (command, env, cancel) =>
/** The descriptor on which the group owner hands its command the token (see `group`). */
const TOKEN = 3;

/** Whether this process already runs in a Quest scope, which then holds whatever it starts. */
export function scoped(self = "/proc/self/cgroup"): boolean {
return enclosingScope(self) !== undefined;
}

/**
* The strongest owner this host offers. A nested call runs inside its admitted command, and a call
* inside a Quest scope inside that scope: moving into an owner of its own would take its command out
* of reach of the enclosing one, which also signals it when it is Quest's (see `inside`). Linux
* owns descendants through a systemd scope, and settles for a process group, like macOS, without a
* systemd user manager. Windows owns only the command.
* The strongest owner this host offers. A nested call runs inside its admitted command, which an
* enclosing Quest owner also signals when it is Quest's group or a Quest scope (see `inside`). A
* call inside a Quest scope owns a child cgroup of its own there once the scope is delegated, and
* otherwise runs inside the scope: a scope of its own would take its command out of reach of the
* enclosing one. Linux owns descendants through a systemd scope, and settles for a process group,
* like macOS, without a systemd user manager. Windows owns them through a Job Object.
*/
export function owner(env = process.env, cgroup = "/proc/self/cgroup"): Owner {
if (process.platform === "win32") return direct;
// A Quest scope around this process holds the command too, since it shares the cgroup.
const enclosed = process.platform === "linux" && scoped(cgroup);
if (env[ADMITTED] === "1" || enclosed) return env[GROUP] === "1" || enclosed ? inside : direct;
// A Quest scope around this process holds the command too, since it shares the cgroup or an ancestor.
const enclosing = process.platform === "linux" ? questScope(cgroup) : undefined;
if (env[ADMITTED] === "1") return env[GROUP] === "1" || enclosing ? inside : direct;
if (process.platform === "win32") return job;
if (enclosing) return delegated(enclosing.unit) ? nested(enclosing.group) : inside;
if (process.platform !== "linux") return group;
try {
systemdScopes.check();
Expand Down
69 changes: 69 additions & 0 deletions src/cli/job.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
// Windows process ownership for the gate. Node offers no Job Object API and libuv's own job lets
// grandchildren break away silently, so a small Bun program joins a job of its own before it starts
// the command: everything the command starts is created inside that job and cannot break away.

/**
* Runs as `<Bun> -e STUB`, with `BUN_BE_BUN=1` so a compiled Quest runs it as Bun would. The command
* and the caller's `BUN_BE_BUN` arrive as JSON in `QUEST_GATE_JOB`, since Bun's `process.argv` for
* `-e` differs between platforms; the stub restores the caller's environment before the command
* starts.
*
* On Windows it creates a job that kills every process left in it once its last handle closes, and
* joins it. Only the stub holds that handle, so the job ends with the stub: when the command exits,
* when the gate terminates the stub, or when the gate dies and libuv's own kill-on-close job ends
* the stub. Once in a job that forbids breakaway, a process stays in it, and so does everything it
* starts, including detached processes and those that break away from libuv's nested job.
*
* The stub waits out console signals, which reach the command directly, so the command's own
* handling runs. It exits with the command's status, or 127 when the command cannot start, as a
* shell would, and 128 plus the signal's number for a command a signal stopped. Without a job it
* warns and still runs the command, as `direct` would.
*
* Joined onto one line of single-quoted strings, so it stays one plain command-line argument.
*/
export const STUB = [
"const { spawn } = require('node:child_process');",
"const { keep: original, command: [file, ...args] } = JSON.parse(process.env.QUEST_GATE_JOB);",
"delete process.env.QUEST_GATE_JOB;",
"if (original === null) delete process.env.BUN_BE_BUN; else process.env.BUN_BE_BUN = original;",
"if (process.platform === 'win32') {",
" try {",
" const { dlopen, FFIType, ptr } = require('bun:ffi');",
" const handle = FFIType.u64;",
" const { symbols: kernel } = dlopen('kernel32.dll', {",
" CreateJobObjectW: { args: [FFIType.ptr, FFIType.ptr], returns: handle },",
" SetInformationJobObject: { args: [handle, FFIType.i32, FFIType.ptr, FFIType.u32], returns: FFIType.i32 },",
" AssignProcessToJobObject: { args: [handle, handle], returns: FFIType.i32 },",
" GetCurrentProcess: { args: [], returns: handle },",
" GetLastError: { args: [], returns: FFIType.u32 },",
" });",
// JOBOBJECT_EXTENDED_LIMIT_INFORMATION is 144 bytes on 64-bit Windows; its LimitFlags sit at
// offset 16. Class 9 is JobObjectExtendedLimitInformation, and 0x2000 is
// JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. No breakaway flag is set, so none is allowed.
" const limits = new BigUint64Array(18);",
" new DataView(limits.buffer).setUint32(16, 0x2000, true);",
" const job = kernel.CreateJobObjectW(null, null);",
" if (!job || !kernel.SetInformationJobObject(job, 9, ptr(limits), 144) || !kernel.AssignProcessToJobObject(job, kernel.GetCurrentProcess()))",
" throw new Error('Windows error ' + kernel.GetLastError());",
" } catch (error) {",
" process.stderr.write('quest gate: cannot own descendants of the command: ' + error.message + String.fromCharCode(10));",
" }",
"}",
"for (const name of ['SIGINT', 'SIGBREAK', 'SIGHUP']) process.on(name, () => {});",
"const child = spawn(file, args, { stdio: 'inherit' });",
"child.once('error', (error) => {",
" process.stderr.write('quest gate: ' + error.message + String.fromCharCode(10));",
" process.exit(127);",
"});",
"child.once('exit', (code, signal) => process.exit(code ?? 128 + require('node:os').constants.signals[signal]));",
]
.map((line) => line.trim())
.join(" ");

/** The stub's command line and environment for running `command` with `env` in a job of its own. */
export function stubbed(command: string[], env: NodeJS.ProcessEnv): [string[], NodeJS.ProcessEnv] {
return [
[process.execPath, "-e", STUB],
{ ...env, BUN_BE_BUN: "1", QUEST_GATE_JOB: JSON.stringify({ keep: env.BUN_BE_BUN ?? null, command }) },
];
}
11 changes: 7 additions & 4 deletions src/cli/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,11 @@ export function main(argv: string[], out: Output): number | Promise<number> {
"that lands PRs should set, the command waits for memory but never for load.\n\n" +
"Once the command exits, or is cancelled and exits, its remaining descendants are killed; a second " +
"signal kills them all at once. Linux with a systemd 254+ user manager owns every descendant through " +
"a systemd user scope. macOS, and Linux without one, own the command's process group in a new session " +
"without a controlling terminal, so a descendant that leaves the group or session escapes. Windows " +
"owns only the command. Exits with the command's status, or 128 plus the signal's number once cancelled.",
"a systemd user scope, or inside a Quest scope through a child cgroup of its own. macOS, and Linux " +
"without one, own the command's process group in a new session without a controlling terminal, so a " +
"descendant that leaves the group or session escapes. Windows owns every descendant through a " +
"kill-on-close Job Object. Exits with the command's status, or 128 plus the signal's number once " +
"cancelled.",
)
.option("--quick", `Start a cheap command after at most ${QUICK_MS / 1000} seconds of waiting`)
.argument("<command>", "Executable to run")
Expand Down Expand Up @@ -743,7 +745,8 @@ export function main(argv: string[], out: Output): number | Promise<number> {
.summary("Run a command in a checkout, then release it once every process stops")
.description(
"Run a command in the checkout, restoring it first when released. The command and its descendants " +
"run in a new systemd user scope; once the command exits, descendants still in the scope are killed. " +
"run in a new, delegated systemd user scope; once the command exits, descendants still in the scope " +
"are killed. A `quest gate` inside it owns its command through a child cgroup of the scope. " +
"When no other supervised run remains, the scratch declared at creation is pruned. Put the whole job, " +
"such as an agent session or a script that installs, builds, and tests, in one run.\n\n" +
"A supervisor that crashes leaves its run recorded; the next worktree command drops it once its scope " +
Expand Down
Loading
Loading