Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions packages/dsh-loopx-plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -549,9 +549,21 @@ pre-step revalidation order remains authoritative.
DSH's LLM retry plugin owns provider-request retries. For CLI calls, this Driver
retries only safe fixed-argv LoopX reads and an idempotent `quota should-run`
receipt, at most twice beyond the first attempt. Every retry uses the same
`--turn-instance-id`. Typed authority denials, incompatible schemas,
cancellation, and human input are never retried. There is no cumulative
eight-failure breaker or plugin-owned durable activation state. The small
`--turn-instance-id`. If a saved quota page loses its CLI response, retry
only that page with the same reference, selection and offset. This also
applies during pre-step revalidation. Typed authority denials, incompatible schemas,
cancellation, and human input are never retried.

On macOS and Linux, the plugin cancels commands in the process group it started
and waits for that group and its output pipes before retrying a timed-out read.
If cleanup cannot be confirmed, it stops the attempt. This does not guarantee
cleanup of a process that starts its own session, or of descendants on Windows.

GoalBar never retries a lifecycle write. If no result can be verified, refresh
to read the Goal's current state before trying the action again.

There is no cumulative eight-failure breaker or plugin-owned durable activation
state. The small
process-local activation projection is bound to one exact Session and is
reconstructed only from its typed event history. Exhausting the three attempts
stops that evaluation; it does not create a periodic error-retry loop. Only a
Expand Down Expand Up @@ -635,7 +647,6 @@ cd packages/dsh-loopx-plugin
DSH_BIN="$(command -v dsh)" ./install.sh
dsh --profile web --dump-config
```

The installer rejects an incompatible host before changing its profile. A
successful profile contains `loopx-goalbar`, `loopx-init-command`, `loopx-driver`,
and the passive `loopx-shadow-observer` in that order. The installer also reads
Expand Down
130 changes: 115 additions & 15 deletions packages/dsh-loopx-plugin/src/cli.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { spawn } from 'node:child_process'
import { execFile, spawn } from 'node:child_process'
import { statSync } from 'node:fs'
import { setTimeout as delay } from 'node:timers/promises'

// 1.2.4 includes the released Windows peer-file fix. Bootstrap and every
// runtime consumer must use the same floor, including an existing global CLI.
Expand Down Expand Up @@ -116,12 +117,64 @@ export interface ResolveLoopXCommandOptions {
const DEFAULT_TIMEOUT_MS = 20_000
const DEFAULT_MAX_OUTPUT_BYTES = 1024 * 1024
const FORCE_KILL_GRACE_MS = 250
const CLEANUP_CONFIRM_MS = 1_000
const PS = process.platform === 'darwin' ? '/bin/ps' : '/usr/bin/ps'

function transportKind(error: NodeJS.ErrnoException): LoopXCliError {
// Match the managed Host's local process-group boundary. This npm package
// also runs standalone with older Core CLIs and cannot load their TS internals.
function signalOwnedGroup(pgid: number, signal: NodeJS.Signals | 0): boolean {
try {
process.kill(-pgid, signal)
return true
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ESRCH') return false
throw error
}
}

async function confirmGroupStopped(pgid: number): Promise<void> {
const deadline = performance.now() + CLEANUP_CONFIRM_MS
for (;;) {
const remaining = Math.ceil(deadline - performance.now())
if (remaining <= 0) throw new Error('CLI group did not exit')
try {
if (!signalOwnedGroup(pgid, 0)) return
} catch (error) {
// Darwin may return EPERM for an unreaped, dead group. Check process
// state; lack of permission to signal is not proof of termination.
if ((error as NodeJS.ErrnoException).code !== 'EPERM') throw error
}
const rows = await new Promise<string>((resolve, reject) => {
execFile(PS, ['-A', '-o', 'pgid=', '-o', 'stat='], {
encoding: 'utf8', timeout: remaining, killSignal: 'SIGKILL', maxBuffer: 1024 * 1024,
env: { LC_ALL: 'C', PATH: '/usr/bin:/bin' },
}, (error, stdout) => { if (error) reject(error); else resolve(stdout) })
})
if (performance.now() >= deadline || !rows.trim()) throw new Error('CLI group observation unavailable')
let live = false
for (const row of rows.trim().split('\n')) {
const [, group, state] = /^\s*(\d+)\s+(\S+)\s*$/u.exec(row) ?? []
if (group === undefined || state === undefined) throw new Error('Invalid CLI group observation')
if (Number(group) === pgid && !state.startsWith('Z')) live = true
}
if (!live) return
await delay(10)
}
}

function transportKind(error: NodeJS.ErrnoException, spawned: boolean): LoopXCliError {
if (error.code === 'ENOENT') {
return new LoopXCliError('missing', 'required executable is unavailable', false, undefined, { osCode: error.code })
}
return new LoopXCliError('transport', 'could not start the LoopX command', true, undefined, { osCode: error.code })
return new LoopXCliError('transport', spawned ? 'LoopX command I/O failed' : 'could not start the LoopX command',
true, undefined, { osCode: error.code })
}

// A non-retryable transport means the adapter could not safely continue to a
// second child. Keep launch/probe fallbacks from interpreting it as "missing."
export function isUnsafeToContinue(error: unknown): boolean {
return error instanceof LoopXCliError
&& (error.kind === 'aborted' || (error.kind === 'transport' && !error.retryable))
}

export const runFile: FileRunner = (file, args, options) => new Promise((resolve, reject) => {
Expand All @@ -130,10 +183,12 @@ export const runFile: FileRunner = (file, args, options) => new Promise((resolve
return
}

const posix = process.platform !== 'win32'
const child = spawn(file, [...args], {
cwd: options.cwd,
env: options.env ?? process.env,
shell: false,
detached: posix,
stdio: ['ignore', 'pipe', 'pipe'],
})
const stdout: Buffer[] = []
Expand All @@ -143,17 +198,59 @@ export const runFile: FileRunner = (file, args, options) => new Promise((resolve
let settled = false
let terminalError: LoopXCliError | undefined
let forceKillTimer: ReturnType<typeof setTimeout> | undefined
const closed = Promise.withResolvers<void>()
const group = posix && child.pid !== undefined && child.pid > 1 ? child.pid : undefined

const cleanup = (): void => {
clearTimeout(timer)
if (forceKillTimer !== undefined) clearTimeout(forceKillTimer)
options.signal?.removeEventListener('abort', abort)
}
const rejectOnce = (error: LoopXCliError): void => {
if (settled) return
settled = true
cleanup()
reject(error)
}
const fail = (error: LoopXCliError): void => {
if (settled || terminalError !== undefined) return
terminalError = error
clearTimeout(timer)
options.signal?.removeEventListener('abort', abort)
if (child.pid === undefined) {
rejectOnce(error)
return
}
if (group !== undefined) {
void (async () => {
try {
if (signalOwnedGroup(group, 'SIGTERM')) {
await delay(FORCE_KILL_GRACE_MS)
signalOwnedGroup(group, 'SIGKILL')
}
await confirmGroupStopped(group)
// If a descendant escaped the group with these pipes, cleanup of
// the owned group does not make this operation safe to retry.
await Promise.race([
closed.promise,
delay(CLEANUP_CONFIRM_MS, undefined, { ref: false }).then(() => {
throw new Error('CLI output pipes did not close')
}),
])
rejectOnce(error)
} catch {
child.stdout.destroy()
child.stderr.destroy()
rejectOnce(new LoopXCliError(
'transport', 'LoopX CLI process cleanup was not confirmed', false, undefined,
{ causeKind: error.kind },
))
}
})()
return
}
// Keep the direct-child/reap path on Windows. Node does not give this
// adapter a Job Object that it owns after the command leader has exited.
child.kill('SIGTERM')
forceKillTimer = setTimeout(() => {
if (!settled) child.kill('SIGKILL')
Expand All @@ -178,15 +275,19 @@ export const runFile: FileRunner = (file, args, options) => new Promise((resolve
options.signal?.addEventListener('abort', abort, { once: true })
child.stdout.on('data', (chunk: Buffer) => collect(stdout, chunk))
child.stderr.on('data', (chunk: Buffer) => collect(stderr, chunk))
child.on('error', (error: NodeJS.ErrnoException) => fail(transportKind(error)))
const onTransportError = (error: NodeJS.ErrnoException) => fail(transportKind(error, child.pid !== undefined))
child.stdout.on('error', onTransportError)
child.stderr.on('error', onTransportError)
child.on('error', onTransportError)
child.on('close', code => {
closed.resolve()
if (settled) return
settled = true
cleanup()
if (terminalError !== undefined) {
reject(terminalError)
if (group === undefined) rejectOnce(terminalError)
return
}
settled = true
cleanup()
resolve({
exitCode: code ?? 1,
stdout: Buffer.concat(stdout).toString('utf8'),
Expand Down Expand Up @@ -266,13 +367,12 @@ export async function runJsonMutationCommand(
},
)
} catch (error: unknown) {
throw error instanceof LoopXCliError
? error
: new LoopXCliError(
'transport',
'LoopX mutation command failed to execute',
false,
)
if (error instanceof LoopXCliError) {
throw error.retryable
? new LoopXCliError(error.kind, error.message, false, error.exitCode, error.diagnostics)
: error
}
throw new LoopXCliError('transport', 'LoopX mutation command failed to execute', false)
}

const payload = parsePayload(result.stdout)
Expand Down Expand Up @@ -447,7 +547,7 @@ export async function resolveLoopXCommand(
if (options.signal?.aborted) {
throw new LoopXCliError('aborted', 'operation was cancelled', false)
}
if (error instanceof LoopXCliError && error.kind === 'aborted') throw error
if (isUnsafeToContinue(error)) throw error
const failure = error instanceof LoopXCliError ? error
: new LoopXCliError('transport', 'LoopX version probe failed to execute', false)
probes.push(new LoopXCliError(failure.kind, 'version probe failed', false, failure.exitCode, {
Expand Down
14 changes: 9 additions & 5 deletions packages/dsh-loopx-plugin/src/driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,7 @@ const QUOTA_REQUIRED_POINTERS = ['/ok', '/mode', '/goal_id', '/should_run', '/ag
const QUOTA_RECEIPT_POINTER = '/heartbeat_receipt'
const QUOTA_OPTIONAL_POINTERS = [QUOTA_RECEIPT_POINTER, '/scheduler_hint', '/reward_memory_recall']

type QuotaReadOptions = Pick<JsonCommandOptions, 'runner' | 'cwd' | 'signal'>
type QuotaReadOptions = Pick<JsonCommandOptions, 'runner' | 'cwd' | 'signal' | 'attempts' | 'retryDelaysMs'>

function quotaNavigationReference(payload: Record<string, unknown>): string | undefined {
if (payload.schema_version !== QUOTA_NAVIGATION_SCHEMA || payload.ok !== true) return undefined
Expand Down Expand Up @@ -1278,7 +1278,13 @@ export class LoopXContinuationDriver {
// The 2.3 CLI supplies immutable navigation and its bounded reader. Older
// supported CLIs continue to answer this Host-only call with raw JSON.
const progressive = supportedLoopXVersion(command.version, '2.3.0') !== undefined
const options = { runner: this.runner, cwd: agent.session.header.cwd, signal }
const options = {
runner: this.runner,
cwd: agent.session.header.cwd,
signal,
attempts: 3,
retryDelaysMs: this.retryDelaysMs,
}
const rawAdmission = (payload: Record<string, unknown>) => payload.goal_id === binding.goalId
&& typeof payload.ok === 'boolean' && typeof payload.should_run === 'boolean'
const quota = await runJsonCommand(
Expand All @@ -1295,15 +1301,13 @@ export class LoopXContinuationDriver {
],
{
...options,
attempts: 3,
retryDelaysMs: this.retryDelaysMs,
...(progressive ? {} : { env: { ...process.env, LOOPX_CONTEXT_DELIVERY: 'off' } }),
validate: payload => rawAdmission(payload) || (progressive && quotaNavigationReference(payload) !== undefined),
},
)
const reference = progressive ? quotaNavigationReference(quota) : undefined
if (reference === undefined) return quota
// Reading a saved capture is outside the guard's transport retry.
// Retry the exact saved page on transport loss, outside the guard retry.
const decision = await readQuotaCapture(command, reference, options, revalidateOnly)
if (!rawAdmission(decision)) {
throw new LoopXCliError('invalid_schema', 'LoopX quota capture did not match the bound Goal', false)
Expand Down
3 changes: 2 additions & 1 deletion packages/dsh-loopx-plugin/src/goalbar/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,8 @@ export class GoalBarService implements GoalBarServiceHandle {
} catch {
return { kind: 'unknown', code: 'operation_result_unknown' }
} finally {
// The runner promise settles only after child close, including forced kill.
// Release the local operation after the runner has verified the result
// or reported it unknown. Never replay a lifecycle mutation here.
admission.resolveMutation()

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 原 writer 尚可能提交,GoalBar 就解除同一绑定的写隔离。在可配置 CLI 的隔离复现中,原 Pause 的组外后代持有 stderr,候选在清理未确认后读回 active,允许新的 Pause 和 Start。释放原进程后,真实 SQLite 接受迟到的原 Pause,Goal 再次变成 stopped。基线仍以 action_in_flight 拒绝后续请求。请在未确认旧写入安全终止时保持该绑定的写隔离,允许读回,并给出原命令不能再提交的解锁条件。

}
if (!verified) return { kind: 'unknown', code: 'operation_result_unknown' }
Expand Down
7 changes: 4 additions & 3 deletions packages/dsh-loopx-plugin/src/init-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import type { UserMessage } from '@deepseek-ai/dsh-session'
import {
LoopXCliError,
MINIMUM_LOOPX_VERSION,
isUnsafeToContinue,
resolveLoopXCommand,
runFile,
runJsonCommand,
Expand Down Expand Up @@ -183,7 +184,7 @@ async function compatible(
&& payload.host_surface === HOST_SURFACE
&& typeof payload.install_required === 'boolean'
} catch (error: unknown) {
if (error instanceof LoopXCliError && error.kind === 'aborted') throw error
if (isUnsafeToContinue(error)) throw error
return false
}
}
Expand Down Expand Up @@ -236,7 +237,7 @@ async function resolveInstallPython(options: LoopXInitOptions): Promise<string>
)
if (pipResult.exitCode === 0) return python
} catch (error: unknown) {
if (error instanceof LoopXCliError && error.kind === 'aborted') throw error
if (isUnsafeToContinue(error)) throw error
}
}
throw new LoopXInitError(
Expand Down Expand Up @@ -324,7 +325,7 @@ export async function initializeLoopX(options: LoopXInitOptions = {}): Promise<L
try {
command = await resolvePluginLoopXCommand(effectiveOptions)
} catch (error: unknown) {
if (error instanceof LoopXCliError && error.kind === 'aborted') throw error
if (isUnsafeToContinue(error)) throw error
}

const configuredCli = effectiveOptions.env?.LOOPX_BIN ?? process.env.LOOPX_BIN
Expand Down
Loading
Loading