Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion apps/cli/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ import {
updateGlobalMcpConfig,
withStepDefaults,
} from "@step-harness/coding-agent";
import { parseArgs, toPrintOutputMode } from "#args/index";
import { parseArgs, resolveAppMode, toPrintOutputMode } from "#args/index";
import { loadStepStartupConfig } from "#bootstrap/config";
import { createStepExtensionFactories } from "#bootstrap/extensions";
import { captureRawStdout, sdkStdioRequested } from "#bootstrap/stdout-capture";
Expand Down Expand Up @@ -554,6 +554,14 @@ try {
syncStepLoginProfileEndpoint(getStepAuthPath());
let shouldLaunchMain = true;
const parsedInteractiveArgs = parseArgs(compatibility?.args ?? stepCodeArgs);
if (
parsedInteractiveArgs.completionCheck &&
!parsedInteractiveArgs.help &&
!parsedInteractiveArgs.version &&
resolveAppMode(parsedInteractiveArgs, process.stdin.isTTY, process.stdout.isTTY) === "interactive"
) {
throw new Error("--completion-check requires print or JSON mode; use --print or --mode json.");
}
const interactiveStartup = isStepInteractiveLoginStartup({
stdinIsTTY: process.stdin.isTTY,
stdoutIsTTY: process.stdout.isTTY,
Expand Down Expand Up @@ -721,6 +729,8 @@ async function dispatchStepAppMode(prep: Extract<MainPreparation, { kind: "dispa
// print / json headless channels.
const exitCode = await runPrintMode(prep.runtimeHost, {
mode: toPrintOutputMode(prep.appMode),
completionCheck: prep.parsed.completionCheck,
completionCheckAttempts: prep.parsed.completionCheckAttempts,
messages: prep.parsed.messages,
initialMessage: prep.initialMessage,
initialImages: prep.initialImages,
Expand Down
75 changes: 75 additions & 0 deletions docs/compaction-integrity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Compaction integrity

Built-in compaction replaces earlier conversation context with a generated
summary. It must preserve the previous checkpoint when no new history needs
summarizing, and must fail before returning replacement context if a required
summary has no text. The coding-agent and agent-core compaction implementations
enforce the same rules.

## Preserve history during repeated split-turn compaction

A previous checkpoint can contain the only remaining copy of the original goal,
constraints, and verified results. When the next cut falls inside the first
retained turn, `messagesToSummarize` is empty but `turnPrefixMessages` is not.
The earlier summary still belongs in the next checkpoint.

For that case, `compact()` copies `previousSummary` verbatim as the history
portion and generates only the turn-prefix summary. The previous summary does
not need another model request. `No prior history.` is used only when there is
no previous summary. When new history exists, the existing update-summary request
continues to receive `previousSummary`.

## Reject empty generated summaries before assembly

Each history and turn-prefix generation extracts text blocks from the provider
response, then requires `text.trim().length > 0` before returning success. Empty
content arrays, empty text, whitespace-only text blocks, and thinking-only
responses fail this check. Thinking mixed with whitespace also fails. Accepted
text retains its original whitespace and formatting.

Validation happens before combining history and prefix text or appending file
operation metadata. In particular, none of the following can make a missing
generated summary valid:

- A preserved or newly generated history summary beside an empty turn prefix.
- `No prior history.`, the split-turn heading, or its separators.
- `<read-files>` and `<modified-files>` metadata.

Failure of either required generation fails the whole compaction. An empty
history response stops before requesting a turn-prefix summary. An empty prefix
response discards the newly generated history result as a candidate checkpoint.
No partial compaction result is returned.

The coding-agent helpers throw `Summarization failed: empty summary` or
`Turn prefix summarization failed: empty summary`. Agent-core returns a
`CompactionError` with code `summarization_failed` and the same message. Existing
session callers therefore keep their checkpoint, retained messages, and active
context when generation fails. Both manual and automatic built-in compaction
use these helpers.

Length-stop and provider-error diagnostics take precedence over the empty-text
check. Cancellation also remains a cancellation: coding-agent throws an
`AbortError` for an aborted summary response, and agent-core returns error code
`aborted`. Existing bounded retries for transient provider errors are unchanged;
an otherwise successful response with empty text fails without an added retry.

This check prevents missing summaries from being persisted. It does not judge
the factual quality of nonempty model text or validate extension-supplied
compaction results.

## Offline regression tests

The tests import the actual compaction and context modules and use the faux
provider. Coding-agent also exercises the real `AgentSession` and in-memory
`SessionManager`, checking that manual and automatic failures do not append a
checkpoint or change the active messages. Automatic cases also cover histories
without file metadata, where an empty generation previously produced either an
empty handoff or only the fixed split-turn boilerplate. No model service is used.

```sh
# From packages/coding-agent
pnpm exec vitest --run test/suite/regressions/compaction-integrity.test.ts

# From packages/agent-core
pnpm exec vitest --run test/harness/compaction-integrity.test.ts
```
77 changes: 77 additions & 0 deletions docs/completion-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Print-mode completion check

The Step CLI can perform a bounded completion check in the existing session:

```sh
step --print --completion-check git-committed --completion-check-attempts 2 "Complete the task and commit the changes."
step --mode json --completion-check git-committed "Complete the task and commit the changes."
```

The feature is off unless `--completion-check git-committed` is supplied. The
attempts option counts **additional prompts**, defaults to 2, and accepts integers
1 through 3. Both flags accept `--flag=value` syntax. Attempts without the check,
unsupported values, interactive mode, RPC, and SDK stdio are rejected. Piped
print mode is supported. Direct `runPrintMode` callers can supply the same
`completionCheck` and `completionCheckAttempts` options.

Before binding extensions or sending the first prompt, the check requires a Git
worktree with an existing HEAD commit and saves that HEAD. It then sends the
initial prompt, its images, and all additional user messages in their original
order. Once those prompts finish, completion requires all of these conditions:

- `starting-HEAD..HEAD` contains at least one commit. A preexisting commit or
moving HEAD backwards is insufficient.
- The committed tree differs from the starting HEAD's tree. An empty commit or
a change fully reverted before completion is insufficient. This tests delivery
of a change, not its correctness; the canonical verifier still owns correctness.
- The index and tracked worktree are clean, including submodule changes.
- No unignored untracked files remain. Ignored files do not block completion.
- The final assistant message has non-whitespace text and no pending tool calls.

If any condition is missing, a short status-only prompt asks the same session to
finish the task's required verification and commit work and give a final answer.
The original conversation and session ID remain in use. Already complete output
costs no extra model calls. The follow-up budget applies to the whole invocation,
not separately to each user message. These prompts consume the original trial's
time budget; no trial timeout is extended or reset, and no new attempt is started.
The checker neither changes source files nor commits changes or runs hidden tests.

An explicit terminating tool denial, or an assistant error/abort observed during
this invocation, prevents further prompts from the checker, including when a
native retry subsequently succeeds. Pending user messages also stop at such a
terminal outcome when the check is enabled. Native provider retry policies are
unchanged. Explicit runtime session replacement continues to rebind listeners and
extensions, but the checker does not carry automatic feedback into another
session or working directory.

After the follow-up budget is exhausted, a valid final answer still returns exit
code **0** even if Git conditions remain unsatisfied. The canonical task verifier
owns the score; an ordinary failed task must not become an infrastructure error
that resamples the attempt. Missing/thinking-only final output returns **2** with
an explicit incomplete diagnostic. Existing terminal denials and final assistant
errors keep exit code **1**. Invalid configuration or failed Git preflight returns
**1** before a model call. If Git becomes unreadable after the model runs, the
checker stops adding prompts and reports that state; final text still returns 0,
and missing final text returns 2.

Text stdout contains only the last assistant answer. Diagnostics use stderr.
JSON mode keeps the ordinary session event stream, including the added user
prompts, and adds `completion_check` events. Each successful inspection includes
`check`, `attempt` (follow-ups already used, starting at 0), `maxAttempts`,
`hasNewCommit`, `hasCommittedChanges`, `trackedDirty`, `untrackedFiles`, `hasFinalText`, `status`
(`passed`, `follow_up`, or `exhausted`), and `willFollowUp`. A failed inspection
emits `status: "unavailable"` and `willFollowUp: false`. No filenames, file
contents, diffs, commit messages, or Git stderr appear in check feedback/events.

Git is invoked directly with fixed argument arrays, no shell, and only a
validated starting object ID as a variable argument. Reads use `rev-parse`,
`rev-list --max-count=1`, `diff --quiet <starting-HEAD> HEAD --`, and NUL-delimited
porcelain `status --no-renames` with normal untracked-directory reporting. The
tree diff disables external diffs, text conversion, and rename detection. Only
its exit status is used: 0 means no committed changes, 1 means committed changes,
and any other code, timeout, or cancellation makes the check unavailable. Each command has a 5-second timeout,
64-KiB stdout/stderr limits, and SIGKILL termination; optional Git index/cache
writes and fsmonitor are disabled. Lazy fetching and interactive Git prompts are
disabled. Normal disposal and SIGINT/SIGTERM/SIGHUP cancel outstanding Git reads
and retain the existing runtime, detached-child, stdout-backpressure, and signal
cleanup paths.
15 changes: 13 additions & 2 deletions packages/agent-core/src/harness/compaction/compaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -609,6 +609,10 @@ export async function generateSummaryWithUsage(
}

const textContent = contentText(response.content);
// Validate model text before split-turn scaffolding or file metadata can make it look nonempty.
if (textContent.trim().length === 0) {
return err(new CompactionError("summarization_failed", "Summarization failed: empty summary"));
}

return ok({ text: textContent, usage: response.usage });
}
Expand Down Expand Up @@ -743,7 +747,8 @@ export async function compact(
let summaryUsage: Usage;

if (isSplitTurn && turnPrefixMessages.length > 0) {
let historyText = "No prior history.";
// With no new history to summarize, the previous checkpoint still carries the earlier context.
let historyText = previousSummary ?? "No prior history.";
let historyUsage: Usage | undefined;
if (messagesToSummarize.length > 0) {
const historyResult = await generateSummaryWithUsage(
Expand Down Expand Up @@ -862,8 +867,14 @@ async function generateTurnPrefixSummary(
);
}

const textContent = contentText(response.content);
// A valid history summary cannot substitute for a missing turn-prefix summary.
if (textContent.trim().length === 0) {
return err(new CompactionError("summarization_failed", "Turn prefix summarization failed: empty summary"));
}

return ok({
text: contentText(response.content),
text: textContent,
usage: response.usage,
});
}
Loading
Loading