From bb7964a412c50debb7bf7cabb7f03bb9dd9a48fb Mon Sep 17 00:00:00 2001 From: Jacob Prall Date: Sat, 24 Oct 2026 16:10:39 -0700 Subject: [PATCH 1/4] Fix the runbook's Workflows doctor step; document today's behavior - The runbook asked for doctor in the factory-workflows Render Shell, but a Workflows service has no Shell: Render starts a short-lived instance per task run. The runbook, operations guide, quickstart, and AGENTS.md now run the Workflows check locally with that service's values and deployed slug. - Operations: GitHub's Recent Deliveries is now the record of why a webhook started nothing, so the troubleshooting guide explains the gateway's responses and ignore reasons, its 422 and 502 log lines, how to read why an agent's submission was rejected, how to refresh a snapshot by hand, and how to clear a status comment stuck on "running". - The runbook expects doctor's "no webhook deliveries yet" warning before the smoke test, and can confirm telemetry from the smoke test's log line. - AGENTS.md: the gateway no longer depends on events; agent output objects must be strict. - Architecture: the Sandbox invariant matches AGENTS.md, two residual risks are added, and the code map lists claiming-task.ts. - /factory cancel now clears a status comment left on "running" by a run that already stopped, instead of replying that nothing is running. - tests/agents-catalog.test.ts enforces strict output schemas for every agent. --- AGENTS.md | 9 ++-- CONTRIBUTING.md | 3 +- app/workflows/cancel.ts | 19 +++++--- docs/architecture.md | 20 ++++++--- docs/configuration.md | 4 +- docs/operations.md | 85 +++++++++++++++++++++++++++++++----- docs/quickstart.md | 4 +- docs/runbooks/setup.md | 44 ++++++++++++++----- tests/agents-catalog.test.ts | 10 +++++ tests/cancel.test.ts | 14 ++++++ 10 files changed, 169 insertions(+), 43 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a630611..25b5c67 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,8 +8,8 @@ This file applies to the whole repository. Read - **Set up or deploy the factory:** follow [docs/runbooks/setup.md](docs/runbooks/setup.md) in order. Do not skip its - human checkpoints or claim completion before both deployed `doctor` checks - and the Triage smoke test pass. + human checkpoints or claim completion before `doctor` passes for both + services and the Triage smoke test passes. - **Change repositories, labels, models, or limits:** use [docs/configuration.md](docs/configuration.md). - **Diagnose a live deployment:** use @@ -60,7 +60,7 @@ agents → sandbox, runner, events, schemas, config changeset → sandbox, github, config sandbox → runner (to bundle it), github (read tokens), config format → schemas, config -gateway → schemas, events, config +gateway → schemas, config github → config runner, events, schemas, config → nothing in app/ ``` @@ -77,6 +77,7 @@ Import another module through its `index.ts` barrel where one exists. A barrel e - Strict TypeScript, ESM, NodeNext. Relative imports end in `.js`. No path aliases. - Biome is configured for tabs and a line width of 150 (`biome.json`); CI runs `biome lint`, which must pass with no warnings. - Validate every external or cross-process input with Zod at the boundary: webhook payloads, task inputs, agent output, markers, sandbox output, environment. +- Build every agent output object with `z.strictObject`, so a misspelled field is rejected by name in the error the model sees. `tests/agents-catalog.test.ts` enforces it. - Task inputs and results stay small and JSON-serializable. Each task opens one sandbox with `withSandbox`, which closes it within the same task. - Throw `ExpectedFailure` for an expected failure whose message is safe to show on GitHub. `withClaim` also treats an agent @@ -115,7 +116,7 @@ each is enforced): 9. Spec-to-PR implements the spec as it stood when the approval label was applied. 10. Every sandbox is created with a timeout and closed by `withSandbox` (in `finally`). A claiming task records its sandbox in the status comment, so the run or `/factory cancel` that stops it can terminate the sandbox: Render kills a cancelled task without running `finally`. -`tests/agents-catalog.test.ts` protects unique agent identities and tool authority. Change it only on purpose. +`tests/agents-catalog.test.ts` protects unique agent identities, tool authority, and strict output schemas. Change it only on purpose. ## Common changes diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9fde781..03faf77 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,7 +96,8 @@ The definition fixes: - a prompt relative to `app/agents/`; - a configured model tier; - tools selected from `read`, `ls`, `bash`, `edit`, and `write`; -- a bounded Zod output schema; +- a bounded Zod output schema, built from `z.strictObject` so unknown fields + are rejected by name; - a maximum number of model calls; and - an optional thinking level. diff --git a/app/workflows/cancel.ts b/app/workflows/cancel.ts index b5cd8e1..f0b9eb2 100644 --- a/app/workflows/cancel.ts +++ b/app/workflows/cancel.ts @@ -11,10 +11,14 @@ type CancelGitHub = StatusGitHub & Pick; interface CancelResult { cancelled: boolean; runId?: string; - reason?: "permission_denied" | "nothing_running"; + reason?: "permission_denied" | "nothing_running" | "already_stopped"; } -/** `/factory cancel`: stops the run the status comment names, if it's still active, and terminates its sandbox. */ +/** + * `/factory cancel`: stops the run the status comment names, if it's still active, and terminates its sandbox. A + * comment that still says "running" for a run that already stopped (it was killed before it could finish) is + * marked cancelled too, so the stale status and working label don't linger. + */ export async function cancelRun( ctx: Pick, raw: unknown, @@ -36,14 +40,15 @@ export async function cancelRun( } const current = await readStatus(gh, input.number); const stop = { repo: repo.fullName, subject: event.subject, by: runId, outcome: "cancelled" } as const; - if (!current || !(await stopRun(deps.runs ?? workflowRuns(), current.data, stop))) { + const active = current ? await stopRun(deps.runs ?? workflowRuns(), current.data, stop) : false; + if (!current || (!active && current.data.state !== "running")) { await gh.createComment(input.number, `@${input.actor} no factory run is in progress here.`); return { cancelled: false, reason: "nothing_running" }; } - const { runId: cancelled, stage, title } = current.data; - const summary = `cancelled by @${input.actor}.`; - await finishStatus({ gh, number: input.number, runId: cancelled, task: stage, title: title ?? stage, state: "cancelled", summary }); - return { cancelled: true, runId: cancelled }; + const { runId: stopped, stage, title } = current.data; + const summary = active ? `cancelled by @${input.actor}.` : `cleared by @${input.actor}; the run had already stopped without finishing.`; + await finishStatus({ gh, number: input.number, runId: stopped, task: stage, title: title ?? stage, state: "cancelled", summary }); + return active ? { cancelled: true, runId: stopped } : { cancelled: false, runId: stopped, reason: "already_stopped" }; })(); emit({ ...event, type: "run.finished", payload: { stage: TASKS.cancel, state: "done", ...result } }); diff --git a/docs/architecture.md b/docs/architecture.md index 1a0d3fc..6ff758a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -171,7 +171,9 @@ Changes must preserve these properties: 6. Publish only after GitHub reproduces the validated tree SHA. 7. End automation at a draft PR and COMMENT review. 8. Implement the exact specification that was approved. -9. Put a timeout and `finally` cleanup around every Sandbox. +9. Put a timeout and `finally` cleanup around every Sandbox, and record a + claiming task's Sandbox in its status comment so whatever stops the run + can terminate it. Tests pin these invariants and each built-in agent's tool authority. @@ -186,6 +188,11 @@ Tests pin these invariants and each built-in agent's tool authority. - The claim relies on reading the status comment back right after writing it. Two triggers within a second of each other can both do work before one stops. - Workflows inputs and results are visible to users with access to the service. +- Telemetry log lines include repository names, issue subjects, and error + messages that can quote repository output, visible to anyone with access to + the service logs or a log stream. +- A cancelled snapshot build leaves its builder Sandbox running until its + one-hour timeout, because builder Sandboxes aren't recorded anywhere. Use a dedicated Render workspace and spend-capped model key when stronger isolation is required. @@ -196,11 +203,12 @@ Start with these files: 1. [`app/gateway/routes.ts`](../app/gateway/routes.ts) — event routing. 2. [`app/claims/claim.ts`](../app/claims/claim.ts) — the claim lifecycle every claiming task shares. -3. [`app/workflows/triage/index.ts`](../app/workflows/triage/index.ts) — one complete claiming task. -4. [`app/agents/run.ts`](../app/agents/run.ts) — Sandbox agent execution. -5. [`app/sandbox/sandbox.ts`](../app/sandbox/sandbox.ts) — Sandbox lifecycle. -6. [`app/workflows/spec-to-pr/index.ts`](../app/workflows/spec-to-pr/index.ts) — implementation and verification. -7. [`app/changeset/index.ts`](../app/changeset/index.ts) — validated publishing. +3. [`app/workflows/claiming-task.ts`](../app/workflows/claiming-task.ts) — how a claiming task is declared. +4. [`app/workflows/triage/index.ts`](../app/workflows/triage/index.ts) — one complete claiming task. +5. [`app/agents/run.ts`](../app/agents/run.ts) — Sandbox agent execution. +6. [`app/sandbox/sandbox.ts`](../app/sandbox/sandbox.ts) — Sandbox lifecycle. +7. [`app/workflows/spec-to-pr/index.ts`](../app/workflows/spec-to-pr/index.ts) — implementation and verification. +8. [`app/changeset/index.ts`](../app/changeset/index.ts) — validated publishing. See [Contributing](../CONTRIBUTING.md) for local development and extension recipes. diff --git a/docs/configuration.md b/docs/configuration.md index 2046b12..4080815 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -61,7 +61,9 @@ After adding a repository: 1. commit and deploy `factory.config.ts`; 2. install the GitHub App on the repository; 3. run `npm run setup:github`; and -4. apply a factory label or push to the base branch to build its first snapshot. +4. apply a factory label, push to the base branch, or + [refresh its snapshot by hand](operations.md#refresh-a-snapshot-by-hand) + to build its first snapshot. ## Behavior and limits diff --git a/docs/operations.md b/docs/operations.md index 3ffbd33..8ea2379 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -35,7 +35,9 @@ To cancel, post this as the first line of an issue or pull request comment: The commenter must have `minimumPermission`. Cancellation stops the run the status comment names and terminates its Sandbox. Render kills a cancelled run -within about a second. +within about a second. If the comment still says "running" for a run that +already stopped, for example one Render killed for exceeding its timeout, +`/factory cancel` marks it cancelled and removes `factory:working`. To restart a task, remove its trigger label and apply it again. @@ -49,14 +51,19 @@ npm run doctor ``` It does not inspect environment variables stored only on Render. To verify -each deployed service's actual environment, run `doctor` from its Render -Shell: +the gateway's actual environment, run `doctor` from its Render Shell: ```bash -# factory-gateway npm run doctor -- --service gateway +``` + +A Workflows service has no Shell, because Render starts a short-lived +instance for each task run. Check it locally instead, with its values and its +deployed `RENDER_WORKFLOW_SLUG` in `.env`, as the +[setup runbook](runbooks/setup.md#phase-9-verify-the-deployed-services) +describes: -# factory-workflows +```bash npm run doctor -- --service workflows ``` @@ -81,25 +88,59 @@ Configuration and code changes take effect at different times: Snapshot refresh keeps the previous usable version when a new build fails. +### Refresh a snapshot by hand + +Start the `refresh-snapshot` task with the repository, its App installation, +and the base branch's head commit: + +```bash +git ls-remote https://github.com//.git refs/heads/ +echo '[{"repo": "/", "installationId": , "sha": ""}]' > input.json +render workflows tasks runs start /refresh-snapshot --input-file input.json +``` + +The installation ID is the number at the end of the App installation's +settings URL (`https://github.com/settings/installations/`, or the +organization's equivalent). + ## Troubleshoot ### No status comment -Inspect the GitHub App's recent webhook deliveries. Confirm: +Open the GitHub App's **Advanced** settings and find the delivery under +**Recent Deliveries**. If there is none, confirm the webhook is active and its +URL ends in `/webhooks/github`. + +The gateway's response says what happened. `202` means a task started; look +for it under `factory-workflows`. A `200` with `{"ignored": ""}` names +why nothing started: + +- `unconfigured repo`: the repository isn't in `factory.config.ts`. +- `unhandled label`: the label doesn't match a configured trigger label. +- `issue not in triage`: a comment arrived on an issue without the Triage + label. +- `bot sender`: bots, including the factory itself, can't start runs. +- `pull request not open` or `not base branch`: the event targets something + the factory skips. -- the webhook is active; -- its URL ends in `/webhooks/github`; -- the repository appears in `factory.config.ts`; and -- the triggering actor has `minimumPermission`. +If a task started but the status comment says the run "did not run", the +triggering actor lacks `minimumPermission`. ### Webhook delivery returns 401 `GITHUB_WEBHOOK_SECRET` differs between GitHub and `factory-gateway`. +### Webhook delivery returns 422 + +The delivery was authenticated but produced an invalid task input. The +gateway logs `authenticated delivery produced an invalid task input` with the +reason; this usually points to a routing bug. + ### Webhook delivery returns 502 -The gateway could not start the task. Check `RENDER_API_KEY`, -`RENDER_WORKFLOW_SLUG`, and the Workflows service. +The gateway could not start the task. Its logs show `could not start task` +with the underlying error. Check `RENDER_API_KEY`, `RENDER_WORKFLOW_SLUG`, and +the Workflows service. ### Snapshot build fails @@ -110,6 +151,26 @@ Inspect the `refresh-snapshot` run or the first task run. Common causes are: - a missing App installation; or - a Sandbox verification failure. +### Agent submitted an invalid result + +The status comment says an agent "submitted an invalid result" when it was +rejected three times for not matching its output schema. The `agent.finished` +log line records why, with the last attempt's schema problems and the field +names it sent: + +```bash +render logs -r --text invalid_submission +``` + +A misspelled or extra field appears among the submitted names. Problems that +recur for one issue usually point to the agent's prompt. + +### Status comment stuck on "running" + +The run stopped without finishing, for example after hitting its timeout. +Post `/factory cancel` to clear the status and terminate any Sandbox it left, +or reapply the trigger label to start a fresh run. + ### Agent cannot start Check the selected provider key in `factory-workflows`, then run: diff --git a/docs/quickstart.md b/docs/quickstart.md index 16dc83f..2c7fd70 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -26,7 +26,7 @@ The agent will: 3. guide creation and installation of a GitHub App; 4. guide deployment of the Render Blueprint; 5. connect the webhook and create labels; -6. verify both deployed services with `doctor`; and +6. verify both services with `doctor`; and 7. run one approved Triage smoke test. The agent must stop for your confirmation before: @@ -52,7 +52,7 @@ Configure using base branch . Stop at every human checkpoint and do not expose secrets. ``` -Setup is complete only when both deployed `doctor` checks pass and applying +Setup is complete only when `doctor` passes for both services and applying `factory:triage` produces an App-authored status comment. ## Continue diff --git a/docs/runbooks/setup.md b/docs/runbooks/setup.md index 27634f7..685db92 100644 --- a/docs/runbooks/setup.md +++ b/docs/runbooks/setup.md @@ -1,8 +1,8 @@ # Setup runbook Use this runbook when helping someone deploy Software Factory for the first -time. Follow it in order. Do not claim setup is complete until both deployed -services pass `doctor` and a labeled issue produces a factory status comment. +time. Follow it in order. Do not claim setup is complete until `doctor` +passes for both services and a labeled issue produces a factory status comment. ## Goal @@ -17,7 +17,8 @@ Setup is complete only when: - the GitHub App is installed on every configured repository; - the App webhook is active and points to the gateway; - the factory labels exist; -- deployed `doctor` checks pass for the gateway and Workflows services; and +- `doctor` passes for the gateway (in its Render Shell) and for the Workflows + service (run locally with that service's values); and - applying `factory:triage` produces an App-authored status comment. ## Required inputs @@ -334,34 +335,55 @@ The user confirms the webhook is active. ## Phase 9: Verify the deployed services -Run `doctor` inside each service's Render Shell so it reads that service's -deployed environment. +`doctor` checks the environment it runs in. It is read-only and does not +print secret values. -### Human checkpoint or agent action with Render access +### Human checkpoint or agent action with Render access: the gateway -In `factory-gateway`: +Run `doctor` in the `factory-gateway` Render Shell, so it reads the deployed +environment: ```bash npm run doctor -- --service gateway ``` -In `factory-workflows`: +### Agent action: the Workflows service + +A Workflows service has no Shell: Render starts a short-lived instance for +each task run. Run `doctor` locally with the values `factory-workflows` holds. +`.env` already has `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY` from Phase 4. +Ask the user to add these to `.env` without pasting them into chat: + +- `RENDER_API_KEY`, `RENDER_WORKSPACE_ID`, and `ANTHROPIC_API_KEY`, matching + the service; and +- `RENDER_WORKFLOW_SLUG`, the deployed workflow's slug. Render may add a + suffix to the service name (such as `factory-workflows-a1b2`); the + gateway's `RENDER_WORKFLOW_SLUG` shows the exact value. + +Then run: ```bash npm run doctor -- --service workflows ``` -`doctor` is read-only and does not print secret values. +### Expected warning + +Until GitHub has delivered a webhook, `doctor` warns "no webhook deliveries +yet, so the hook may be inactive". This is expected before the smoke test and +does not block setup. If it persists after Phase 10, recheck Phase 8. ### Pass condition -Both commands exit successfully with every required check passing. +Both commands exit successfully with every required check passing. Warnings +do not block. ### Recovery - Gateway failure: check webhook secret, Render API key, and workflow slug. - Workflows failure: check Render workspace ID, Render API key, GitHub App credentials, provider key, and registered tasks. +- "No tasks found" for the workflow: `RENDER_WORKFLOW_SLUG` in `.env` doesn't + match the deployed slug. - App failure: compare required permissions and events with Phase 4. - Label or installation failure: rerun `npm run setup:github` after fixing App access. @@ -407,6 +429,8 @@ snapshot. - `factory:working` appears while the task is running. - Triage ends with questions, a specification, or an intentional decline. - The status comment includes the final outcome or a useful failure. +- Optional: the `factory-workflows` logs contain a `run.finished` JSON line + for the run (`render logs -r --text run.finished`). If the status comment never appears, use [Operations](../operations.md#no-status-comment). diff --git a/tests/agents-catalog.test.ts b/tests/agents-catalog.test.ts index a5a3882..b3f8989 100644 --- a/tests/agents-catalog.test.ts +++ b/tests/agents-catalog.test.ts @@ -15,4 +15,14 @@ describe("agent authority", () => { expect(writers).toEqual(["implementer"]); expect(agents.every((agent) => agent.tools.length > 0)).toBe(true); }); + + it("rejects unknown fields in every object an agent can submit, so a misspelled field is named in the error", () => { + const looseObjects = (schema: unknown, path: string): string[] => { + if (typeof schema !== "object" || schema === null) return []; + const node = schema as Record; + const own = node.type === "object" && node.additionalProperties !== false ? [path || "(root)"] : []; + return [...own, ...Object.entries(node).flatMap(([key, value]) => looseObjects(value, `${path}/${key}`))]; + }; + for (const agent of agents) expect(looseObjects(agent.outputJsonSchema, ""), agent.id).toEqual([]); + }); }); diff --git a/tests/cancel.test.ts b/tests/cancel.test.ts index 9286c5b..8d0cb62 100644 --- a/tests/cancel.test.ts +++ b/tests/cancel.test.ts @@ -91,6 +91,20 @@ describe("cancel task", () => { expect(events[1]).toMatchObject({ traceId: "trn-cancel", runId: "trn-cancel", repo: REPO }); }); + it("clears a status left running by a run that already stopped, and terminates its leftover sandbox", async () => { + const { gh, comments, labels, add } = fakeGitHub("write"); + const marker = renderMarker("status", { runId: "trn-dead", stage: "triage", title: "Triage", state: "running", sandboxId: "sbx-dead" }); + add(BOT, `running\n\n${marker}`); + const runs = fakeRuns({}); + const result = await cancelRun(ctx, { repo: REPO, installationId: 5, number: 12, kind: "issue", actor: "alice" }, { gh, runs }); + expect(result).toEqual({ cancelled: false, runId: "trn-dead", reason: "already_stopped" }); + expect(runs.cancelTaskRun).not.toHaveBeenCalled(); + expect(runs.terminateSandbox).toHaveBeenCalledWith("sbx-dead"); + expect(comments).toHaveLength(1); + expect(comments[0]?.body).toContain("**Triage** was cancelled: cleared by @alice; the run had already stopped without finishing."); + expect(labels.has(WORKING_LABEL)).toBe(false); + }); + it("refuses an actor without permission and leaves the run alone", async () => { const { gh, comments, add } = fakeGitHub("read"); add(BOT, renderMarker("status", { runId: "trn-1", stage: "triage", state: "running" })); From 9cb69aaf0364c3a1583581a2229cef705535afed Mon Sep 17 00:00:00 2001 From: Jacob Prall Date: Sat, 24 Oct 2026 16:12:08 -0700 Subject: [PATCH 2/4] Rename Software Factory to software-factory Use the repository's own spelling for the product name in docs, the runbook, the Blueprint comment, and the GitHub App description that setup:app sends. Agent prompts keep "an unattended software factory", which describes the concept rather than naming the product. --- README.md | 4 ++-- docs/architecture.md | 2 +- docs/design-notes.md | 2 +- docs/quickstart.md | 2 +- docs/runbooks/setup.md | 12 ++++++------ render.yaml | 2 +- scripts/lib/manifest.ts | 2 +- 7 files changed, 13 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 2b26878..67038cb 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# Software Factory +# software-factory -Software Factory turns a labeled GitHub issue into an approved specification, +software-factory turns a labeled GitHub issue into an approved specification, a verified draft pull request, and an automated code review. People approve every transition; the factory never approves or merges changes. diff --git a/docs/architecture.md b/docs/architecture.md index 6ff758a..cfa916b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # Architecture -Software Factory separates model-directed work from authority. Agents inspect +software-factory separates model-directed work from authority. Agents inspect and modify code inside short-lived Render Sandboxes; trusted Workflows code decides what may be written to GitHub. diff --git a/docs/design-notes.md b/docs/design-notes.md index 10717a1..6bb5236 100644 --- a/docs/design-notes.md +++ b/docs/design-notes.md @@ -1,6 +1,6 @@ # Design notes -These are the reusable ideas behind Software Factory. They are explanatory, +These are the reusable ideas behind software-factory. They are explanatory, not required for deployment or operation. ## Let existing systems own their state diff --git a/docs/quickstart.md b/docs/quickstart.md index 2c7fd70..c017755 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -47,7 +47,7 @@ directly in `.env` or the Render Dashboard. Open the fork in your coding agent and ask: ```text -Set up Software Factory by following docs/runbooks/setup.md. +Set up software-factory by following docs/runbooks/setup.md. Configure using base branch . Stop at every human checkpoint and do not expose secrets. ``` diff --git a/docs/runbooks/setup.md b/docs/runbooks/setup.md index 685db92..c7bde20 100644 --- a/docs/runbooks/setup.md +++ b/docs/runbooks/setup.md @@ -1,12 +1,12 @@ # Setup runbook -Use this runbook when helping someone deploy Software Factory for the first +Use this runbook when helping someone deploy software-factory for the first time. Follow it in order. Do not claim setup is complete until `doctor` passes for both services and a labeled issue produces a factory status comment. ## Goal -Deploy Software Factory from the user's fork, connect it to one GitHub +Deploy software-factory from the user's fork, connect it to one GitHub repository, and complete one Triage run. ## Success criteria @@ -58,7 +58,7 @@ Render—not in chat, issues, commits, or command output: ### Agent action -Confirm the current directory is the Software Factory root and inspect: +Confirm the current directory is the software-factory root and inspect: - `.node-version`; - `package.json`; @@ -400,7 +400,7 @@ disposable issue in a configured test repository. Suggested issue: ```text -Title: Software Factory setup smoke test +Title: software-factory setup smoke test Confirm the repository can be inspected and return a minimal specification. Do not request a code change. @@ -412,7 +412,7 @@ when the user has approved CLI writes: ```bash gh issue create \ --repo \ - --title "Software Factory setup smoke test" \ + --title "software-factory setup smoke test" \ --body "Confirm the repository can be inspected and return a minimal specification. Do not request a code change." gh issue edit \ @@ -440,7 +440,7 @@ If the status comment never appears, use Report setup using this format without including secrets: ```text -Software Factory setup: complete +software-factory setup: complete Fork: Configured repositories: diff --git a/render.yaml b/render.yaml index 5e2cd09..622cfdf 100644 --- a/render.yaml +++ b/render.yaml @@ -1,4 +1,4 @@ -# Software Factory: two services. +# software-factory: two services. # - factory-gateway: verifies GitHub webhooks and starts the matching Workflows task. # - factory-workflows: runs every task (agents, snapshot refresh, cancel). It has no `repo` field, so it # deploys from the same repository as this Blueprint, including forks. diff --git a/scripts/lib/manifest.ts b/scripts/lib/manifest.ts index d19b908..a9c9fca 100644 --- a/scripts/lib/manifest.ts +++ b/scripts/lib/manifest.ts @@ -27,7 +27,7 @@ export function buildManifest(opts: { name: string; redirectUrl: string }): Mani return { name: opts.name, url: HOMEPAGE, - description: "Software Factory: labeled issue to spec to draft PR to review, on Render.", + description: "software-factory: labeled issue to spec to draft PR to review, on Render.", hook_attributes: { url: PLACEHOLDER_WEBHOOK_URL, active: false }, redirect_url: opts.redirectUrl, public: false, From 637804240377e5e06eca413cab67e9082a73eea2 Mon Sep 17 00:00:00 2001 From: Jacob Prall Date: Sat, 24 Oct 2026 16:13:32 -0700 Subject: [PATCH 3/4] Group the Blueprint's services in a software-factory project render.yaml now creates factory-gateway and factory-workflows in the production environment of a software-factory project, so a deployment shows up as one project in the Render Dashboard. The docs that describe what the Blueprint creates, and the add-a-database recipe, follow the new layout. --- CONTRIBUTING.md | 5 +-- docs/architecture.md | 2 +- docs/operations.md | 3 +- docs/quickstart.md | 4 +-- docs/runbooks/setup.md | 2 +- render.yaml | 78 ++++++++++++++++++++++-------------------- 6 files changed, 50 insertions(+), 44 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 03faf77..33642bf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -174,8 +174,9 @@ The factory keeps no database: GitHub and Render Workflows hold the state it needs, and telemetry goes to logs. When a feature needs its own data, such as a spec library, budgets, or run history you query: -1. Add a Render Postgres database to `render.yaml` and wire `DATABASE_URL` - into the services that need it with `fromDatabase`. +1. Add a Render Postgres database to the `production` environment's + `databases` list in `render.yaml`, and wire `DATABASE_URL` into the + services that need it with `fromDatabase`. 2. Run migrations from the gateway's `preDeployCommand`, so each deploy migrates before new code starts. 3. Use the internal connection string on Render. The external one is for diff --git a/docs/architecture.md b/docs/architecture.md index cfa916b..bfd4638 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -17,7 +17,7 @@ flowchart LR Snapshot -.->|restore| Sandbox ``` -The Blueprint creates two services: +The Blueprint creates two services in a `software-factory` project: - **Gateway** verifies GitHub webhooks and maps each delivery to one task. - **Workflows** runs the claiming tasks, agents, cancellation, snapshot refresh, and diff --git a/docs/operations.md b/docs/operations.md index 8ea2379..1c3bcf7 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -7,7 +7,8 @@ Use this guide after completing the [Quickstart](quickstart.md). While a task is running, its issue or pull request has `factory:working`. The factory's status comment shows progress, the current run ID, and final links. -Find that run under `factory-workflows` in the Render Dashboard to inspect +Find that run under `factory-workflows`, in the `software-factory` project in +the Render Dashboard, to inspect logs, inputs, and its result. Every task run ends with a `run.finished` JSON line, and every agent run with diff --git a/docs/quickstart.md b/docs/quickstart.md index c017755..9523455 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -14,8 +14,8 @@ only you should take. - admin access to each target GitHub repository; and - the Node.js version in [`.node-version`](../.node-version). -The Blueprint creates paid Render resources: a Starter web service and a -Workflows service. Sandbox and model usage are charged separately. +The Blueprint creates paid Render resources in a `software-factory` project: a +Starter web service and a Workflows service. Sandbox and model usage are charged separately. ## What the setup agent will do diff --git a/docs/runbooks/setup.md b/docs/runbooks/setup.md index c7bde20..e01145e 100644 --- a/docs/runbooks/setup.md +++ b/docs/runbooks/setup.md @@ -238,7 +238,7 @@ The user confirms installation on every exact configured repository. ### Agent action Explain that [`render.yaml`](../../render.yaml) creates paid resources in -Oregon: +Oregon, in the `production` environment of a `software-factory` project: - `factory-gateway`, a Starter web service; and - `factory-workflows`, a Workflows service. diff --git a/render.yaml b/render.yaml index 622cfdf..cae47ad 100644 --- a/render.yaml +++ b/render.yaml @@ -1,43 +1,47 @@ -# software-factory: two services. +# software-factory: two services in a software-factory project. # - factory-gateway: verifies GitHub webhooks and starts the matching Workflows task. # - factory-workflows: runs every task (agents, snapshot refresh, cancel). It has no `repo` field, so it # deploys from the same repository as this Blueprint, including forks. # Run and agent telemetry is logged as JSON lines; find it in each service's logs or a log stream. -services: - - type: web - name: factory-gateway - region: oregon - runtime: node - plan: starter - buildCommand: npm ci - startCommand: npm run start:gateway - healthCheckPath: /ready - envVars: - - key: GITHUB_WEBHOOK_SECRET - sync: false - - key: RENDER_API_KEY - sync: false - - key: RENDER_WORKFLOW_SLUG - fromService: - type: workflow - name: factory-workflows - property: slug +projects: + - name: software-factory + environments: + - name: production + services: + - type: web + name: factory-gateway + region: oregon + runtime: node + plan: starter + buildCommand: npm ci + startCommand: npm run start:gateway + healthCheckPath: /ready + envVars: + - key: GITHUB_WEBHOOK_SECRET + sync: false + - key: RENDER_API_KEY + sync: false + - key: RENDER_WORKFLOW_SLUG + fromService: + type: workflow + name: factory-workflows + property: slug - - type: workflow - name: factory-workflows - region: oregon - runtime: node - buildCommand: npm ci - startCommand: npm run start:workflows - envVars: - - key: RENDER_API_KEY - sync: false - - key: RENDER_WORKSPACE_ID - sync: false - - key: GITHUB_APP_ID - sync: false - - key: GITHUB_APP_PRIVATE_KEY - sync: false - - key: ANTHROPIC_API_KEY - sync: false + - type: workflow + name: factory-workflows + region: oregon + runtime: node + buildCommand: npm ci + startCommand: npm run start:workflows + envVars: + - key: RENDER_API_KEY + sync: false + - key: RENDER_WORKSPACE_ID + sync: false + - key: GITHUB_APP_ID + sync: false + - key: GITHUB_APP_PRIVATE_KEY + sync: false + - key: ANTHROPIC_API_KEY + sync: false From 10d5d0bdf231d0891eaab6b058c98b7086ed1498 Mon Sep 17 00:00:00 2001 From: Jacob Prall Date: Sat, 24 Oct 2026 17:39:29 -0700 Subject: [PATCH 4/4] Rewrite documentation for clarity --- AGENTS.md | 232 ++++++++----------- CONTRIBUTING.md | 217 +++++------------- README.md | 146 ++++++------ docs/architecture.md | 291 +++++++++++++----------- docs/configuration.md | 201 ++++++++++++---- docs/design-notes.md | 61 ----- docs/operations.md | 202 ----------------- docs/quickstart.md | 84 +++---- docs/runbooks/extend.md | 261 +++++++++++++++++++++ docs/runbooks/setup.md | 491 ++++++++++++++++++---------------------- 10 files changed, 1062 insertions(+), 1124 deletions(-) delete mode 100644 docs/design-notes.md delete mode 100644 docs/operations.md create mode 100644 docs/runbooks/extend.md diff --git a/AGENTS.md b/AGENTS.md index 25b5c67..2fcf00b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,145 +1,103 @@ # Repository guide for agents -This file applies to the whole repository. Read -[docs/architecture.md](docs/architecture.md) for how the system works and -[docs/design-notes.md](docs/design-notes.md) for why it is built this way. - -## Task routing - -- **Set up or deploy the factory:** follow - [docs/runbooks/setup.md](docs/runbooks/setup.md) in order. Do not skip its - human checkpoints or claim completion before `doctor` passes for both - services and the Triage smoke test passes. -- **Change repositories, labels, models, or limits:** use - [docs/configuration.md](docs/configuration.md). -- **Diagnose a live deployment:** use - [docs/operations.md](docs/operations.md). -- **Add an agent, claiming task, or trigger:** use - [CONTRIBUTING.md](CONTRIBUTING.md). - -## What this is - -software-factory turns a labeled GitHub issue into a spec, a draft PR, and a COMMENT review. Three claiming tasks (Triage, Spec-to-PR, PR Reviewer) run on Render Workflows, each holding a claim on its issue or PR while it works. Agents run in Render Sandboxes restored from a per-repository snapshot. The status comment on the issue or PR names the run that holds the claim. There is no database: GitHub and Workflows hold all state, and run telemetry is logged as JSON lines. - -Two processes deploy: - -- `app/entrypoints/gateway.ts`: the gateway web service. Verifies webhooks, routes each to one task, and calls `startTask` with the delivery ID as the idempotency key. -- `app/entrypoints/workflows.ts`: the Workflows service. Importing `app/workflows/index.ts` registers every task. - - -## Module map - -```text -factory.config.ts Repositories (base branch, install, verify), labels, permission, model tiers, publish and repair limits, sandbox timeout -app/ - entrypoints/ Gateway and Workflows process entry points - config/ Loads factory.config.ts; RepoContext and resolveRepo; TASKS names; modelFor; requireEnv - schemas/ Zod schemas: task inputs (tasks.ts) and agent outputs (triage, spec, implementation, review) - gateway/ Hono app: signature check, body cap, the routing table (routes.ts), which validates task inputs - workflows/ The tasks: triage/, spec-to-pr/, pr-review/, cancel.ts, refresh-snapshot.ts; claiming-task.ts and run-verify.ts are shared - claims/ withClaim lifecycle: permission, the status-comment claim, takeover and sandbox cleanup, checkpoints, expected failures - agents/ One directory with index.ts and prompt.md per agent; shared defineAgent/runAgent runtime - runner/ The agent runner: one Pi agent loop, tools, read-only shell policy, submit. Bundled into the snapshot - sandbox/ Sandbox API client with pacing and retries; snapshot build, openSandbox, withSandbox, verification - changeset/ Collects, validates, and publishes sandbox changes through the Git Data API - github/ REST client, App JWT and installation tokens, Git Data API, hidden comment markers - format/ Renders specs, questions, PR bodies, and reviews as GitHub markdown; sanitizes model text - events/ emit(): run.finished and agent.finished as JSON lines on stdout -sandbox/ Shell scripts copied into the snapshot (run-agent, as-runner, verify, collect-changes, ...) -scripts/ setup-app, setup-github, doctor -tests/ Vitest unit tests; fakes in tests/helpers/ -``` - -Dependencies point one way, from entry points down to leaves: - -```text -entrypoints → gateway, workflows -workflows → claims, changeset, agents, sandbox, github, format, events, schemas, config -claims → agents, sandbox, github, events, config -agents → sandbox, runner, events, schemas, config -changeset → sandbox, github, config -sandbox → runner (to bundle it), github (read tokens), config -format → schemas, config -gateway → schemas, config -github → config -runner, events, schemas, config → nothing in app/ -``` - -Two edges must never appear: - -- **Nothing in `runner/` imports from the rest of `app/`.** esbuild bundles `runner/main.ts` into a self-contained file that runs in the sandbox with only Node. -- **`gateway/` never imports `github/`, `sandbox/`, `agents/`, or `workflows/`.** The gateway holds no GitHub credential and runs no model. - -Import another module through its `index.ts` barrel where one exists. A barrel exports what other modules use; keep module-internal helpers out of it. - -## Conventions - -- Strict TypeScript, ESM, NodeNext. Relative imports end in `.js`. No path aliases. -- Biome is configured for tabs and a line width of 150 (`biome.json`); CI runs `biome lint`, which must pass with no warnings. -- Validate every external or cross-process input with Zod at the boundary: webhook payloads, task inputs, agent output, markers, sandbox output, environment. -- Build every agent output object with `z.strictObject`, so a misspelled field is rejected by name in the error the model sees. `tests/agents-catalog.test.ts` enforces it. -- Task inputs and results stay small and JSON-serializable. Each task opens - one sandbox with `withSandbox`, which closes it within the same task. -- Throw `ExpectedFailure` for an expected failure whose message is safe to show on GitHub. `withClaim` also treats an agent - that stops on its own (turn limit, timeout, no or invalid submission) as expected; every other error fails the task. - Outcome names are snake_case. -- Quote every shell argument you build with `shq` (`app/sandbox/shell.ts`). Anything long or security-sensitive that runs in a sandbox belongs in a script in `sandbox/`, which CI checks with shellcheck. -- Model text that reaches GitHub goes through the helpers in `app/format/`. -- Task implementations take GitHub, sandboxes, `runAgent`, and run control as injectable dependencies. Tests never call live Render, GitHub, or model APIs. -- Keep the suite focused; it currently has 100+ tests. Test durable behavior at process and trust - boundaries: authenticated inputs, user-visible outcomes, authority, - cleanup, and security invariants. Prefer one end-to-end happy path and the - meaningful failure modes over helper permutations. Do not assert private - helper calls, incidental command sequences, task registration, barrel - exports, or exact intermediate formatting. Assert ordering only when the - ordering is itself a security contract, such as scrubbing a credential - before untrusted code runs. -- Add or update a test when externally observable behavior or a security - invariant changes; refactors with unchanged behavior should not require - test changes. -- Write comments only for constraints the code can't show (platform limits, security reasons, ordering requirements), not to narrate the next line. - -## Invariants - -Don't weaken these without an explicit change to the security model -([docs/architecture.md](docs/architecture.md#trust-boundaries) says where -each is enforced): - -1. The gateway verifies the webhook signature before parsing the body. -2. Every run checks the triggering actor's live repository permission before doing anything. -3. Only workflow code writes to GitHub, and every write finds its marker before creating anything. -4. No write credential enters a sandbox. A read-only token is present only to clone or fetch. -5. Tools run only in sandboxes, never in a Workflows task. -6. Models never choose sandbox IDs, branches, repositories, or GitHub targets. -7. A PR opens only after GitHub's tree SHA equals the verified sandbox tree. -8. Automation ends at a draft PR and a COMMENT review. It never approves, requests changes, marks a PR ready, or merges. -9. Spec-to-PR implements the spec as it stood when the approval label was applied. -10. Every sandbox is created with a timeout and closed by `withSandbox` (in `finally`). A claiming task records its sandbox in the status comment, so the run or `/factory cancel` that stops it can terminate the sandbox: Render kills a cancelled task without running `finally`. - -`tests/agents-catalog.test.ts` protects unique agent identities, tool authority, and strict output schemas. Change it only on purpose. - -## Common changes - -- **An agent or a claiming task:** follow - [CONTRIBUTING.md](CONTRIBUTING.md#customize-an-agent) or - [CONTRIBUTING.md](CONTRIBUTING.md#add-a-claiming-task). -- **A configuration key:** add it to `FactoryConfig` in `factory.config.ts` - with a doc comment, and to - [docs/configuration.md](docs/configuration.md#behavior-and-limits). -- **An environment variable:** add it to `render.yaml`, `.env.example`, - `SERVICE_ENV` in `scripts/lib/requirements.ts` (so `doctor` checks it), and - [docs/configuration.md](docs/configuration.md#environment-variables). -- **A webhook event, label, or App permission:** update `scripts/lib/requirements.ts`, which `setup:app`, `setup:github`, and `doctor` all read. -- **The runner or `sandbox/`:** these are baked into each snapshot, so they reach sandboxes only after a rebuild (the next push to each base branch, or a `refresh-snapshot` run). +This file applies to the whole repository. + +## Use the right guide + +- First deployment: + [docs/runbooks/setup.md](docs/runbooks/setup.md) +- Repositories, labels, models, permissions, limits, or snapshots: + [docs/configuration.md](docs/configuration.md) +- Agents, Workflows tasks, or GitHub triggers: + [docs/runbooks/extend.md](docs/runbooks/extend.md) +- State, coordination, publishing, or security: + [docs/architecture.md](docs/architecture.md) +- Local development: + [CONTRIBUTING.md](CONTRIBUTING.md) + +Follow runbooks in order. Stop at every human checkpoint. Do not claim setup +is complete until both `doctor` checks and the Triage smoke test pass. + +## System boundaries + +software-factory deploys two processes: + +- `app/entrypoints/gateway.ts` verifies webhooks and starts tasks. +- `app/entrypoints/workflows.ts` registers and runs tasks. + +Agents run inside Render Sandboxes. Only code in the Workflows service writes +to GitHub. GitHub and Render Workflows hold application state; there is no +database. + +Two dependency rules are mandatory: + +1. `app/runner/` imports nothing from the rest of `app/`. It is bundled into + a self-contained Sandbox program. +2. `app/gateway/` does not import GitHub authentication, Sandboxes, agents, or + workflows. The gateway holds no GitHub credential and runs no model. + +Read the canonical +[security invariants](docs/architecture.md#security-invariants) before +changing credentials, claims, Sandboxes, publishing, or GitHub writes. + +## Code conventions + +- Use strict TypeScript, ESM, and NodeNext. Relative imports end in `.js`. + There are no path aliases. +- Biome uses tabs and a line width of 150. +- Import public module APIs through `index.ts` barrels. Do not export + module-internal helpers. +- Validate every external or cross-process input with Zod at its boundary. +- Build agent output objects with `z.strictObject`. Bound strings and arrays. +- Keep task inputs and results small and JSON-serializable. +- Open Sandboxes through `withSandbox`. In claiming tasks, call + `claim.recordSandbox()` immediately after opening one. +- Use `ExpectedFailure` only for expected messages safe to publish on GitHub. + Keep outcome names in snake_case. +- Quote generated shell arguments with `shq`. Put long or + security-sensitive Sandbox commands in `sandbox/` scripts. +- Pass model text through `app/format/` before writing it to GitHub. +- Inject GitHub, Sandbox, agent, and run-control dependencies into task + implementations. Tests use fakes, never live services. +- Comment security reasons, platform constraints, and required ordering. Do + not narrate obvious code. + +## Tests + +Test behavior at process and trust boundaries: + +- authenticated input; +- user-visible output; +- authority and target selection; +- cleanup and cancellation; and +- security invariants. + +Do not test private helper calls, incidental command sequences, task +registration, barrel exports, or formatting that users cannot observe. Assert +ordering only when it is part of a security contract. + +Update tests when observable behavior or an invariant changes. A refactor +with unchanged behavior should not require new assertions. + +`tests/agents-catalog.test.ts` protects unique agent IDs, tool authority, and +strict output schemas. + +## Cross-cutting changes + +- Configuration key: update `FactoryConfig` in `factory.config.ts` and + `docs/configuration.md`. +- Environment variable: update `render.yaml`, `.env.example`, `SERVICE_ENV` + in `scripts/lib/requirements.ts`, and `docs/configuration.md`. +- App permission, event, or label: update + `scripts/lib/requirements.ts`, setup behavior, and doctor tests. +- `app/runner/` or `sandbox/`: refresh every repository snapshot after + deployment. ## Verify ```bash -npm run check # Biome lint, tsc --noEmit, all Vitest tests -shellcheck sandbox/*.sh sandbox/build/*.sh -render blueprints validate render.yaml # when render.yaml changes -npm audit --omit=dev --audit-level=high +npm run check ``` -For one test file: `npx vitest run tests/gateway.test.ts`. `npm run doctor` is read-only; see -[Operations](docs/operations.md#check-a-deployment) for local and deployed usage. +See [Contributing](CONTRIBUTING.md#verify-changes) for targeted and +change-specific checks. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 33642bf..506c85d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,20 +1,24 @@ # Contributing -Read [Architecture](docs/architecture.md) before changing run coordination, -Sandbox boundaries, credentials, or publishing. +Read [Architecture](docs/architecture.md) before changing claims, credentials, +Sandboxes, publishing, or GitHub authority. Use the +[extension runbook](docs/runbooks/extend.md) for agents, Workflows tasks, and +GitHub triggers. ## Local development -Local development requires Render CLI 2.29 or later. It still uses -real Render Sandboxes, a real GitHub App, and a real model. Use a separate -test repository and preferably a separate App. +Local development requires Render CLI 2.29 or later. It still uses real +Render Sandboxes, a real GitHub App, and a real model. Use a test repository +and, when possible, a separate App. + +Install dependencies and create a local environment file: ```bash npm ci cp .env.example .env ``` -Fill `.env`, then start each process in its own terminal: +Fill `.env` without committing it. Start each process in its own terminal: ```bash # Terminal 1 @@ -24,20 +28,23 @@ npm run dev:workflows RENDER_USE_LOCAL_DEV=true npm run dev:gateway ``` -Expose port 3000 through a public HTTPS tunnel and point the App at it: +Expose port 3000 over public HTTPS: ```bash cloudflared tunnel --url http://localhost:3000 npm run setup:github -- --gateway-url https:// ``` -A GitHub App has one webhook URL. Repointing a production App interrupts its -production deliveries. +A GitHub App has one webhook URL. Pointing a production App at local +development interrupts production deliveries. + +### Start a task directly -You can also invoke a local task directly: +List and invoke local tasks with the Render CLI: ```bash render workflows tasks list --local + render workflows tasks runs start --local \ --task \ --input='[{...}]' @@ -48,14 +55,16 @@ Task input schemas are in ## Verify changes +Run the standard check before handing off: + ```bash npm run check ``` -This runs Biome, TypeScript, and Vitest. Tests fake GitHub, Render, and model +It runs Biome, TypeScript, and Vitest. Tests fake GitHub, Render, and model calls. -When relevant, also run: +Run additional checks when relevant: ```bash shellcheck sandbox/*.sh sandbox/build/*.sh @@ -63,163 +72,53 @@ render blueprints validate render.yaml npm audit --omit=dev --audit-level=high ``` -For one test: +Run one test file with: ```bash npx vitest run tests/gateway.test.ts ``` -## Customize an agent +## Extend the factory -An agent consists of a Markdown prompt and typed definition: +Follow [docs/runbooks/extend.md](docs/runbooks/extend.md) to: -```text -app/agents// - index.ts - prompt.md -``` - -```typescript -export const implementer = defineAgent({ - id: "implementer", - prompt: "implementer/prompt.md", - model: "large", - tools: ["read", "ls", "bash", "edit", "write"], - output: Implementation, - maxTurns: 150, -}); -``` +- add a typed agent and prompt; +- add a claiming or maintenance task; +- route a GitHub event, label, or command; +- update GitHub App requirements; +- test authority and user-visible behavior; and +- identify required deploys and snapshot refreshes. -The definition fixes: - -- a unique ID; -- a prompt relative to `app/agents/`; -- a configured model tier; -- tools selected from `read`, `ls`, `bash`, `edit`, and `write`; -- a bounded Zod output schema, built from `z.strictObject` so unknown fields - are rejected by name; -- a maximum number of model calls; and -- an optional thinking level. - -Put repository- and run-specific context in the typed message passed to -`runAgent`, not in the system prompt. Tell the model to call `submit` exactly -once; the tool supplies the output schema. - -Only an implementer should normally receive `edit` or `write`. The read-only -shell policy is a behavioral guard, not a security boundary. - -To add an agent: - -1. add its prompt and definition under `app/agents//`; -2. export it from `app/agents/index.ts`; -3. add or reuse a bounded schema in `app/schemas/`; -4. call it from a task with that task's Sandbox and `claim.agentOptions(sandbox)`; -5. add it to `tests/agents-catalog.test.ts`; and -6. run `npm run check`. - -Prompt changes require a Workflows deploy. Changes under `app/runner/` or -`sandbox/` also require repository snapshot refreshes. - -## Add a claiming task - -A claiming task acts on one issue or PR while it holds a claim on it. -Triage, Spec-to-PR, and PR Reviewer are claiming tasks. The claim is the -status comment: the run it names holds it. Declare a claiming task with -`claimingTask()` (`app/workflows/claiming-task.ts`). It registers the -Workflows task, validates the input, resolves the repository, and runs your -work inside `withClaim` (`app/claims/claim.ts`). `withClaim` checks the -actor's permission, stops the previous run and takes over the status comment, -keeps the comment current, and reports how the run ended. - -1. Add the task name to `TASKS` in `app/config/index.ts`. -2. Add a bounded input schema in `app/schemas/tasks.ts`. -3. Declare the task under `app/workflows//` with - `claimingTask({ name, title, input, subject, timeoutSeconds }, work)`. - `title` is how the status comment names the task. -4. Open Sandboxes through `withSandbox`, and call - `claim.recordSandbox(sandbox)` first thing, so a run that replaces yours - can terminate its Sandbox. -5. Call `claim.progress()` before major steps and `claim.checkpoint()` before - durable writes. -6. Throw `ExpectedFailure` only for expected messages safe to show on GitHub. -7. Import the task module from `app/workflows/index.ts`. -8. Route an authenticated webhook to it in `app/gateway/routes.ts`. -9. Update setup requirements and add boundary-focused tests. - -Claiming tasks never retry automatically, because a retry can repeat model -spend and durable work. A person reapplies a label to start a fresh run. - -## Add or change a GitHub trigger - -The gateway routing table is a pure mapping from authenticated event payload -to one validated task input. - -1. Update `route()` in `app/gateway/routes.ts`. -2. Update `REQUIRED_EVENTS`, `REQUIRED_PERMISSIONS`, or `LABELS` in - `scripts/lib/requirements.ts`. -3. Update the existing App when permissions or subscriptions changed. -4. Rerun `npm run setup:github`. -5. Add route and setup/doctor tests. - -Use the GitHub delivery ID as the task idempotency key. Derive repositories, -installations, actors, subjects, and commits from the authenticated payload. -Never accept write destinations from model output. - -Every GitHub write should update or reuse an App-authored artifact marker. -Workflows idempotency alone has a finite scope and is not sufficient. +The runbook is the canonical procedure. Keep implementation details in code +and architectural reasons in [Architecture](docs/architecture.md). ## Add persistent state -The factory keeps no database: GitHub and Render Workflows hold the state it -needs, and telemetry goes to logs. When a feature needs its own data, such as -a spec library, budgets, or run history you query: +The default template has no database. GitHub owns collaboration artifacts, +Render Workflows owns task execution, and telemetry goes to logs. -1. Add a Render Postgres database to the `production` environment's - `databases` list in `render.yaml`, and wire `DATABASE_URL` into the - services that need it with `fromDatabase`. -2. Run migrations from the gateway's `preDeployCommand`, so each deploy - migrates before new code starts. -3. Use the internal connection string on Render. The external one is for - local tools and requires TLS. -4. Keep run coordination in the status comment. A database that also decides - who owns an issue becomes a second source of truth to reconcile. - -## Code structure - -```text -app/ - entrypoints/ Gateway and Workflows process entry points - gateway/ Webhook authentication and routing - workflows/ Triage, Spec-to-PR, PR Reviewer, cancel, snapshots - claims/ withClaim: permission, status-comment claim, takeover, failure reporting - agents/ Agent definitions and host-side execution - runner/ Self-contained agent runtime bundled into snapshots - sandbox/ Sandbox and snapshot lifecycle - changeset/ Candidate-tree validation and publishing - github/ App authentication, REST, Git Data, and markers - events/ Run and agent telemetry as JSON log lines - schemas/ Cross-process Zod contracts -sandbox/ Runtime scripts copied into snapshots -scripts/ Setup and doctor -tests/ Unit and boundary tests with fakes -``` - -Two dependency boundaries are mandatory: - -- `app/runner/` must not import from the rest of `app/`; it is bundled into a - self-contained Sandbox program. -- `app/gateway/` must not import GitHub authentication, Sandboxes, agents, or - workflows; the gateway holds no GitHub credential and runs no model. - -## Documentation rules +Add a database only for new data with its own lifecycle, such as a searchable +spec library, budgets, or retained run history: -Keep each fact in its canonical document: - -- agent deployment protocol: `docs/runbooks/setup.md`; -- short human setup guide: `docs/quickstart.md`; -- supported settings: `docs/configuration.md`; -- runtime procedures and failures: `docs/operations.md`; -- state, boundaries, and invariants: `docs/architecture.md`; and -- contributor workflows: this file. - -Link to the canonical explanation instead of copying it. +1. Add a Render Postgres database to the `production` environment's + `databases` list in `render.yaml`. +2. Wire `DATABASE_URL` into only the services that need it. +3. Run migrations from a service `preDeployCommand`. +4. Use the internal connection string on Render. Local tools use the external + connection with TLS. + +Do not move claim ownership into the database. The status comment must remain +the single source of truth for the run that owns an issue or pull request. + +## Documentation ownership + +- `README.md`: product entry point. +- `docs/quickstart.md`: human setup handoff. +- `docs/runbooks/setup.md`: first-deployment agent protocol. +- `docs/configuration.md`: settings, run control, deployment checks, and + snapshot procedures. +- `docs/architecture.md`: state, boundaries, invariants, and accepted risks. +- `docs/runbooks/extend.md`: extension agent protocol. +- `CONTRIBUTING.md`: local development and verification. + +Link to the canonical document instead of copying its content. diff --git a/README.md b/README.md index 67038cb..fd0be5f 100644 --- a/README.md +++ b/README.md @@ -1,95 +1,93 @@ # software-factory -software-factory turns a labeled GitHub issue into an approved specification, -a verified draft pull request, and an automated code review. People approve -every transition; the factory never approves or merges changes. +software-factory turns GitHub issues into specifications and verified draft +pull requests. It also reviews pull requests. Labels start the three main +tasks; comments and base-branch pushes handle cancellation, follow-up Triage, +and snapshot refresh. The factory never approves or merges code. -It runs in your Render workspace using one web service, one Workflows -service, and Render Sandboxes. +It runs in your Render workspace as a web service, a Workflows service, and +short-lived Render Sandboxes. -**Start by [forking this repository](https://github.com/render-examples/software-factory/fork).** -The factory must deploy from your fork because `factory.config.ts` defines -which repositories it may access. - -## Workflow +## How it works ```text issue --factory:triage--> specification -specification --factory:spec-approved--> verified draft PR +specification --factory:spec-approved--> draft pull request pull request --factory:pr-review--> COMMENT review ``` -- **Triage** reads the issue, discussion, and code, then asks focused - questions, writes a specification, or declines. -- **Spec-to-PR** implements the exact approved specification, runs configured - verification, repairs bounded failures, and opens a draft pull request. +- **Triage** reads the issue, discussion, and repository. It asks questions, + writes a specification, or declines the request. +- **Spec-to-PR** implements the approved specification, runs your verification + commands, and opens a draft pull request. - **PR Reviewer** runs verification and reviews security, correctness, - performance, and tests. - -GitHub remains where people discuss and approve work. Render Workflows owns -durable execution. Sandboxes contain model-directed tools and repository -code. - -## Deploy - -You need a Render workspace with Sandboxes access in Oregon, a Render API -key, a dedicated spend-capped model API key, GitHub repository admin access, -and the Node.js version in `.node-version`. - -For the fastest path, ask a coding agent to follow -[`docs/runbooks/setup.md`](docs/runbooks/setup.md). It separates work the -agent can perform from GitHub, secret, and paid-resource checkpoints that -require you. - -1. Fork the repository and configure one target in `factory.config.ts`. -2. Run `npm ci` and `npm run setup:app -- --write-env`. -3. Install the generated GitHub App on the target repository. -4. Create a Render Blueprint from your fork and provide its secret values. -5. Run `npm run setup:github -- --gateway-url https://.onrender.com`. -6. Activate the webhook in GitHub App settings. -7. Run `npm run doctor`, then apply `factory:triage` to an issue. - -See the [Quickstart](docs/quickstart.md) for the short human guide or the -[Setup runbook](docs/runbooks/setup.md) for the complete agent protocol. - -## Security model - -- Agents and model-directed tools run only inside short-lived Sandboxes. -- No GitHub write credentials enter the Sandbox. -- Clone and fetch use temporary read-only credentials that are removed before - repository code or agents run. -- Models cannot choose repositories, branches, issue numbers, pull requests, - or GitHub write targets. -- Trusted Workflows code validates candidate changes and requires GitHub to - reproduce their tree SHA before publishing. -- Publishing rejects unsafe paths, workflow files, symlinks, submodules, - oversized changes, and branches containing human commits. - -Sandboxes use allow-all egress, and a model key exists in runner memory during -a call. Treat repository and pull request content as untrusted, and use a -dedicated Render workspace and provider-capped key when isolation matters. -Read the [trust boundaries and residual risks](docs/architecture.md#trust-boundaries) -before deploying against private code. + performance, and tests. It posts a non-approving GitHub COMMENT review. + +GitHub holds the discussion, labels, specifications, pull requests, and +reviews. Render Workflows runs the tasks. Sandboxes contain the checkout, +agent loop, and model-directed tools. + +## Set up + +Start by +[forking this repository](https://github.com/render-examples/software-factory/fork). +Deploy from your fork: its `factory.config.ts` defines the repositories the +factory may access. + +The recommended setup is agent-guided: + +1. Check the prerequisites in the [Quickstart](docs/quickstart.md). +2. Ask a coding agent to follow the + [Setup runbook](docs/runbooks/setup.md). +3. Complete the GitHub, secret, and paid-resource checkpoints when the agent + asks. + +The runbook configures the fork before deployment, creates a GitHub App, +deploys the Render Blueprint, connects the webhook, verifies both services, +and runs one Triage smoke test. + +## Security + +- Model-directed code and tools run inside Sandboxes, not on the Workflows + host. +- GitHub write credentials never enter a Sandbox. +- Clone and fetch credentials are read-only and removed before repository code + or agents run. +- Models propose typed results and file trees. Code in the Workflows service + chooses every repository, ref, branch, issue, pull request, and write target. +- Publishing rejects unsafe paths and modes, workflow files, symlinks, + submodules, oversized changes, and branches with human commits. +- GitHub must reproduce the candidate tree SHA before the factory publishes + it. + +Sandboxes use allow-all egress, and the model key exists in runner memory +during a call. Read the [trust boundaries](docs/architecture.md#trust-boundaries) +and [residual risks](docs/architecture.md#residual-risks) before using private +code or untrusted pull requests. ## Limits -- Tasks never chain automatically. -- Verification is command-based; browser testing is not built in. -- Turn limits and timeouts bound agents, but the factory has no hard dollar +- Main tasks do not chain. Labels, comments, commands, and base-branch pushes + start work through the gateway. +- Verification runs shell commands; browser testing is not built in. +- Turn limits and timeouts bound execution, but the factory has no hard dollar budget. -- Anyone with the configured GitHub permission can start model spend. -- PR Reviewer executes configured verification against the labeled PR code. -- Sandbox API calls share the Render workspace's rate limit. +- Anyone with `minimumPermission` can start model spend. +- PR Reviewer runs configured commands against the labeled pull request. +- Sandbox API calls count against the Render workspace's request limit. ## Documentation -- [Setup runbook](docs/runbooks/setup.md) — agent-guided deployment with verified checkpoints. -- [Quickstart](docs/quickstart.md) — deploy and complete the first run. -- [Configuration](docs/configuration.md) — repositories, labels, models, and limits. -- [Operations](docs/operations.md) — logs, cancellation, snapshots, and failures. -- [Architecture](docs/architecture.md) — state ownership and trust boundaries. -- [Contributing](CONTRIBUTING.md) — local development, agents, tasks, and triggers. -- [Design notes](docs/design-notes.md) — reusable patterns behind the system. +- [Quickstart](docs/quickstart.md) — what you need and how to start setup. +- [Setup runbook](docs/runbooks/setup.md) — agent protocol for the first + deployment. +- [Configuration](docs/configuration.md) — settings, logs, run control, + deployment checks, and snapshots. +- [Architecture](docs/architecture.md) — execution, state, coordination, and + trust boundaries. +- [Extension runbook](docs/runbooks/extend.md) — add agents, Workflows tasks, + and GitHub triggers. +- [Contributing](CONTRIBUTING.md) — local development and verification. ## License diff --git a/docs/architecture.md b/docs/architecture.md index bfd4638..6b8fcfe 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,8 +1,8 @@ # Architecture -software-factory separates model-directed work from authority. Agents inspect -and modify code inside short-lived Render Sandboxes; trusted Workflows code -decides what may be written to GitHub. +Agents inspect and modify repositories inside short-lived Render Sandboxes. +Code in the Workflows service validates their results and performs every +GitHub write. ## System @@ -10,205 +10,230 @@ decides what may be written to GitHub. flowchart LR GitHub -->|signed webhook| Gateway Gateway -->|start task| Workflows - Workflows -->|restore and execute| Sandbox - Sandbox -->|proposed result| Workflows - Workflows -->|validated writes| GitHub + Workflows -->|restore| Sandbox + Sandbox -->|typed result or candidate tree| Workflows + Workflows -->|validated write| GitHub Workflows -->|build| Snapshot Snapshot -.->|restore| Sandbox ``` -The Blueprint creates two services in a `software-factory` project: +The Blueprint deploys two services: -- **Gateway** verifies GitHub webhooks and maps each delivery to one task. -- **Workflows** runs the claiming tasks, agents, cancellation, snapshot refresh, and - GitHub publishing. +- **Gateway** verifies webhooks and maps each delivery to one task. +- **Workflows** runs tasks, manages Sandboxes, and writes to GitHub. -Each claiming task receives a fresh Sandbox restored from a repository snapshot. -Render Workflows owns task state and timeouts. GitHub owns the human -conversation and artifacts. The application has no custom queue, dispatcher, -heartbeat loop, or maintenance cron. +The application does not implement its own queue, dispatcher, heartbeat +process, maintenance loop, or database. -## Human-gated flow +## Workflow ```text issue --factory:triage--> specification -specification --factory:spec-approved--> verified draft PR +specification --factory:spec-approved--> draft pull request pull request --factory:pr-review--> COMMENT review ``` -The tasks never chain automatically. The factory cannot approve, request -changes, mark ready, or merge. - ### Triage -Triage reads the issue discussion and base branch, then asks up to five -questions, writes a bounded specification, or declines. Questions and specs -are GitHub comments with hidden structured markers. +Triage reads the issue discussion and the base branch. The triage agent asks +up to five questions, declines the request, or sends it to the spec-writing +agent. The spec-writing agent produces a bounded specification. Questions and +specifications are GitHub comments with hidden markers. ### Spec-to-PR -Spec-to-PR verifies that the specification has not changed since approval, -implements it in a Sandbox, runs configured checks, permits bounded repair -attempts, and opens a draft pull request on -`factory/issue-`. +Spec-to-PR confirms that the approved specification has not changed. It +implements that specification in a Sandbox, runs the configured verification +commands, allows a bounded number of repairs, and opens a draft pull request +on `factory/issue-`. ### PR Reviewer -PR Reviewer checks out the labeled head commit, runs configured verification, -and posts one COMMENT review covering security, correctness, performance, and -tests. +PR Reviewer checks out the labeled pull request commit, runs the configured +verification commands, and posts one COMMENT review. The review covers +security, correctness, performance, and tests. + +## State -## State ownership +Each system owns the state it already manages: | State | Owner | |---|---| | Task status, timing, input, and result | Render Workflows | -| Issues, labels, specifications, PRs, and reviews | GitHub | -| Current run for an issue or PR, and its Sandbox | App-authored status comment marker | -| User-visible run status | The same status comment, and the working label | +| Issues, labels, specifications, pull requests, and reviews | GitHub | +| Current run and Sandbox for an issue or pull request | App-authored status comment | | Prepared repository filesystem | Render Sandbox snapshot | -| Run and agent outcomes, usage, and cost | JSON lines in the service logs | +| Run and agent outcomes, usage, and cost | JSON lines in Workflows logs | + +Telemetry is not application state. Nothing reads log events to decide what a +task may do. -Telemetry is write-only: nothing reads it back to decide what a task does. +## Coordination -## Idempotency and competing runs +Webhook delivery is not exactly once. GitHub can redeliver events, and people +can apply labels or post comments at the same time. The factory handles this +at three boundaries: -The system handles retries and overlap at three levels: +1. The GitHub delivery ID becomes the Workflows idempotency key. +2. The status comment names the run that holds the claim on an issue or pull + request. +3. Hidden markers identify App-authored specifications, pull requests, and + reviews so writes can update or reuse them. -1. GitHub's delivery ID is the Workflows idempotency key. -2. The status comment is the claim: a claiming task works only while the - comment names its run. -3. App-authored markers let writes update or reuse existing artifacts. +A new claiming run cancels the run named in the status comment, terminates +its recorded Sandbox, and writes its own run ID to the comment. The run whose +write remains visible holds the claim. Other runs stop when they next check +the comment. -A new trigger takes over by cancelling the run the status comment names, -terminating that run's Sandbox, and rewriting the comment to name itself. -Render kills a cancelled task run's process within about a second, so a -replaced run can't keep writing. Two triggers that arrive together both take -over; the run whose status write lands last keeps the claim, and the other -stops at its next checkpoint. Every claiming task checks the claim before -progress updates and durable writes. Publishing also uses a deterministic -branch and refuses to overwrite human commits. +Claiming tasks check ownership before progress updates and durable writes. +Publishing also uses a deterministic branch and refuses to overwrite human +commits. -## Sandbox lifecycle +This design keeps coordination beside the GitHub object it controls. Adding a +database would create a second owner for the same claim. -A base-branch push builds a repository snapshot containing runtime tools, a -base checkout, and installed dependencies. Every claiming task: +## Sandboxes and snapshots + +A base-branch push builds a repository snapshot with the runner, a base +checkout, and installed dependencies. A claiming task then: 1. restores the newest snapshot; -2. verifies its permissions and process hardening; +2. verifies runtime permissions and process hardening; 3. fetches the exact requested ref with a temporary read-only token; -4. removes and verifies removal of that token; -5. reinstalls dependencies if lockfiles or the install command changed; +4. removes the token and verifies its removal; +5. reinstalls dependencies when the lockfiles or install command changed; 6. runs agents and repository commands as an unprivileged user; and 7. closes the Sandbox in `finally`. -A cancelled task run is killed without reaching `finally`, so each claiming -task records its Sandbox in the status comment, and the run or `/factory cancel` -that stops it terminates the Sandbox. The provider-side timeout is the backstop -for anything else that dies before cleanup. +Render terminates a cancelled task without running `finally`. For that case, +each claiming task records its Sandbox ID in the status comment. A replacement +run or `/factory cancel` terminates the recorded Sandbox. The provider timeout +is the final cleanup backstop. + +Snapshots avoid reinstalling dependencies on every run. Restored Sandboxes +still fetch the requested commit and verify their current state; a snapshot is +prepared state, not trusted current state. -## Agent contract +## Agents -An agent definition fixes its prompt, model tier, tools, output schema, turn -limit, and optional thinking level. The complete model loop and every -model-directed tool run inside the Sandbox. +An agent definition fixes: -Agents finish through a typed `submit` tool. The Sandbox validates the -submission against JSON Schema, and the host validates it again with the -authoritative Zod schema. Repositories, refs, issue numbers, pull requests, -and write destinations come from authenticated task input—not model output. +- its prompt; +- model tier and thinking level; +- available tools; +- output schema; +- turn limit; and +- stable ID. -## Publishing boundary +The model loop and every model-directed tool run in the Sandbox beside the +checkout. Agent tools never run on the Workflows host. -The Sandbox has no GitHub write credential. It produces a candidate tree, -which trusted Workflows code validates before publishing: +Every agent finishes through a typed `submit` tool. The runner validates the +submission against JSON Schema. The host validates it again with the agent's +Zod schema. Repositories, refs, issue numbers, pull requests, branches, and +write targets come from authenticated task input, never model output. -1. collect changes through a throwaway Git directory; -2. validate paths, modes, file counts, sizes, and blob hashes; -3. reject workflow files, symlinks, and submodules; -4. ask GitHub to build the same tree independently; -5. require GitHub's tree SHA to match the Sandbox tree SHA; and -6. move the deterministic branch only when ownership is safe. +## Publishing -This makes the model a source of proposed bytes, not authority over GitHub. +The Sandbox has no GitHub write credential. It returns a candidate tree to +the Workflows host, which: + +1. collects changes through a throwaway Git directory; +2. validates paths, modes, file counts, sizes, and blob hashes; +3. rejects workflow files, symlinks, and submodules; +4. asks GitHub to build the same tree; +5. compares GitHub's tree SHA with the Sandbox tree SHA; and +6. moves the deterministic branch only when the factory still owns it. ## Trust boundaries -The factory treats issue text, comments, source files, dependency scripts, -and pull request diffs as untrusted. +Issue text, comments, source files, dependency scripts, and pull request diffs +are untrusted. ### Gateway -The gateway verifies the webhook signature before parsing JSON. It holds no -GitHub credential and runs no model or repository code. +The gateway verifies the webhook signature before parsing the body. It holds +no GitHub credential and runs no model or repository code. ### Workflows host -The Workflows service holds service credentials. Trusted code selects -destinations, enforces run ownership, validates Sandbox output, and performs -GitHub writes. No agent tool runs in this process. +The Workflows service holds GitHub and Render credentials. It selects every +destination, checks claim ownership, validates Sandbox output, and performs +GitHub writes. -### Sandboxes +### Sandbox -Clone and fetch use temporary read-only credentials that are removed before -repository code or agents run. Runtime files are root-owned and read-only. -Agents and repository commands run unprivileged. +Clone and fetch credentials are read-only and temporary. Runtime files are +root-owned and read-only. Agents and repository commands run as an +unprivileged user. -The model key enters the Sandbox for an agent call. It is delivered through a -root-only path and stdin, removed from the environment before tools run, and -protected by a non-dumpable runner. These controls reduce exposure but do not -eliminate it. +The model key enters the Sandbox during an agent call. The runner receives it +through a root-only path and stdin, removes provider keys from tool +environments, and runs under a non-dumpable Node binary. These controls reduce +exposure; they do not make untrusted code safe. ## Security invariants Changes must preserve these properties: -1. Verify webhook signatures before parsing. -2. Check the triggering actor's live permission for every run. -3. Keep GitHub write credentials out of Sandboxes. -4. Run all model-directed tools inside Sandboxes. -5. Reconstruct repositories, refs, branches, and write targets from trusted input. -6. Publish only after GitHub reproduces the validated tree SHA. -7. End automation at a draft PR and COMMENT review. -8. Implement the exact specification that was approved. -9. Put a timeout and `finally` cleanup around every Sandbox, and record a - claiming task's Sandbox in its status comment so whatever stops the run - can terminate it. - -Tests pin these invariants and each built-in agent's tool authority. +1. Verify a webhook signature before parsing its body. +2. Check the triggering actor's current repository permission for every + claiming run. +3. Let only code in the Workflows service write to GitHub. +4. Keep GitHub write credentials out of Sandboxes. +5. Run every model-directed tool inside a Sandbox. +6. Build repositories, refs, branches, and write targets from trusted input. +7. Publish only after GitHub reproduces the validated tree SHA. +8. Stop automation at a draft pull request and a COMMENT review. +9. Implement the specification as it existed when approved. +10. Give every Sandbox a provider timeout and `finally` cleanup. Record a + claiming task's Sandbox so cancellation can terminate it. + +Tests pin these invariants and the tool authority of every built-in agent. ## Residual risks -- Sandboxes use allow-all egress, leaving network and DNS exfiltration paths. +- Sandboxes use allow-all egress. - The model key exists in runner memory during a call. -- Any collaborator with `minimumPermission` can initiate model spend. -- PR Reviewer executes configured verification against the labeled PR code. -- Prompt injection can influence specs, code, and reviews. +- Prompt injection can influence specifications, code, and reviews. +- Anyone with `minimumPermission` can start model spend. +- PR Reviewer runs repository-defined commands from the labeled pull request. - The Render API key may have workspace-wide authority. -- The claim relies on reading the status comment back right after writing it. - Two triggers within a second of each other can both do work before one stops. -- Workflows inputs and results are visible to users with access to the service. -- Telemetry log lines include repository names, issue subjects, and error - messages that can quote repository output, visible to anyone with access to - the service logs or a log stream. -- A cancelled snapshot build leaves its builder Sandbox running until its - one-hour timeout, because builder Sandboxes aren't recorded anywhere. - -Use a dedicated Render workspace and spend-capped model key when stronger -isolation is required. +- Near-simultaneous triggers can both work until one loses the status-comment + claim. +- Workflows inputs and results are visible to people with service access. +- Telemetry can include repository names, issue subjects, and errors that + quote repository output. +- The filesystem of each deployed Render service is ephemeral. Durable state + belongs in GitHub, Workflows, or Sandbox snapshots. +- A cancelled snapshot build can leave its builder Sandbox running until the + one-hour provider timeout because builders are not recorded in a status + comment. + +Use a dedicated Render workspace and a spend-capped model key when stronger +isolation matters. + +Expected work failures produce a useful status-comment result. Unexpected +platform or programming failures fail the Workflows task and require +operator investigation. ## Code map -Start with these files: - -1. [`app/gateway/routes.ts`](../app/gateway/routes.ts) — event routing. -2. [`app/claims/claim.ts`](../app/claims/claim.ts) — the claim lifecycle every claiming task shares. -3. [`app/workflows/claiming-task.ts`](../app/workflows/claiming-task.ts) — how a claiming task is declared. -4. [`app/workflows/triage/index.ts`](../app/workflows/triage/index.ts) — one complete claiming task. -5. [`app/agents/run.ts`](../app/agents/run.ts) — Sandbox agent execution. -6. [`app/sandbox/sandbox.ts`](../app/sandbox/sandbox.ts) — Sandbox lifecycle. -7. [`app/workflows/spec-to-pr/index.ts`](../app/workflows/spec-to-pr/index.ts) — implementation and verification. -8. [`app/changeset/index.ts`](../app/changeset/index.ts) — validated publishing. - -See [Contributing](../CONTRIBUTING.md) for local development and extension -recipes. +Start with: + +1. [`app/gateway/routes.ts`](../app/gateway/routes.ts) — webhook-to-task + routing. +2. [`app/workflows/claiming-task.ts`](../app/workflows/claiming-task.ts) — + claiming task declaration. +3. [`app/claims/claim.ts`](../app/claims/claim.ts) — permission, ownership, + progress, and completion. +4. [`app/workflows/triage/index.ts`](../app/workflows/triage/index.ts) — one + complete claiming task. +5. [`app/agents/run.ts`](../app/agents/run.ts) — host-side agent execution. +6. [`app/runner/main.ts`](../app/runner/main.ts) — Sandbox runner entry point. +7. [`app/sandbox/sandbox.ts`](../app/sandbox/sandbox.ts) — Sandbox lifecycle. +8. [`app/changeset/index.ts`](../app/changeset/index.ts) — validated + publishing. + +Use the [extension runbook](runbooks/extend.md) to add agents, tasks, or +triggers. diff --git a/docs/configuration.md b/docs/configuration.md index 4080815..87dac2e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,9 +1,13 @@ # Configuration -[`factory.config.ts`](../factory.config.ts) is the factory's committed -application configuration. Both deployed services import it. +[`factory.config.ts`](../factory.config.ts) is committed application +configuration. The gateway and Workflows service both import it. -## Complete example +Edit it before the first deployment and whenever you change repositories, +labels, model assignments, permissions, publishing limits, or Sandbox +lifetimes. + +## Example ```typescript export default defineFactoryConfig({ @@ -43,75 +47,191 @@ export default defineFactoryConfig({ ## Repositories -`repos` is keyed by exact `owner/name`. Webhooks from any other repository -are ignored, even when the App is installed there. +`repos` is keyed by exact GitHub `owner/name`. The factory ignores webhooks +from any other repository, even if the App is installed there. -| Field | Required | Purpose | +| Field | Required | Meaning | |---|---|---| | `baseBranch` | Yes | Branch used for Triage, implementation, and snapshots | -| `install` | No | Dependency command run after repository credentials are removed | -| `verify` | Yes | Commands required before publishing and run during PR review | +| `install` | No | Dependency command run after clone credentials are removed | +| `verify` | Yes | Commands required before publishing and during PR review | Use deterministic install commands such as `npm ci`. Snapshot creation fails -if installation changes a tracked file. An empty `verify` list explicitly -disables command-based verification. +if installation changes a tracked file. Use `verify: []` only when you +intentionally want no command-based verification. After adding a repository: -1. commit and deploy `factory.config.ts`; +1. commit and deploy the configuration; 2. install the GitHub App on the repository; 3. run `npm run setup:github`; and -4. apply a factory label, push to the base branch, or - [refresh its snapshot by hand](operations.md#refresh-a-snapshot-by-hand) - to build its first snapshot. +4. push to the base branch, start a factory task, or + [refresh the snapshot directly](#refresh-a-snapshot). -## Behavior and limits +## Labels, permission, and limits -| Setting | Default | Purpose | +| Setting | Default | Meaning | |---|---|---| | `labels.triage` | `factory:triage` | Starts or continues Triage | | `labels.specApproved` | `factory:spec-approved` | Starts Spec-to-PR | | `labels.prReview` | `factory:pr-review` | Starts PR Reviewer | | `commandPrefix` | `/factory` | Prefix for commands such as `/factory cancel` | -| `minimumPermission` | `write` | Required live GitHub permission | +| `minimumPermission` | `write` | Live repository permission required to start work | | `models.medium` | `anthropic/claude-sonnet-5` | Triage and PR Reviewer model | | `models.large` | `anthropic/claude-opus-5` | Spec writer and implementer model | | `limits.repairRounds` | `2` | Repair attempts after failed verification | | `limits.maxChangedFiles` | `200` | Maximum files in a published change | | `limits.maxFileBytes` | `1000000` | Maximum size of one changed file | -| `sandbox.timeoutSeconds` | `14400` | Provider-side Sandbox lifetime | +| `sandbox.timeoutSeconds` | `14400` | Provider-side Sandbox lifetime in seconds | + +`minimumPermission` accepts `triage`, `write`, `maintain`, or `admin`. The +factory checks the triggering actor's current permission at the start of +every claiming task. -`setup:github` also creates `factory:working`, which marks an active run. It -is operational state rather than a configurable trigger label. +`setup:gibthub` also creates `factory:working`. This label records an active +run; it is not a trigger. -After renaming a label, deploy both services and rerun -`npm run setup:github`. Existing labels are not removed automatically. +If you rename a trigger label, deploy both services and rerun +`npm run setup:github`. The command creates required labels but does not +remove old ones. -Keep the Sandbox timeout at least as long as the longest Workflows task. +Keep `sandbox.timeoutSeconds` at least as long as the longest Workflows task. ## Models -Each agent selects `medium` or `large`; configuration maps the tier to a -`provider/model` in Pi's model registry. +Agent definitions select the `medium` or `large` tier. +`factory.config.ts` maps each tier to a `provider/model` supported by +`@earendil-works/pi-ai`, the model client used by the Sandbox runner. -To change Anthropic models, update the mappings and redeploy the Workflows -service. To use another supported provider: +Changing a model mapping requires a Workflows redeploy. To add another +provider: -1. choose its registered `provider/model`; -2. find its key variable in [`app/runner/keys.ts`](../app/runner/keys.ts); -3. add the variable to the Workflows service in [`render.yaml`](../render.yaml); -4. add it to [`.env.example`](../.env.example) and setup requirements; +1. choose a registered `provider/model`; +2. find its key name in [`app/runner/keys.ts`](../app/runner/keys.ts); +3. add the key to `factory-workflows` in [`render.yaml`](../render.yaml); +4. add it to [`.env.example`](../.env.example) and + `SERVICE_ENV` in `scripts/lib/requirements.ts`; 5. set the key in Render; and 6. run `npm run doctor -- --service workflows`. -Use a key dedicated to this factory with a provider-enforced spend limit. -Turn limits and timeouts bound execution, but the factory has no hard dollar -budget. +Use a key dedicated to this deployment with a provider-enforced spend limit. +The factory has no hard dollar budget. + +## What must be redeployed + +| Change | Required action | +|---|---| +| Repository, label, permission, or publishing limit | Deploy both services | +| Model mapping | Deploy `factory-workflows` | +| Prompt or Workflows service code | Deploy `factory-workflows` | +| Gateway routing or validation | Deploy `factory-gateway` | +| Label, App permission, or subscribed event | Run `npm run setup:github`; update the App when prompted | +| `app/runner/` or `sandbox/` | Deploy Workflows, then refresh each repository snapshot | +| Repository `install` command or lockfiles | Refresh the snapshot, or let the next task reinstall | + +## Snapshots + +Each configured repository has a dependency-installed Sandbox snapshot. A +push to its base branch refreshes it. If no snapshot exists, the first task +builds one. + +A failed refresh does not delete the previous usable snapshot. + +### Refresh a snapshot + +Start the `refresh-snapshot` task with the repository, its GitHub App +installation ID, and the base branch's current commit: + +```bash +git ls-remote https://github.com//.git refs/heads/ + +cat > input.json <<'JSON' +[{"repo":"/","installationId":,"sha":""}] +JSON + +render workflows tasks runs start \ + /refresh-snapshot \ + --input-file input.json +``` + +The installation ID is the number at the end of the installation settings +URL: + +```text +https://github.com/settings/installations/ +``` + +Organization installations use the equivalent organization settings URL. + +## Follow and control runs + +While a claiming task runs, the issue or pull request has `factory:working`. +The App-authored status comment shows progress, the run ID, and the final +result. Its hidden marker records the current Sandbox for cancellation. + +Open that run under `factory-workflows` in the Render Dashboard to inspect its +input, logs, and result. + +The service also writes structured events: + +- `run.finished` records the task, state, outcome, and duration. +- `agent.finished` records the agent, model, turns, token usage, reported + cost, and stop reason. + +Search them with the Render CLI: + +```bash +render logs -r --text run.finished +render logs -r --text agent.finished +``` + +To cancel a run, post this as the first line of an issue or pull request +comment: + +```text +/factory cancel +``` + +The commenter needs `minimumPermission`. Cancellation stops the run named in +the status comment and terminates its recorded Sandbox. It also clears a stale +`running` status when the task has already stopped. + +To restart work, remove the trigger label and apply it again. + +## Check a deployment + +`doctor` is read-only, does not print secret values, and exits nonzero when a +required check fails. + +From a local checkout, this checks the services and credentials represented +by local `.env`: + +```bash +npm run doctor +``` + +It cannot inspect values stored only in Render. Check the deployed gateway +from its Render Shell: + +```bash +npm run doctor -- --service gateway +``` + +Workflows services have no Shell. Put the deployed Workflows values and exact +`RENDER_WORKFLOW_SLUG` in local `.env`, then run: + +```bash +npm run doctor -- --service workflows +``` + +The [setup runbook](runbooks/setup.md#phase-9-run-doctor) lists the required +values and common failures. ## Environment variables -Every local variable appears in [`.env.example`](../.env.example). The -Blueprint wires the gateway's reference to the Workflows service. +Local variables are listed in [`.env.example`](../.env.example). The +Blueprint supplies `RENDER_WORKFLOW_SLUG` to the gateway from the Workflows +service. ### Gateway @@ -133,9 +253,8 @@ Blueprint wires the gateway's reference to the Workflows service. | `ANTHROPIC_API_KEY` | Call the default models | | `GITHUB_API_URL` | Optional GitHub API base URL | -`GITHUB_API_URL` is not declared by the Blueprint. GitHub Enterprise Server -users must set it in local `.env` for setup commands and manually on -`factory-workflows` in the Render Dashboard. +`GITHUB_API_URL` is not in the Blueprint. GitHub Enterprise Server users must +set it in local `.env` for setup commands and on `factory-workflows` in the +Render Dashboard. -The gateway must bind to `0.0.0.0:$PORT` when deployed. Never commit `.env` -or provider keys. +Never commit `.env` or provider keys. diff --git a/docs/design-notes.md b/docs/design-notes.md deleted file mode 100644 index 6bb5236..0000000 --- a/docs/design-notes.md +++ /dev/null @@ -1,61 +0,0 @@ -# Design notes - -These are the reusable ideas behind software-factory. They are explanatory, -not required for deployment or operation. - -## Let existing systems own their state - -Render Workflows owns task execution. GitHub owns collaboration artifacts, -including the status comment that says which run currently works on an issue -or pull request. There is no database; telemetry goes to logs. Every -additional source of truth creates reconciliation work. - -## Make approval an explicit transition - -Each label authorizes one bounded task. Triage cannot start implementation, -and implementation cannot start review. Native GitHub actions make these -gates visible and auditable without a separate control plane. - -## Design for overlapping execution - -Webhook redelivery, relabels, and near-simultaneous comments make “exactly -once” execution unrealistic. The factory combines task-start idempotency, a -claim held in the status comment, and artifact markers that make effects -repeatable. - -A new run stops the old one (Render kills a cancelled task run) and terminates -its Sandbox, then checks that it still holds the claim immediately before each -durable effect. - -## Prepare expensive state, verify current state - -Snapshots avoid reinstalling tools and dependencies for every task. A -restored Sandbox still fetches the requested ref, checks its manifest, and -reinstalls when lockfiles or configuration changed. - -## Put tools next to code, away from authority - -The agent loop runs in the Sandbox beside the checkout. The security boundary -is isolated tool placement, an unprivileged user, temporary read-only -credentials, bounded lifetime, and host-side acceptance—not a shell-command -filter. - -## Separate proposal from authority - -Agents return bounded typed values and candidate file trees. Trusted host code -reconstructs destinations, validates content against independent ground -truth, and performs writes. - -## Bound work and classify failures - -Turn limits, call timeouts, repair limits, task timeouts, and -provider-enforced spend caps create practical bounds. Expected work failures -produce useful GitHub messages; platform failures fail the Workflows task for -an operator. - -## State residual risk plainly - -Layered controls reduce authority; they do not make arbitrary code and model -execution perfectly safe. See -[Architecture](architecture.md#residual-risks) for this project's accepted -risks. diff --git a/docs/operations.md b/docs/operations.md deleted file mode 100644 index 1c3bcf7..0000000 --- a/docs/operations.md +++ /dev/null @@ -1,202 +0,0 @@ -# Operations - -Use this guide after completing the [Quickstart](quickstart.md). - -## Follow a run - -While a task is running, its issue or pull request has `factory:working`. The -factory's status comment shows progress, the current run ID, and final links. - -Find that run under `factory-workflows`, in the `software-factory` project in -the Render Dashboard, to inspect -logs, inputs, and its result. - -Every task run ends with a `run.finished` JSON line, and every agent run with -an `agent.finished` line, in the `factory-workflows` logs. Search for them, or -send them to a log stream: - -```bash -render logs -r --text agent.finished -``` - -`run.finished` carries the task, state, outcome, and duration. `agent.finished` -carries the agent, model, turns, token usage, reported cost, and, when an agent -is stopped, why. A run stopped by a newer run or by `/factory cancel` gets its -`run.finished` line from whatever stopped it, because Render kills a cancelled -run before it can write one. Task state remains in Render Workflows; GitHub remains the -source of truth for issues, specifications, pull requests, and reviews. - -## Cancel or restart - -To cancel, post this as the first line of an issue or pull request comment: - -```text -/factory cancel -``` - -The commenter must have `minimumPermission`. Cancellation stops the run the -status comment names and terminates its Sandbox. Render kills a cancelled run -within about a second. If the comment still says "running" for a run that -already stopped, for example one Render killed for exceeding its timeout, -`/factory cancel` marks it cancelled and removes `factory:working`. - -To restart a task, remove its trigger label and apply it again. - -## Check a deployment - -From a configured local checkout, this checks the credentials and services -named by your local `.env`: - -```bash -npm run doctor -``` - -It does not inspect environment variables stored only on Render. To verify -the gateway's actual environment, run `doctor` from its Render Shell: - -```bash -npm run doctor -- --service gateway -``` - -A Workflows service has no Shell, because Render starts a short-lived -instance for each task run. Check it locally instead, with its values and its -deployed `RENDER_WORKFLOW_SLUG` in `.env`, as the -[setup runbook](runbooks/setup.md#phase-9-verify-the-deployed-services) -describes: - -```bash -npm run doctor -- --service workflows -``` - -`doctor` is read-only, does not print secret values, and exits nonzero when a -required check fails. - -## Understand snapshots - -Each configured repository has a dependency-installed Sandbox snapshot. A -push to its base branch refreshes that snapshot. If no snapshot exists, the -first task to run builds one inline. - -Configuration and code changes take effect at different times: - -- `factory.config.ts`, application code, and prompt changes require a deploy. -- `app/runner/` and `sandbox/` changes require a snapshot refresh. -- A changed `install` command triggers reinstallation when a Sandbox opens; - the next base-branch push bakes it into a new snapshot. -- Added repositories and renamed labels require `npm run setup:github`. -- GitHub permission or event changes require updating the App and rerunning - `doctor`. - -Snapshot refresh keeps the previous usable version when a new build fails. - -### Refresh a snapshot by hand - -Start the `refresh-snapshot` task with the repository, its App installation, -and the base branch's head commit: - -```bash -git ls-remote https://github.com//.git refs/heads/ -echo '[{"repo": "/", "installationId": , "sha": ""}]' > input.json -render workflows tasks runs start /refresh-snapshot --input-file input.json -``` - -The installation ID is the number at the end of the App installation's -settings URL (`https://github.com/settings/installations/`, or the -organization's equivalent). - -## Troubleshoot - -### No status comment - -Open the GitHub App's **Advanced** settings and find the delivery under -**Recent Deliveries**. If there is none, confirm the webhook is active and its -URL ends in `/webhooks/github`. - -The gateway's response says what happened. `202` means a task started; look -for it under `factory-workflows`. A `200` with `{"ignored": ""}` names -why nothing started: - -- `unconfigured repo`: the repository isn't in `factory.config.ts`. -- `unhandled label`: the label doesn't match a configured trigger label. -- `issue not in triage`: a comment arrived on an issue without the Triage - label. -- `bot sender`: bots, including the factory itself, can't start runs. -- `pull request not open` or `not base branch`: the event targets something - the factory skips. - -If a task started but the status comment says the run "did not run", the -triggering actor lacks `minimumPermission`. - -### Webhook delivery returns 401 - -`GITHUB_WEBHOOK_SECRET` differs between GitHub and `factory-gateway`. - -### Webhook delivery returns 422 - -The delivery was authenticated but produced an invalid task input. The -gateway logs `authenticated delivery produced an invalid task input` with the -reason; this usually points to a routing bug. - -### Webhook delivery returns 502 - -The gateway could not start the task. Its logs show `could not start task` -with the underlying error. Check `RENDER_API_KEY`, `RENDER_WORKFLOW_SLUG`, and -the Workflows service. - -### Snapshot build fails - -Inspect the `refresh-snapshot` run or the first task run. Common causes are: - -- an invalid install command; -- installation changing tracked files; -- a missing App installation; or -- a Sandbox verification failure. - -### Agent submitted an invalid result - -The status comment says an agent "submitted an invalid result" when it was -rejected three times for not matching its output schema. The `agent.finished` -log line records why, with the last attempt's schema problems and the field -names it sent: - -```bash -render logs -r --text invalid_submission -``` - -A misspelled or extra field appears among the submitted names. Problems that -recur for one issue usually point to the agent's prompt. - -### Status comment stuck on "running" - -The run stopped without finishing, for example after hitting its timeout. -Post `/factory cancel` to clear the status and terminate any Sandbox it left, -or reapply the trigger label to start a fresh run. - -### Agent cannot start - -Check the selected provider key in `factory-workflows`, then run: - -```bash -npm run doctor -- --service workflows -``` - -### Publishing refuses a change - -Inspect task logs for rejected workflow files, unsafe paths, symlinks, -submodules, file-count or size limits, branch ownership, or a tree-SHA -mismatch. - -## Capacity, cost, and risk - -- Sandbox API calls share the workspace's request limit. Expect a few - concurrent tasks rather than dozens. -- The factory has no dollar budget. Use a dedicated, provider-capped model - key. -- Anyone with `minimumPermission` can start model spend. -- PR Reviewer executes the labeled PR's configured verification commands. -- Sandboxes use allow-all egress. -- Local filesystem writes on Render services are ephemeral; durable state - belongs in GitHub, Workflows, or Sandbox snapshots. - -Review [Architecture](architecture.md#residual-risks) before using private -code or untrusted pull requests. diff --git a/docs/quickstart.md b/docs/quickstart.md index 9523455..cdd5f04 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -1,64 +1,64 @@ # Quickstart -The fastest path is to ask a coding agent to follow the -[Setup runbook](runbooks/setup.md). It gives the agent exact commands, -verification criteria, recovery steps, and explicit checkpoints for actions -only you should take. +Use a coding agent for the first deployment. The +[Setup runbook](runbooks/setup.md) gives it the commands, checks, recovery +steps, and points where you must take over. -## What you need +## Before you start + +Prepare: - a fork of this repository; - a Render workspace with Sandboxes access in Oregon; - a Render API key; -- a dedicated, spend-capped Anthropic API key; +- a dedicated Anthropic API key with a provider-enforced spend limit; - admin access to each target GitHub repository; and - the Node.js version in [`.node-version`](../.node-version). -The Blueprint creates paid Render resources in a `software-factory` project: a -Starter web service and a Workflows service. Sandbox and model usage are charged separately. +The Blueprint creates paid resources: a Starter web service and a Workflows +service. Sandbox and model usage are billed separately. -## What the setup agent will do +## Start the setup -The agent will: +Open your fork in a coding agent and use this prompt: -1. inspect the repository and run its checks; -2. configure target repositories in `factory.config.ts`; -3. guide creation and installation of a GitHub App; -4. guide deployment of the Render Blueprint; -5. connect the webhook and create labels; -6. verify both services with `doctor`; and -7. run one approved Triage smoke test. +```text +Set up software-factory by following docs/runbooks/setup.md. +Configure with base branch . +Stop at every human checkpoint. Do not print or expose secrets. +``` -The agent must stop for your confirmation before: +The agent will configure the fork, create a GitHub App, deploy the Blueprint, +connect the webhook, verify the deployment, and guide one Triage smoke test. -- committing or pushing configuration unless already authorized; -- creating the GitHub App; -- installing it on repositories; -- creating paid Render resources; -- entering secrets; -- activating the webhook; or -- creating and labeling a test issue. +You must take over for: -Never paste API keys, private keys, or webhook secrets into chat. Enter them -directly in `.env` or the Render Dashboard. +- approval to commit or push; +- GitHub App creation and installation; +- paid Render resource creation; +- secret entry; +- webhook activation; and +- creation or labeling of the smoke-test issue. -## Start +Enter secrets directly in `.env`, GitHub, or the Render Dashboard. Never put +them in chat, issues, commits, or command output. -Open the fork in your coding agent and ask: +## Definition of done -```text -Set up software-factory by following docs/runbooks/setup.md. -Configure using base branch . -Stop at every human checkpoint and do not expose secrets. -``` +Setup is complete when: -Setup is complete only when `doctor` passes for both services and applying -`factory:triage` produces an App-authored status comment. +- `factory-gateway` and `factory-workflows` are live; +- the GitHub App is installed and its webhook is active; +- gateway `doctor` passes in its Render Shell and Workflows `doctor` passes + locally with the deployed service's values; and +- applying `factory:triage` produces an App-authored status comment and a + useful outcome. -## Continue +## Next steps -- [Setup runbook](runbooks/setup.md): complete agent execution protocol. -- [Configuration](configuration.md): repositories, models, labels, and limits. -- [Operations](operations.md): logs, cancellation, snapshots, and troubleshooting. -- [Architecture](architecture.md): state ownership and trust boundaries. -- [Contributing](../CONTRIBUTING.md): local development and implementation changes. +- [Configuration](configuration.md) — change settings, follow or cancel runs, + check a deployment, and manage snapshots. +- [Architecture](architecture.md) — understand state, coordination, and + security. +- [Extension runbook](runbooks/extend.md) — add agents, Workflows tasks, or + GitHub triggers. diff --git a/docs/runbooks/extend.md b/docs/runbooks/extend.md new file mode 100644 index 0000000..4169308 --- /dev/null +++ b/docs/runbooks/extend.md @@ -0,0 +1,261 @@ +# Extension runbook + +Use this runbook when adding an agent, a Render Workflows task, or a GitHub +trigger. It is written for coding agents working in a fork of +software-factory. + +Read [Architecture](../architecture.md) first. Do not change the authority +boundary as part of an extension: agents propose typed values or file changes; +code in the Workflows service selects targets, validates results, and writes +to GitHub. + +## Before editing + +Inspect: + +- `factory.config.ts`; +- `app/agents/`; +- `app/workflows/`; +- `app/schemas/`; +- `app/gateway/routes.ts`; +- `app/config/index.ts`; +- `scripts/lib/requirements.ts`; and +- the closest existing test. + +Run `npm run check` before changing code. Record any existing failure and +preserve unrelated work. + +Decide which change you need: + +- **Agent only:** a new prompt, tool set, and typed result used by an existing + task. +- **Claiming task:** work attached to one issue or pull request that must stop + when another run takes over. +- **Plain task:** maintenance work without an issue or pull request claim, + such as snapshot refresh. +- **GitHub trigger:** a new authenticated event, label, or command that starts + a task. + +Do not add a task when an existing task can call the new agent and own the +result. + +## Add an agent + +### 1. Define the output + +Add or reuse a schema in `app/schemas/`. Build every output object with +`z.strictObject`: + +```typescript +export const Finding = z.strictObject({ + summary: z.string().min(1).max(4000), + paths: z.array(z.string().min(1)).max(100), +}); + +export type Finding = z.infer; +``` + +Bound strings and arrays. The model sees this schema through the `submit` +tool, and the Workflows host validates the result again. + +Export the schema from `app/schemas/index.ts`. + +### 2. Add the prompt and definition + +Create: + +```text +app/agents// + index.ts + prompt.md +``` + +Define the agent: + +```typescript +export const finder = defineAgent({ + id: "finder", + prompt: "finder/prompt.md", + model: "medium", + tools: ["read", "ls", "bash"], + output: Finding, + maxTurns: 40, +}); +``` + +The definition fixes the agent's authority: + +- `id` must be unique and stable; +- `prompt` is relative to `app/agents/`; +- `model` is `medium` or `large`; +- `tools` may include `read`, `ls`, `bash`, `edit`, and `write`; +- `output` is a bounded strict Zod object; +- `maxTurns` is a positive model-call limit; and +- `thinking` is optional. + +Grant `edit` or `write` only when the agent must change the checkout. The +read-only shell policy in `app/runner/tools.ts` guides read-only agents; it is +not a security boundary. + +The prompt should explain the job, evidence requirements, limits, and when to +call `submit`. Put repository-, issue-, and run-specific data in the message +passed to `runAgent`, not in the prompt. + +Export the agent from `app/agents/index.ts`. + +### 3. Call the agent + +Call the agent inside a Sandbox owned by the task: + +```typescript +const result = await runAgent(finder, message, claim.agentOptions(sandbox)); +``` + +Use the typed `result.output`. Do not parse assistant text or accept a target +chosen by the model. + +### 4. Test the authority + +Add the agent to `tests/agents-catalog.test.ts`. Update the expected writer +list only if write tools are intentional. + +Add task-level tests for the user-visible result and meaningful failures. Use +fakes; tests must not call GitHub, Render, or a model provider. + +## Add a claiming task + +Use a claiming task for work on one issue or pull request. The status comment +records the run that owns the subject and the Sandbox that cancellation must +terminate. + +### 1. Define the task contract + +1. Add a name to `TASKS` in `app/config/index.ts`. +2. Add a bounded input schema to `app/schemas/tasks.ts`. +3. Export the schema from `app/schemas/index.ts`. + +Task input must carry trusted routing data from the authenticated webhook: +repository, installation, actor, subject number, and any commit SHA the task +must pin. Do not accept these values from agent output. + +### 2. Declare the task + +Create `app/workflows//index.ts`: + +```typescript +export const runExample = claimingTask( + { + name: TASKS.example, + title: "Example", + input: ExampleInput, + subject: (input) => ({ kind: "issue", number: input.issue }), + timeoutSeconds: 3600, + }, + async (claim, input, deps): Promise => { + // Code in the Workflows service owns all external writes. + return { outcome: "completed", summary: "completed the example task." }; + }, +); +``` + +`claimingTask()` validates input, resolves the configured repository, checks +the actor's permission, acquires the status-comment claim, and records the +final outcome. + +Claiming tasks do not retry automatically. A retry can repeat model spend or +durable writes; a person starts a fresh run instead. + +### 3. Use the claim correctly + +- Call `claim.progress()` before a major step. +- Open Sandboxes only through `withSandbox`. +- Call `claim.recordSandbox(sandbox)` immediately after opening one. +- Pass `claim.agentOptions(sandbox)` to `runAgent`. +- Call `claim.checkpoint()` before a durable effect that is not already + guarded by the claim helper. +- Throw `ExpectedFailure` only for an expected message safe to show on + GitHub. +- Return a short snake_case `outcome`, user-facing `summary`, and relevant + links. + +Only code in the Workflows service may call GitHub write methods. Format model +text through `app/format/` before publishing it. + +### 4. Register the task + +Import the task module from `app/workflows/index.ts`. Render registers tasks +when the module loads. + +Keep task inputs and results small and JSON-serializable. + +## Add a plain Workflows task + +Use the Render SDK's `task()` directly only when the work has no issue or pull +request claim. `app/workflows/refresh-snapshot.ts` is the reference. + +A plain task must still: + +- validate raw input at entry with Zod; +- resolve repositories through trusted configuration; +- set an explicit timeout and intentional retry policy; +- bound external work; +- return a small JSON-serializable result; and +- clean up every Sandbox it creates. + +Do not use a plain task to bypass permission checks or status-comment +ownership for issue or pull request work. + +## Add a GitHub trigger + +The gateway route is a pure mapping from an authenticated delivery to a +validated task input. + +1. Add the task name and input schema first. +2. Update `route()` in `app/gateway/routes.ts`. +3. Derive repository, installation, actor, issue or pull request, and commit + from the authenticated payload. +4. Use the GitHub delivery ID as the task idempotency key. For content-addressed + maintenance work, use a deterministic key such as `repo@sha`. +5. Update `REQUIRED_EVENTS`, `REQUIRED_PERMISSIONS`, or `LABELS` in + `scripts/lib/requirements.ts`. +6. Add route tests for the start case, ignored cases, and invalid input. +7. Rerun `npm run setup:github` after deployment. + +Never put GitHub credentials or model calls in `app/gateway/`. The gateway +authenticates and routes; it does not perform the task. + +Every GitHub write must create, update, or reuse an App-authored artifact +marker. Workflows idempotency alone expires and cannot protect a permanent +GitHub effect. + +## Verify + +Run: + +```bash +npm run check +``` + +Use the additional checks in +[Contributing](../../CONTRIBUTING.md#verify-changes) when relevant. + +Before handing off, confirm: + +- all external and cross-process inputs are validated; +- the new agent has the minimum tools it needs; +- model output cannot select an external write target; +- the gateway still imports no GitHub auth, Sandbox, agent, or workflow code; +- `app/runner/` still imports nothing from the rest of `app/`; +- claiming tasks record their Sandbox before running an agent; +- tests cover the user-visible success and important failures; and +- documentation and setup requirements changed when the App contract changed. + +## Deploy the extension + +- Prompt, agent, schema, or Workflows changes require a Workflows deploy. +- Gateway route changes require a gateway deploy. +- App permission, event, or label changes require `npm run setup:github`. +- Changes under `app/runner/` or `sandbox/` also require a snapshot refresh for + every configured repository. + +State the required deploy and refresh actions in the handoff. diff --git a/docs/runbooks/setup.md b/docs/runbooks/setup.md index e01145e..2fa38b3 100644 --- a/docs/runbooks/setup.md +++ b/docs/runbooks/setup.md @@ -1,71 +1,54 @@ # Setup runbook -Use this runbook when helping someone deploy software-factory for the first -time. Follow it in order. Do not claim setup is complete until `doctor` -passes for both services and a labeled issue produces a factory status comment. +Use this runbook to deploy software-factory from a fork and connect its first +repository. Follow the phases in order. Stop at every **Human checkpoint**. -## Goal +Setup is complete only after gateway `doctor` passes in its Render Shell, +Workflows `doctor` passes locally with the deployed values, and a Triage smoke +test reaches an App-authored status comment. -Deploy software-factory from the user's fork, connect it to one GitHub -repository, and complete one Triage run. +## Inputs -## Success criteria - -Setup is complete only when: - -- `factory-gateway` and `factory-workflows` are live; -- the GitHub App is installed on every configured repository; -- the App webhook is active and points to the gateway; -- the factory labels exist; -- `doctor` passes for the gateway (in its Render Shell) and for the Workflows - service (run locally with that service's values); and -- applying `factory:triage` produces an App-authored status comment. - -## Required inputs - -Collect or infer these before editing: +Collect: - fork owner and repository; - target repository in `owner/name` form; - target base branch; - deterministic dependency install command, if needed; - verification commands; -- GitHub App owner: the user or an organization; +- GitHub App owner: a user or organization; - Render workspace with Sandboxes access in Oregon; and -- whether the default Anthropic models are acceptable. +- confirmation that the default Anthropic models are acceptable. -The user must provide these secrets directly to local environment files or -Render—not in chat, issues, commits, or command output: +The user must enter these secrets directly in local files, GitHub, or Render: - Render API key; -- model provider API key; and -- generated GitHub App private key and webhook secret. +- Anthropic API key; +- GitHub App private key; and +- webhook secret. + +Never ask the user to paste a secret into chat. Never print `.env`. -## Non-negotiable constraints +## Rules - Deploy from the user's fork, not `render-examples/software-factory`. -- Use the Node.js version pinned in `.node-version`. -- Never commit `.env` or print secret values. -- Do not weaken GitHub App permissions or security invariants to make setup - pass. -- Do not change existing GitHub App permissions, Render resources, or paid - plans without explaining the effect and receiving approval. -- Do not create a test issue or apply a label without the user's approval. -- Stop at every **Human checkpoint** until the user confirms completion. +- Use the Node.js version in `.node-version`. - Preserve unrelated working-tree changes. +- Do not guess repository install or verification commands. +- Do not weaken App permissions or security checks to make setup pass. +- Ask before committing, pushing, creating paid resources, changing external + settings, creating issues, or applying labels. -## Phase 1: Inspect and preflight +## Phase 1: Preflight -### Agent action - -Confirm the current directory is the software-factory root and inspect: +Confirm that the current directory is the software-factory root. Inspect: - `.node-version`; - `package.json`; - `factory.config.ts`; - `render.yaml`; - `.env.example`; and -- the current Git status. +- Git status. Run: @@ -75,27 +58,13 @@ npm ci npm run check ``` -### Pass condition - -- Node satisfies the pinned project version. -- Dependencies install successfully. -- Lint, type checking, and tests pass. -- Existing unrelated changes have been identified and left untouched. - -### Recovery - -- Wrong Node version: install or select the version in `.node-version`, then - rerun the phase. -- `npm ci` failure: report the failing command and relevant error; do not - replace it with a lockfile-changing install. -- Existing test failure: distinguish it from changes made during setup before - continuing. - -## Phase 2: Configure one repository +Continue only when the pinned Node version is active, dependencies install, +checks pass, and unrelated changes are accounted for. Do not replace a failed +`npm ci` with a lockfile-changing install. -### Agent action +## Phase 2: Configure repositories -Edit `factory.config.ts` so `repos` contains the intended targets: +Replace the example `repos` entry in `factory.config.ts`: ```typescript repos: { @@ -107,16 +76,17 @@ repos: { }, ``` -Rules: +Requirements: -- The key must be the exact GitHub `owner/name`. -- `baseBranch` must already exist. -- `install` must be deterministic and must not change tracked files. -- `verify` must contain the commands required before publishing. Use an empty - list only when skipping command-based verification is intentional. +- use the exact GitHub `owner/name`; +- name an existing base branch; +- use an install command that is deterministic and leaves tracked files + unchanged; and +- list every command required before publishing. -Keep the default labels, permission, limits, and Sandbox timeout unless the -user requested a change. +Use `verify: []` only when the user explicitly wants no command-based +verification. Keep the default labels, permission, limits, and Sandbox timeout +unless the user requested changes. Run: @@ -125,46 +95,28 @@ npm run typecheck git diff --check ``` -### Pass condition +Use [Configuration](../configuration.md) for field semantics. -- Configuration parses and type-checks. -- The diff contains only the intended repository and explicitly requested - behavior changes. +## Phase 3: Publish the configured fork -### Recovery +Confirm that the Git remote points to the user's fork. Show the configuration +diff and check results. -Use [Configuration](../configuration.md) for field semantics. If install or -verification commands are unknown, stop and ask the user; do not guess -project-specific commands when the target repository does not establish -them. +### Human checkpoint: commit and push -## Phase 3: Save the fork +Ask the user to confirm: -### Agent action +- the destination fork and branch; +- the configured repositories and commands; and +- permission to commit and push, if not already granted. -Confirm the repository remote points to the user's fork. Present the -configuration diff and verification result. - -Commit and push only when the user has asked the agent to do so. The Render -Blueprint must be able to read the committed configuration from the fork. - -### Human checkpoint: fork ownership and push - -The user must confirm: - -- the destination fork is correct; -- the configuration is safe to deploy; and -- the configured commit is available on the fork's deployment branch. - -### Pass condition - -The fork contains the intended `factory.config.ts` and `render.yaml`. +Do not continue until the deployment branch on the fork contains the intended +`factory.config.ts` and `render.yaml`. The Blueprint reads these files from +that branch. ## Phase 4: Create the GitHub App -### Agent action - -Install local dependencies if needed, then start the manifest flow: +Start the GitHub App manifest flow: ```bash npm run setup:app -- --write-env @@ -176,17 +128,12 @@ For an organization-owned App: npm run setup:app -- --org --write-env ``` -The command starts a local callback server and prints a browser URL. Keep it -running while the user completes the checkpoint. +Keep the command running while the user completes the browser flow. ### Human checkpoint: create the App -Ask the user to: - -1. open the printed local URL; -2. review the App name, repositories, permissions, and events; -3. select **Create on GitHub**; and -4. return when the command reports success. +Ask the user to open the printed URL, review the manifest, and select +**Create on GitHub**. The manifest requests: @@ -194,17 +141,15 @@ The manifest requests: - Metadata: read; and - Issues, Issue comments, Pull requests, and Pushes events. -It does not request Actions, Workflows, or Administration permission. +It does not request GitHub Actions, GitHub Workflows, or Administration +permission. -### Agent verification +After the callback completes, confirm that the command reports the App slug +and ID, prints an installation URL, and writes these values to `.env`: -Confirm the command: - -- reports the App slug and ID; -- prints an installation URL; -- writes `GITHUB_APP_ID`, `GITHUB_APP_PRIVATE_KEY`, and - `GITHUB_WEBHOOK_SECRET` to `.env`; and -- exits successfully. +- `GITHUB_APP_ID`; +- `GITHUB_APP_PRIVATE_KEY`; and +- `GITHUB_WEBHOOK_SECRET`. Verify that Git ignores the file: @@ -212,232 +157,237 @@ Verify that Git ignores the file: git check-ignore .env ``` -Do not display `.env`. +## Phase 5: Install the App -### Pass condition +### Human checkpoint: repository access -The GitHub App exists, `.env` is ignored, and the three generated values are -available locally without being exposed. +Ask the user to open the installation URL and install the App on every +repository in `factory.config.ts`. If an organization requires approval, wait +for an owner to approve the installation. -## Phase 5: Install the GitHub App +Continue only after the user confirms access to every configured repository. -### Human checkpoint: repository installation +## Phase 6: Deploy the Blueprint -Ask the user to open the installation URL printed by `setup:app` and install -the App on every repository in `factory.config.ts`. +[`render.yaml`](../../render.yaml) creates two paid resources in the +`production` environment of a `software-factory` project: -When organization approval is required, stop until an organization owner has -approved the installation. +- `factory-gateway`, a Starter web service in Oregon; and +- `factory-workflows`, a Workflows service in Oregon. -### Pass condition +### Human checkpoint: resources and secrets -The user confirms installation on every exact configured repository. +Explain the resources and expected charges. Ask the user to create the +Blueprint from the configured fork and branch. -## Phase 6: Deploy the Render Blueprint +Enter these values for `factory-gateway`: -### Agent action +- `GITHUB_WEBHOOK_SECRET`; +- `RENDER_API_KEY`. -Explain that [`render.yaml`](../../render.yaml) creates paid resources in -Oregon, in the `production` environment of a `software-factory` project: +Enter these values for `factory-workflows`: -- `factory-gateway`, a Starter web service; and -- `factory-workflows`, a Workflows service. +- `RENDER_API_KEY`; +- `RENDER_WORKSPACE_ID`; +- `GITHUB_APP_ID`; +- `GITHUB_APP_PRIVATE_KEY`; +- `ANTHROPIC_API_KEY`. -The user should create the Blueprint from the configured fork and deployment -branch. +The workspace ID starts with `tea-`. Copy GitHub values from local `.env` +without showing them in chat or command output. -### Human checkpoint: paid resources and secrets +Continue when both services have deployed and the gateway is live. -Before creation, ask the user to confirm the resources and expected charges. -Then ask them to enter every `sync: false` value in the Render Dashboard. +## Phase 7: Configure GitHub -For `factory-gateway`: +Ask for the public gateway URL. It is not a secret. Run: -- `GITHUB_WEBHOOK_SECRET` -- `RENDER_API_KEY` +```bash +npm run setup:github -- \ + --gateway-url https://.onrender.com +``` -For `factory-workflows`: +The command: -- `RENDER_API_KEY` -- `RENDER_WORKSPACE_ID` -- `GITHUB_APP_ID` -- `GITHUB_APP_PRIVATE_KEY` -- `ANTHROPIC_API_KEY` +- sets the App webhook URL to + `https://.onrender.com/webhooks/github`; +- preserves the generated webhook secret; and +- creates `factory:triage`, `factory:spec-approved`, + `factory:pr-review`, and `factory:working`. -The user should copy GitHub values from the local `.env` without pasting them -into chat. The Render workspace ID starts with `tea-`. +It is safe to rerun. Continue only when it reports no failed repository or +webhook operation. -### Pass condition +## Phase 8: Activate the webhook -Both services are created, the gateway reports live, and the Workflows -service has deployed successfully. +`setup:github` cannot activate an App webhook through GitHub's API. -### Recovery +### Human checkpoint: webhook activation -- Blank `sync: false` field: set the missing value and redeploy. -- Build failure: inspect the failing service's build log before changing - configuration. -- Port or health failure: confirm the gateway binds to `0.0.0.0:$PORT` and - serves `/ready`. +Ask the user to open the App's **Webhook** settings, confirm that the URL ends +in `/webhooks/github`, enable **Active**, and save without changing the secret. -Use [Operations](../operations.md#troubleshoot) for service-specific failures. +## Phase 9: Run `doctor` -## Phase 7: Connect the webhook and create labels +`doctor` checks the environment in which it runs. It is read-only and does +not print secret values. -### Agent action +### Gateway -Ask the user for the public gateway URL; this URL is not a secret. Run: +Run this in the `factory-gateway` Render Shell: ```bash -npm run setup:github -- \ - --gateway-url https://.onrender.com +npm run doctor -- --service gateway ``` -The command reads GitHub credentials from `.env`. It is idempotent and should: - -- set the App webhook URL to - `https://.onrender.com/webhooks/github`; -- preserve the generated webhook secret; and -- create `factory:triage`, `factory:spec-approved`, - `factory:pr-review`, and `factory:working` in each configured repository. +This checks the gateway's deployed environment. -### Pass condition +### Workflows -The command exits successfully and reports no failed repository or webhook -operation. +Workflows services have no Shell. Run `doctor` locally with the values used by +`factory-workflows`. -### Recovery +Local `.env` already contains the GitHub App values. Ask the user to add: -- Missing credentials: confirm `setup:app --write-env` completed and `.env` - exists without printing it. -- Repository failure: confirm exact `owner/name`, App installation, and the - installing user's repository administration access. -- Wrong gateway URL: use the public HTTPS URL for `factory-gateway`, without - adding `/webhooks/github`; the command adds the path. - -## Phase 8: Activate the webhook +- `RENDER_API_KEY`; +- `RENDER_WORKSPACE_ID`; +- `ANTHROPIC_API_KEY`; and +- `RENDER_WORKFLOW_SLUG`. -GitHub's API does not allow `setup:github` to activate an App webhook. +Use the deployed workflow slug, including any suffix Render added. Copy the +exact value from `factory-gateway`'s deployed `RENDER_WORKFLOW_SLUG` into +local `.env`. -### Human checkpoint: webhook activation +Run: -Ask the user to: +```bash +npm run doctor -- --service workflows +``` -1. open the GitHub App settings; -2. open **Webhook**; -3. verify the URL ends in `/webhooks/github`; -4. enable **Active**; and -5. save without changing the URL or secret. +The warning `no webhook deliveries yet, so the hook may be inactive` is +expected before the smoke test. Do not continue while either command has a +failed check. -### Pass condition +## Phase 10: Smoke-test Triage -The user confirms the webhook is active. +### Human checkpoint: issue and label -## Phase 9: Verify the deployed services +Ask before creating an issue or changing labels. Use a disposable issue in a +configured test repository: -`doctor` checks the environment it runs in. It is read-only and does not -print secret values. +```text +Title: software-factory setup smoke test -### Human checkpoint or agent action with Render access: the gateway +Confirm the repository can be inspected and return a minimal specification. +Do not request a code change. +``` -Run `doctor` in the `factory-gateway` Render Shell, so it reads the deployed -environment: +After approval, create and label it in GitHub or with `gh`: ```bash -npm run doctor -- --service gateway +gh issue create \ + --repo \ + --title "software-factory setup smoke test" \ + --body "Confirm the repository can be inspected and return a minimal specification. Do not request a code change." + +gh issue edit \ + --repo \ + --add-label factory:triage ``` -### Agent action: the Workflows service +The first run may spend several minutes building the repository snapshot. -A Workflows service has no Shell: Render starts a short-lived instance for -each task run. Run `doctor` locally with the values `factory-workflows` holds. -`.env` already has `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY` from Phase 4. -Ask the user to add these to `.env` without pasting them into chat: +The smoke test passes when: -- `RENDER_API_KEY`, `RENDER_WORKSPACE_ID`, and `ANTHROPIC_API_KEY`, matching - the service; and -- `RENDER_WORKFLOW_SLUG`, the deployed workflow's slug. Render may add a - suffix to the service name (such as `factory-workflows-a1b2`); the - gateway's `RENDER_WORKFLOW_SLUG` shows the exact value. +- the issue receives an App-authored status comment; +- `factory:working` appears while the task runs; +- Triage ends with questions, a specification, or an intentional decline; + and +- the status comment shows the final outcome or a useful failure. -Then run: +Optionally confirm that the Workflows logs contain `run.finished`: ```bash -npm run doctor -- --service workflows +render logs -r --text run.finished ``` -### Expected warning +## Troubleshooting setup -Until GitHub has delivered a webhook, `doctor` warns "no webhook deliveries -yet, so the hook may be inactive". This is expected before the smoke test and -does not block setup. If it persists after Phase 10, recheck Phase 8. +### A service does not deploy -### Pass condition +- For a missing `sync: false` value, set it and redeploy. +- For a build failure, inspect the failing service's build log before changing + configuration. +- For a gateway health failure, confirm that it binds to `0.0.0.0:$PORT` and + serves `/ready`. -Both commands exit successfully with every required check passing. Warnings -do not block. +### `doctor` fails -### Recovery +- Gateway: check the webhook secret, Render API key, and workflow slug. +- Workflows: check the workspace ID, Render API key, App credentials, + provider key, and registered tasks. +- `No tasks found`: use the deployed workflow slug, not an assumed service + name. +- App or label failure: fix App access and rerun `npm run setup:github`. -- Gateway failure: check webhook secret, Render API key, and workflow slug. -- Workflows failure: check Render workspace ID, Render API key, GitHub App - credentials, provider key, and registered tasks. -- "No tasks found" for the workflow: `RENDER_WORKFLOW_SLUG` in `.env` doesn't - match the deployed slug. -- App failure: compare required permissions and events with Phase 4. -- Label or installation failure: rerun `npm run setup:github` after fixing - App access. +### No status comment appears -Do not proceed to the smoke test while either service fails `doctor`. +Open the App's **Advanced** settings and inspect **Recent Deliveries**. -## Phase 10: Smoke-test Triage +- No delivery: activate the webhook and verify its URL. +- `401`: GitHub and `factory-gateway` have different webhook secrets. +- `422`: inspect the gateway log entry + `authenticated delivery produced an invalid task input`. +- `502`: inspect `could not start task`; then check `RENDER_API_KEY`, + `RENDER_WORKFLOW_SLUG`, and the Workflows deployment. +- `200` with `{"ignored":""}`: the event did not start a task. Common + reasons are `unconfigured repo`, `unhandled label`, `issue not in triage`, + `bot sender`, `pull request not open`, and `not base branch`. +- `202`: the task started; inspect it under `factory-workflows`. -### Human checkpoint: external write +If a task starts but the comment says it did not run, the triggering actor +lacks `minimumPermission`. -Ask permission before creating an issue or modifying labels. Use a small, -disposable issue in a configured test repository. +### A snapshot build fails -Suggested issue: +Inspect the `refresh-snapshot` run or the first task run. Check the install +command, tracked-file changes after install, App installation, and Sandbox +verification. -```text -Title: software-factory setup smoke test +### An agent submits an invalid result -Confirm the repository can be inspected and return a minimal specification. -Do not request a code change. +The runner accepts three attempts. Search Workflows logs for the last schema +errors and submitted field names: + +```bash +render logs -r --text invalid_submission ``` -Create the issue and apply `factory:triage`. This can be done in GitHub or, -when the user has approved CLI writes: +Repeated errors for one agent usually require a prompt or schema change. -```bash -gh issue create \ - --repo \ - --title "software-factory setup smoke test" \ - --body "Confirm the repository can be inspected and return a minimal specification. Do not request a code change." +### An agent cannot start -gh issue edit \ - --repo \ - --add-label factory:triage +Check the selected provider key on `factory-workflows`, then run: + +```bash +npm run doctor -- --service workflows ``` -The first run may take several minutes while it builds the repository -snapshot. +### Publishing refuses a change -### Pass condition +Inspect the task logs. Publishing rejects workflow files, unsafe paths, +symlinks, submodules, file-count and file-size limits, branches with human +commits, and candidate trees GitHub cannot reproduce by SHA. -- The issue receives an App-authored status comment. -- `factory:working` appears while the task is running. -- Triage ends with questions, a specification, or an intentional decline. -- The status comment includes the final outcome or a useful failure. -- Optional: the `factory-workflows` logs contain a `run.finished` JSON line - for the run (`render logs -r --text run.finished`). +### The status comment remains on `running` -If the status comment never appears, use -[Operations](../operations.md#no-status-comment). +Post `/factory cancel` as the first line of a comment. This clears stale +status and terminates the recorded Sandbox. Reapplying the trigger label +starts a new run. -## Phase 11: Handoff +## Handoff -Report setup using this format without including secrets: +Report: ```text software-factory setup: complete @@ -454,17 +404,8 @@ Local uncommitted changes: Follow-up actions: ``` -If any success criterion is missing, report setup as **incomplete** and name -the exact checkpoint, evidence, and next action. - -## Ready to extend - -After setup: +If any required check is missing, report setup as **incomplete**. Name the +failed phase, evidence, and next action. -- add or change repositories, labels, models, and limits through - [Configuration](../configuration.md); -- diagnose live behavior through [Operations](../operations.md); -- read [Architecture](../architecture.md) before changing authority, - credentials, coordination, or publishing; and -- use [Contributing](../../CONTRIBUTING.md) to add agents, tasks, and GitHub - triggers. +For later changes, use [Configuration](../configuration.md) or the +[extension runbook](extend.md).