Add durable coroutine-style yield and return to native work queues

Goal

Implement durable coroutine-style Yield (new state checkpoint, intermediate output) in native work queues. A Work is the persistent coroutine; a Claim is one activation. Each successful yield saves distinct resume state and ordered downstream messages via safe-outputs. without terminating Work:

  "name": "work_queue_claim_finish",
  "arguments": {
    "claim_handle": "h1",
    "outcome": "yielded"
  }
}

A terminal return is a separate claim finish.

  "name": "work_queue_claim_finish",
  "arguments": {
    "claim_handle": "h1",
    "outcome": "completed"
  }
}

Files, artifacts, stdout, and final chat must not become alternative return mechanisms. Any
arbitrary-value return capability belongs in safe-outputs.

Yield/resume contract

Snapshot: emit progress, partial results, cursor, next action, etc. through a claim-scoped memory safe-output.
Yield: finish the current Claim with outcome: "yielded", replaces keep_open: true. The Claim completes; the Work remains open.
Verify: trusted processing publishes and verifies the snapshot. Pending or failed delivery blocks another Claim.
Resume: load the last verified snapshot belonging to that same Work, then continue under the new Claim’s authority.

Conceptually:

safeoutputs.persist_work_queue_memory({
 claim_handle,
 memory: state
});

work_queue_claim_finish({
 claim_handle,
 outcome: "yielded", // new variant, it replaces keep_open: true
});

State is logically Work-scoped; each snapshot write is Claim-authorized. It remains data—not instructions, credentials, or a replacement for the immutable Work definition. This is logical resumption, not automatic restoration of an LLM session or running process.

What exists today

Persistence exists: tools.work-queue.memory (docs/src/content/docs/patterns/linter-factory.md:269) generates a schema-validated safe-output tool backed by protected Git-tree publication.
Retaining Work exists: keep_open: true (docs/src/content/docs/reference/work-queue.md:445) permits another Claim after verified delivery.
Generic automatic same-Work restoration is missing: result_refs (actions/setup/js/work_queue_graph.cjs:275) currently contains DAG predecessor results, not earlier Claims’ results for the same Work. The ESLint worker has an explicit, workflow-specific history loader (.github/workflows/eslint-refiner.md:40).

So the persistence mechanism fits already. Seamless resumption needs an explicit restore path that selects the same Work’s latest ledger-verified snapshot and supplies it to the next Claim—not merely whichever memory branch looks newest.

how much do you need to store in "memory"? or is it better to rely on push_to_pull_request_branch to have the agent serialize stuff

The size of memories depends on the workflow. We might want to store an agent's context between claims, for example, or we can just store a few sentences like "tried X, failed; ...". What are the tradeoffs with push_to_pull_request_branch?

Durable queue coroutines

Problem and approach

Implement issue #69585, incorporating the user's scope refinements: a Work is a persistent logical coroutine, and each Claim is one activation. A successful yield finishes the Claim, publishes its declared safe-outputs and any optional inline result, and leaves the Work eligible for another fair Claim only after delivery is independently verified. A successful terminal return uses outcome: "completed" and satisfies dependent Work only after its own verified delivery.

When a worker chooses inline result, persist it directly in the queue's existing Git-backed work-queue.jsonl through a claim-scoped safe-output, with a hard 16 KiB canonical-JSON result ceiling. Workers may instead keep state entirely on an authorized working branch. Freeze the latest verified yielded Result and the last verified inline-result reference, each with its own provenance, into the next accepted Claim assignment. Reuse queue publication, replay, readback, and compaction; do not introduce repo-memory or dedicated result branches.

Naming

Use the user-requested name result: optional structured data attached to a Claim and published with its verified Result. The name does not imply a memory service, process snapshot, or mandatory continuation state, and applies to terminal Results as well as yielded Results.

Use tools.work-queue.result.schema for configuration, set_work_queue_result({claim_handle, result}) for staging, ClaimResult.result for the claim-scoped ledger operation, and resume_ref.result for its verified resumable reference. This distinguishes the output from immutable Work.payload input. Avoid checkpoint as the public field name because Checkpoint already names ledger compaction. Existing source paths containing memory below identify current implementation files, not retained public API names.

The optional lowercase result data is distinct from the existing uppercase Result settlement operation. Calling the tool or publishing ClaimResult stages/persists data; neither proves delivery or replaces Result. Successful delivery still requires verified Result settlement even when the optional data is absent.

Confirmed decisions

Inline result is optional for every outcome, including yielded. This supersedes the earlier mandatory-checkpoint decision. A worker may keep all its state on a working branch and yield without calling a result tool or declaring result configuration.
If emitted, inline result is a schema-valid object of at most 16 KiB (16384 bytes) of canonical UTF-8 JSON, stored in the queue log. Emit at most one result value per Claim.
Declare the result schema in workflow frontmatter under tools.work-queue.result.schema. The MCP server must validate calls immediately and return actionable validation errors to the agent before staging anything; do not defer feedback until workflow conclusion.
Larger results belong on the existing authorized working/PR branch. Verified branch-output receipts retain immutable commit identities; optional inline result data can include relevant paths or extra references. Branch-only resumption must not require a placeholder result object.
completed is terminal without inline result data; cancelled also requires none. Do not impose any automatic inline-result minimum for any finish outcome.
All of that Claim's declared safe-outputs must be verified before the Work can receive another Claim.
Omitting inline result preserves the last verified inline result with its original source Claim/Result provenance. Output references still advance to the latest verified yield; these sources need not be the same Claim.
Replace keep_open entirely. No deprecated alias, legacy reader, or existing-ledger migration is required.
Treat old queue ledgers as corrupt/unsupported: fail explicitly, do not reinterpret them, reset them, delete them, or bootstrap over them.
Keep completed terminal. Preserve the existing cancellation/retry policy rather than redesigning it.

Contract and invariants

Finish outcome	Claim	Work and delivery
`yielded`	Successfully finishes this activation	Inline `result` data optional; blocks reclaims during pending/failed delivery; verified Result settlement advances yield references and, only if emitted, inline data
`completed`	Successfully finishes this activation	No checkpoint required; terminal Work; successors become eligible only after verified delivery of declared outputs
`cancelled`	Cancels this activation	No checkpoint required; existing retry, backoff, and reservation-release rules continue

Use a new current-only native queue protocol version, version 4, for the affected transaction, assignment, intent, activation-snapshot, and checkpoint contracts. Do not change unrelated versioned formats such as the safe-output Result descriptor unless their shape actually changes.

Record the explicit successful outcome on Completion and Claim projections, replacing keep_open throughout both engines. Fresh effects/queue controls require the original Claim's pending delivery authority; already accepted requests remain recoverable without creating new effects. Neither a resume reference nor restored data grants authority, and no settled historical Claim becomes writable again.

Maintain a Work-scoped resume_ref with two independently sourced parts:

yield_result: the latest verified yielded Result for this exact Work, including source Claim, Completion, Result QueueCommit ID, and bounded verified output descriptor.
Optional result: the last inline result whose owning yielded Result verified successfully, including its source Claim, Completion, Result QueueCommit ID, ClaimResult QueueCommit ID, schema identity, and content digest.

On a verified yield, advance yield_result and update result only if that yield published result. Omission preserves prior verified result; an explicit empty object is a real new value when its schema permits it. Pending/failed result never replaces the last verified value. Cancellation and terminal completion do not advance this resume state.

These two sources are not an automatically coherent application snapshot: carried-forward result may describe an older branch commit. Expose their separate provenance; schema validation proves structure, not semantic consistency with newer outputs. Latest-yield output descriptors are not an accumulation of every earlier output: a no-write yield can have an empty descriptor, and references not emitted again are not implicitly copied into it.

Freeze a value copy of resume_ref into each assignment at that Claim's accepted grant, separately from DAG result_refs. The reference is selected from the accepted causal prefix immediately preceding that Claim, not whichever Work projection is newest at activation. Keep QueueCommit IDs distinct from Git object SHAs: queue IDs identify decisions, while branch-backed output descriptors carry actual immutable Git commit identities.

Optional result bytes live in a closed, claim-scoped ClaimResult operation in the same canonical queue log. Its result value must fit within 16384 canonical UTF-8 JSON bytes; measure the canonical value rather than characters, formatted input, or compressed size. Existing stricter declared/queue bounds still apply. When result is staged, trusted processing publishes Completion first, publishes it under that Claim's authority, and independently reads it back before Result can make it resumable. The safe-output call supplies only its original Claim selector and structured result; trusted processing derives ownership, schema identity, content digest, and request/commit provenance. None of these result-specific steps run when no result was emitted.

Select references by accepted ledger order, not timestamps, ref enumeration, or branch naming. A pending or failed candidate must not replace verified resume state and must still block another Claim. Return the result source explicitly so carried-forward result is not misrepresented as freshly written by the latest yield. Do not copy result bodies into Result descriptors or dispatch inputs.

For branch-backed state, carry independently verified exact-commit output references in the yielded Result; optional result may add paths/cursors but is not required. Add the missing queue delivery verifier for the supported working/PR-branch publication path rather than assuming a successful push already supplies one. A branch name is navigation, not identity. The queue guarantees which reference was accepted, not perpetual external object availability, exclusive ownership of a shared branch, or automatic restoration of a checkout.

Worker preparation must use the recorded commit with the workflow's declared checkout/read authority. If it cannot resolve required data, report an explicit input-resolution failure, not empty state or the current branch tip. The agent and MCP server remain credential-free; any necessary remote preparation uses the correct per-checkout credential in trusted setup. No arbitrary-file deserializer or new branch-storage backend is introduced.

Protocol review: inconsistencies found and corrections

The source/model review below is supported by the completed baseline TLC run recorded separately below. Existing bounded models still do not prove the proposed inline result data, ClaimResult publication, or resume_ref semantics.

Finding	Existing evidence	Correction and required coverage
The current model does not establish `resume_ref` correctness.	`specs/work-queue/FairWorkQueue.tla:111-143,239-299` models retained delivery and terminal Result authority, but no inline result data, resume reference, schema, or branch content. `specs/work-queue/README.md:47-90` states its limits.	Treat all new resume invariants as proof obligations; extend the model and add runtime/parity coverage.
Selecting from current Work state would not preserve historical assignment meaning.	`specs/work-queue/FairWorkQueue.tla:212-231` freezes grants under a checked prefix; `actions/setup/js/work_queue_binding.cjs:87-98` compares exact stored assignments.	Freeze references at the grant's operation position. This is an immutability/recovery requirement, not a claim that a later yield can overtake a still-open Claim's first activation. Recover the original assignment, not current Work state.
Two same-Work yielded Results cannot normally overtake each other.	`specs/work-queue/FairWorkQueue.tla:62-69,129-138,485-494` and `actions/setup/js/work_queue_scheduler.cjs:96-100` fence reclaims during retained pending/failed delivery.	Do not invent a timestamp tie-breaker or parallel-claim winner. Test real interleavings: settlement versus dispatch CAS, duplicate recovery, failure publication, and compaction.
Independent readback before Result is not enough if a new `ClaimResult` request can commit afterward.	`actions/setup/js/finish_work_queue_claim.cjs:358-379` rechecks the queue-control inventory inside the Result publication callback.	Include `ClaimResult` in that same inventory/CAS recheck. If `ClaimResult` commits first, stale Result proof must refresh/fail; if `Result` commits first, fresh `ClaimResult` publication must fail authority checks.
Stable request IDs alone do not enforce one result value per Claim.	`docs/src/content/docs/specs/work-queue-specification.md:828-832` prevents conflicting reuse of one request, not competing different request IDs.	Enforce a unique result slot for the source Claim/Completion in native replay/publication as well as MCP counts. Two distinct requests cannot each install result.
Result and DeliveryFailure have a first-accepted terminal decision, not “eventual good evidence wins.”	`docs/src/content/docs/specs/work-queue-specification.md:2131-2137`; `specs/work-queue/QueueLifecycle.tla:141-172`.	Revalidate exact Claim/Completion and pending barrier under every CAS retry; failure first blocks late success and leaves prior resume state unchanged.
A new Claim need not wait for the old native run/reservation to end.	`specs/work-queue/README.md:51-55` and `specs/work-queue/FairWorkQueue.tla:621-623` explicitly allow verified reclaims with an old bound reservation.	Keep capacity accounting and effect gates independent. `resume_ref` is not proof of process quiescence, API fencing, branch exclusivity, or absence of in-flight external work.
Branch push verification was assumed but is not implemented by the generic queue verifier.	`actions/setup/js/work_queue_delivery.cjs:393-424` rejects branch-update delivery; `actions/setup/js/push_to_pull_request_branch.cjs:1720-1728,1807-1820` returns handler/local evidence rather than independent readback.	Implement the scoped commit/tree verifier and guarded branch-publication path required for branch-only acceptance tests; never promote a handler SHA/URL alone to verified Result.
Pinned identity is not a global external transaction or availability guarantee.	`docs/src/content/docs/specs/work-queue-specification.md:2079-2084`; `specs/work-queue/QueueLifecycle.tla:4-11` abstracts trusted receipts and excludes host/Git behavior.	Pins prevent silent retargeting, not deletion, permission loss, or concurrent writers. Use expected-head publication guards for supported branch writes and fail explicitly on conflicts/missing objects.
Keeping only a Work's newest result can break older frozen assignments or request recovery.	`specs/work-queue/FairWorkQueue.tla:188-192` assumes full-prefix equivalence; `docs/src/content/docs/specs/work-queue-specification.md:1403-1415` preserves accepted requests, historical Claims, and Git ancestry proof.	Retain/deduplicate all result and provenance required by supported frozen assignments, pending settlement, and accepted-request recovery. Test old assignments after newer yields and repeated compaction.
Projection hashes alone do not authenticate a compacted reference.	`docs/src/content/docs/specs/work-queue-specification.md:787-800`; `actions/setup/js/work_queue_store.cjs:214-218` invokes Git-history verification.	Preserve actual prior Git ancestry/history verification, and derive retained resume state from verified prefix replay. Do not claim a self-consistent forged state proves its own origin.
Optional result still creates a possible future publication cost.	`actions/setup/js/work_queue_limits.cjs:40-73` reserves remaining retry/delivery/closure bytes but has no result capability term.	Reserve the declared worst-case result cost before accepting relevant obligations, not just after an agent emits it. Only workers with no result capability avoid that reserve.
Schema compatibility is not determined by whatever worker is deployed now.	`specs/work-queue/WorkerEvolution.tla:151-168` authenticates against frozen Dispatch; `docs/src/content/docs/specs/work-queue-specification.md:1299-1318` makes compatible evolution prospective.	Bind schema/limit identity to the approved immutable logical contract, validate source data under its original schema, and reject incompatible routes without implicit migration.
Cancellation must not be allowed to erase pending delivery authority.	`docs/src/content/docs/specs/work-queue-specification.md:1743-1746` rejects Work cancellation while retained delivery is pending; `specs/work-queue/README.md:88-93` excludes this from its focused abstraction.	Preserve that runtime guard, model its relevant extension explicitly, and prove/test that later callbacks cannot reopen cancelled Work.
Optional-result omission had an unconfirmed destructive interpretation.	Current model has no inline result-data omission/carry-forward semantics. User explicitly chose preservation during this review.	Track latest yielded Result settlement and last verified inline `result` data separately; preserve source provenance when their Claims differ.

Precise `resume_ref` guarantees to implement

These are new conditional safety requirements, not guarantees already supplied by today's TLA+ model:

Same Work and prior grant boundary: each frozen reference resolves to accepted earlier operations for that Work. A ClaimResult record can supply resumable data only after its exact owning yielded Result verifies.
Exact sources: yield_result names the most recent verified yield before that grant; result, if present, names the most recent successfully verified result write before that grant, even if a later yield omitted result.
Atomic advancement: applying the accepted yielded Result advances its resume projection and clears the regrant barrier in one deterministic replay transition. Publishing Completion, ClaimResult, verification evidence, or Release alone never does.
Stable replay: cancellation, unrelated Work, newer route declarations, lost acknowledgements, duplicate accepted requests, and compaction cannot retarget a previously accepted assignment.
No authority transfer: references and result are data, not permission to write or to replay the source Claim's effects.

These guarantees assume the existing authenticated publisher and checked single-branch Git history. They do not promise correct agent-written state, eventual scheduling/delivery, indefinite log capacity, exactly-once external effects, branch/object retention, or protection from a repository writer replacing/deleting the ledger.

TLC verification performed

Ran the official TLC 2.19 (tla2tools.jar v1.7.4) with OpenJDK 21.0.12.1, two workers, seed 1, fingerprint 0, and the repository runner's 1 GiB heap bound. Verified the jar against the SHA-256 pinned in specs/work-queue/README.md.

All 62 selected configurations returned their exact expected verdicts: 9 positive searches exhausted their bounded state spaces with no errors, 35 deliberate mutation controls produced the expected named invariant violations, and 18 witnesses produced their expected reachability counterexamples. Counterexamples in those last two groups are expected evidence, not failures of the guarded positive models or tooling failures.

Model family	Configurations run	Positive searches	Distinct states exhausted in positive searches
`specs/work-queue/QueueLifecycle.tla`	17	2	208,108
`specs/work-queue/WorkerEvolution.tla`	20	2	603,164
`specs/work-queue/ClaimScopedWorker.tla`	9	2	120,976
`specs/work-queue/FairWorkQueue.tla`	13	2	84,033
`specs/work-queue/IssueProjection.tla`	3	1	42
Total across separate bounded runs	62	9	1,016,323

Used specs/work-queue/check.sh for all registered cases in the first three families. Selected FairKeepOpen, its five BrokenKeepOpen* controls, four KeepOpen*Witness configurations, and BrokenProjection from the fair model; selected IssueKeepOpen, IssueKeepOpenWitness, and IssueBrokenKeepOpenClose from the Issue model. Ran specs/work-queue/FairCheckpoint.cfg and specs/work-queue/CheckpointRaceWitness.cfg directly through the same TLC tool/settings because they are not registered in that runner. Checked exact success/counterexample diagnostics and exit statuses rather than accepting any nonzero exit as a witness.

Relevant concrete evidence:

specs/work-queue/FairKeepOpen.cfg: 83,865 distinct states exhausted, depth 37; retained delivery, regrant fencing, ownership, historical settlement, and projection invariants held in this bounded abstraction.
specs/work-queue/KeepOpenReservationWitness.cfg: reached a verified retained round with the old native run still bound and a new Claim open, confirming that resume cannot imply old-run quiescence.
specs/work-queue/FairCheckpoint.cfg: 168 distinct states exhausted, depth 10; both Safety and bounded CheckpointProgress passed.
specs/work-queue/CheckpointRaceWitness.cfg: reached the expected stale-preparation race, confirming that compaction publication needs source/head revalidation.
Lifecycle mutation controls rejected Result-after-failure, premature release, and rerun effects. Worker-evolution controls rejected rewriting frozen Dispatch/credential identity and incompatible grants.

Logs, the run summary, and machine-readable results.json are retained as session artifacts under files/tlc-audit/ (relative to the session folder, not repository source files). No repository model/configuration was changed.

Limit: this verifies only the existing bounded protocol models, including their current keep_open vocabulary. They have no ClaimResult, result omission/carry-forward semantics, schema/byte validation, or resume_ref. The new invariants listed in implementation step 6 remain unverified until those models are extended. The state counts are a sum across separate configurations, not a proof of one combined system, unbounded behavior, physical compaction, GitHub operations, or runtime refinement.

Implementation todos

1. Replacing the native protocol contract

Update the TypeSpec source, regenerate the checked-in schemas, and migrate current repository fixtures and protocol producers/consumers to version 4.

Primary files:

specs/work-queue/transactions.tsp
specs/work-queue/generate_contract.py
pkg/workqueue/schema/
pkg/workqueue/types.go

Define the closed finish enum yielded | completed | cancelled, explicit successful Completion outcomes, the ClaimResult operation/request contract, and bounded resume_ref metadata. Checkpoint records bind the original Claim and Completion, configured schema identity, canonical result value, and content digest. Enforce the 16 KiB result cap consistently in both languages, not solely through the compiler or MCP schema. Reject every keep_open field, unsupported protocol version, invalid outcome, null selector, and conflicting duplicate finish. Preserve the existing omitted-outcome default of completed unless a current contract already requires an explicit outcome.

Do not add a return or result argument to the finish tool. Final and intermediate values continue through declared safe-outputs, including the renamed set_work_queue_result capability; it does not publish a settlement decision or bypass verification. Do not add a second return channel.

2. Implementing matching Go and JavaScript lifecycle behavior

Update native replay, request validation, finish normalization, scheduling/packing, Claim authority, delivery settlement, inspection, checkpoints, and compaction in both engines.

Primary implementation areas:

pkg/workqueue/
actions/setup/js/work_queue_replay.cjs
actions/setup/js/work_queue_scheduler.cjs
actions/setup/js/work_queue_graph.cjs

Reuse the current retained-Work delivery barrier and queue-control publication/readback patterns rather than adding an authoritative outbox or another state store. Preserve fair charging per grant, immutable Work definitions, original Claim selectors, independent batch-member settlement, reservation-release evidence, and retry reset after a verified successful yield.

Replay pending ClaimResult records separately from verified resume state. Apply a verified yielded Result, update its yield/inline-result projections, and remove the delivery fence in one transition; omitted data leaves its previous verified source intact. Build assignments from that transition's accepted prefix, copy reference values rather than sharing mutable Work objects, and use identical derivation in packing, committed grants, assignment recovery, and explanations. A later Claim cancellation/retry preserves verified references. Intermediate yielded Results never satisfy successors or close backing Issues. No finish outcome requires inline result data; both successful outcomes still require verified Result settlement for delivery success.

Queue checkpoint restoration must validate resume references against exact settlement provenance, including separate settlement identities for the latest yield and last inline result data, plus referenced bytes. Preserve equivalence across full replay, checkpoint-plus-suffix replay, repeated compaction, retries, and subsequent Claims, using the existing Git ancestry verification as well as structural checks. Retain/deduplicate data needed by all supported frozen assignments, pending deliveries, and accepted-request recovery; do not retain only the current Work pointer or duplicate every historical body into each receipt.

Enforce result-slot uniqueness in replay so different request IDs cannot bypass it. On publication retries, revalidate the original Claim/Completion, current pending barrier, and complete verified output/control inventory. Preserve Result/DeliveryFailure exclusivity and existing pending-delivery cancellation guards. Recovering an old accepted result/finish/Result request returns its historical receipt without changing newer Work resume state or granting new authority.

3. Publishing inline result safe-outputs into the queue log

Update finish staging/ingestion, result-data validation, trusted queue publication/readback, and Result construction. Reuse existing schema-validation code, but replace the current memory tool's generated Git-tree adapter with a native queue-backed result safe-output.

Primary files:

actions/setup/js/finish_work_queue_claim.cjs
actions/setup/js/work_queue_memory.cjs
actions/setup/js/work_queue_control_receipts.cjs
actions/setup/js/work_queue_delivery.cjs
actions/setup/js/push_to_pull_request_branch.cjs
pkg/workflow/work_queue_memory.go
pkg/parser/schemas/main_workflow_schema.json

Replace the old default persist_work_queue_memory tool with claim-scoped set_work_queue_result({claim_handle, result}) and the frontmatter field memory with result; do not add compatibility aliases. Preserve explicitly configured custom tool names when migrating declarations. The declaration needs schema and byte bounds, not file paths, target repositories, base revisions, or branch prefixes. Default max-bytes to 16384 and permit only lower positive values; reject declarations above the hard cap. Bind schema/limit capability to the approved immutable worker/Work logical contract, making the trusted declaration available to native validation and capacity accounting rather than trusting a worker-supplied schema hash. Schema changes are contract changes, not an implicit migration; finish/source validation uses frozen source configuration, never a newly deployed route's incompatible schema. Remove obsolete external-storage configuration rather than maintaining two result backends.

Frontmatter schema and agent-visible validation

Use the renamed tools.work-queue.result.schema location with the existing bounded declarative schema subset, without executable validators or remote schema references. result configuration is optional; when declared, its schema defines the result object's structure. For example:

tools:
  work-queue:
    worker: true
    require-assignment: true
    result:
      schema:
        type: object
        additionalProperties: false
        required: [cursor]
        properties:
          cursor: {type: string, maxLength: 1024}
          processed: {type: integer, minimum: 0}

Compile this schema into the generated safe-output tool's result input schema and protected runtime configuration. Advertise the exact schema through MCP tools/list, including custom tool names. Reuse the existing schema-generation and validation helpers rather than introducing a separate schema language or validator framework.

The MCP server must enforce the schema, strict JSON value rules, Claim selector, and canonical byte limit before appending a safe-output intent or consuming an accepted-call/cardinality budget. Advertising inputSchema alone is insufficient. Use a shared validation path for stdio and HTTP MCP transports and their runtime wrappers, so client-side validation differences cannot bypass it.

Return the repository-standard MCP error response with the failing property path, expected constraint, and correction guidance, for example result.cursor must have type string. For oversized result, include the limit and guidance to use a working branch. Do not echo the full rejected value or log secrets. Invalid calls leave staged outputs and the ledger unchanged, allowing the agent to correct the arguments and retry.

Keep trusted publication-time validation of the same schema and bounds as defense in depth against altered intents or bypassed MCP calls. Keep compile-time schema validation and MCP/publication validation aligned across the supported keywords: required fields, nested types, additional properties, enums, numeric bounds, string lengths, and array bounds/items. Reject unsupported schema definitions at compilation rather than accepting constraints the server will ignore.

Additional relevant entry points:

pkg/workflow/safe_scripts.go
actions/setup/js/generate_safe_outputs_tools.cjs
actions/setup/js/safe_outputs_handlers.cjs
actions/setup/js/mcp_server_core.cjs
actions/setup/js/safe_outputs_mcp_server_http.cjs
actions/setup/js/schema-validation.cjs

Publication and settlement

If inline result data was emitted, validate its schema and at-most-one-per-Claim cardinality, then let trusted safe-output processing append its ClaimResult operation after Completion with stable request identity and CAS publication. Independently read back that exact accepted record, bytes/digest, Claim, and Completion before verifying this output. Include it in the existing protected queue-control inventory and Result-time CAS digest recheck; no fresh ClaimResult may commit after settlement. Already accepted requests remain recoverable after settlement. Mere staging or successful publication is not verified delivery. If no inline data was emitted, publish no ClaimResult and add no implicit data-output requirement; the normal verified Result settlement barrier still applies.

Do not generate min: 1 or another implicit requirement for result on any outcome. Both yielded and completed can omit result entirely, including on workers with no result declaration. An explicitly configured Work output minimum remains that Work's own independent contract, not a native yield requirement; examples must not accidentally require result to yield. If a terminal Claim emits an authorized optional result output, verify it normally without turning it into resumable state; cancellation retains its existing effect-suppression behavior.

A second distinct result output, malformed/foreign value, or unverified result prevents verified delivery; exact retry/recovery of an already accepted request remains idempotent. An absent result output is valid. Preserve independent sibling processing and per-Claim descriptor order. Descriptor ordering does not itself prove externally ordered or atomic effects.

For branch-only support, add a scoped independent delivery verifier for the supported push_to_pull_request_branch path, reusing trusted prepared mutation, repository/ref/path authorization, and native commit/tree readback patterns. Use guarded expected-head publication and verify the actual published commit/tree against prepared content, including signing/squashing or auxiliary updates where supported. Account for every auxiliary mutation in the effect inventory; otherwise leave delivery unverified. Distinguish PR/base repository from head repository, reject foreign/missing/unreadable evidence, and thread appropriate credentials only into trusted operations. A returned/local commit_sha, handler success, moving ref, or URL is not sufficient proof. This is not general verification support for unrelated PR updates.

Do not rerun uncertain effects or imply atomic/exactly-once publication across external resources. Concurrent unrelated writers are outside queue ownership; an expected-head guard detects stale publication but does not reserve the branch indefinitely.

Reduce the old external-result ceiling to the confirmed 16 KiB queue-checkpoint cap; retain bounded schema/data nesting and strict JSON numeric rules. Oversized input fails with an actionable error directing the worker to put bulk data on its authorized working branch and checkpoint only pinned references. Do not truncate, compress around the limit, split one checkpoint across multiple messages, or silently spill to another store.

Extend admission/grant and closure/recovery accounting in both engines using the declared result capability before accepting obligations. A result-enabled worker may choose to emit up to its configured bound, so reserve that worst-case publication cost even if it eventually yields without result. Workers with no result capability need no inline-result reserve. Include request/operation/checkpoint envelope costs and pending accepted publications; consume/release reserves as obligations settle. Reject unfundable admissions/grants explicitly while preserving existing closure/recovery paths. Do not reserve infinitely many future yields or promise unlimited service on a bounded ledger.

Keep Result descriptors within 4 KiB and assignments within 48 KiB by carrying result references, not bodies. Measure full envelopes, including independently sourced result metadata. Packing must stop at an unfit fair winner without skipping it or charging rejected Claims.

4. Supplying verified same-Work state from queue replay

Use authenticated activation queue replay to resolve each Claim's frozen resume reference, not reselect from live/latest Work state. Inline result needs no external restore helper, repository scan, or extra storage credential. Branch-backed files retain their own declared checkout/read-authority requirements.

Primary integration files:

actions/setup/js/write_work_queue_snapshot.cjs
actions/setup/js/work_queue_claim_scope.cjs
actions/setup/js/work_queue_mcp_server.cjs
pkg/workflow/compiler_activation_outputs.go

For each assigned Claim, validate both frozen sources against the ledger-verified prior operations for that same Work. Supply the frozen yield's verified output references. If result exists, resolve it directly from the log or verified compacted projection and validate its bytes/digest, original schema identity, bound, and owning verified yielded Result, which may precede the latest yield. Do not relabel old result as a new checkpoint or validate it against an unrelated new schema. Branch-only resumption must work without a result declaration or record.

Expose resume context through the existing worker work_queue_read surface for explicitly selected assigned Work: frozen prior yielded Result provenance, verified output references, and optional last verified result with its independent provenance. Distinguish fresh Work, branch-only resumed Work with no prior result, and carried-forward older result. Use the read-only activation snapshot's verified queue projection, without duplicating bodies in another section. Do not return full result in general listings or make the agent-facing MCP credentialed.

Generated per-Claim prompt guidance directs the worker to read available resume context before continuing. State explicitly that yielded does not require a result call: use inline result up to 16 KiB or keep state entirely on the authorized working branch. When continuing from branch-backed state, read the recorded commit/path, not an unverified moving branch tip; unavailable referenced data is an explicit error, not empty state. This is logical resumption, not restoration of an LLM transcript/process.

A genuinely fresh Work has no prior yielded Result. A resumed Work can legitimately have output references but no inline result data; this is not corruption. If data actually referenced by resume_ref is missing, altered, oversized, or incompatible, fail explicitly rather than silently restarting or falling back. Neither yielding nor terminal workers need result configuration unless they choose to emit inline data. Reads reuse existing queue transport and authority; do not broaden grants or expose credentials to agents.

5. Updating all public and operational surfaces

Update finish tool schemas/help, prompt generation, runtime status consumers, backing-Issue projection, summaries, operator inspection, and finish-log parsing so yielded is consistently distinct from terminal completion.

Relevant entry points include:

actions/setup/js/create_prompt.cjs
pkg/cli/logs_work_queue_current.go
.github/aw/work-queue.md
docs/src/content/docs/reference/work-queue.md
docs/src/content/docs/specs/work-queue-specification.md
docs/src/content/docs/patterns/linter-factory.md

Migrate repository-owned consumers to the new protocol, finish vocabulary, and optional queue-backed result declaration. In .github/workflows/eslint-refiner.md, replace obsolete checkpoint-storage configuration/restore wiring, but preserve its independent reporting inputs, aggregate historical mission, and terminal behavior. A same-Work checkpoint is not a replacement for cross-Work history: do not remove that functionality merely to migrate storage, claim historical continuity across this cutover, or import old result/ledgers. Remove helpers only if no independent consumers remain. Keep state compact rather than storing large findings arrays. Document both inline-result and branch-only yields without accidental result requirements.

Regenerate affected lock files through the compiler. Preserve provenance-managed workflow sources and their locks; use the approved upstream update path if they are affected. Add a breaking-change changeset/release note documenting the hard cutover and unsupported old ledgers, not a migration promise.

6. Verifying coroutine behavior and persistence

Add focused Go/JavaScript conformance fixtures and tests alongside existing retained-Claim, result, delivery, snapshot, MCP, compiler, CLI-log, and compaction coverage.

Extend specs/work-queue/FairWorkQueue.tla with focused configurations for explicit yielded outcomes, at least two successive yields plus a terminal third Claim, optional result, independently sourced carried-forward result, and frozen assignment references. Add invariants for same-Work verified provenance, latest-at-grant selection, atomic resume/availability transition, unique result slots, settlement exclusivity, immutable historical assignments, and replay/checkpoint equivalence. Include deliberate broken transitions that must violate each new invariant. If state-space isolation needs a focused companion module, reuse the lifecycle semantics rather than silently assuming a runtime refinement.

Exercise competing ClaimResult/Result publications and rejected late old-Claim writes, with accepted-request recovery still allowed. Model pending-delivery Work-cancellation rejection explicitly where relevant; preserve original-Claim authority and native reservation independence. Keep fairness/retention and existing deployment/Issue/lifecycle checks applicable to renamed outcomes. Document that abstract receipt/hash/schema/size tokens do not verify real cryptography, JSON/MCP validation, Git transport, physical pruning, or runtime refinement. Full-prefix ghost-state equivalence requires separate native compaction/ancestry tests.

Required cases:

Fresh Claim -> optional result A + ordered intermediate outputs -> yield -> verified delivery -> next Claim restores A -> result B -> yield -> next Claim restores B -> terminal completion.
Branch-only yield with no result declaration/call -> verified working-branch commit receipt -> next Claim receives that exact reference and continues. No synthetic result object is required.
A legitimate no-write yield succeeds without result when its immutable output contract permits it; its empty output descriptor does not implicitly inherit an older branch receipt. Existing inline result, if any, is carried separately with its source.
Pending delivery, failed emitted-result delivery, and failed branch/other output delivery all block reclaims.
Absent result permits a valid yield; duplicate/invalid emitted result blocks verified delivery.
completed succeeds without result both with and without a result declaration; cancelled requires none and retains existing retry/effect suppression. Prior inline result must not impose a result requirement on a later yielded or terminal Claim.
Inline-result A yield -> branch-only yield -> next Claim gets the latest branch Result and carried-forward A with its original provenance. A later invalid/failed B cannot replace A.
Foreign Work/Claim references, result from terminal/failed/unverified Claims, forged Result provenance, altered checkpoint bytes, corrupt JSON, schema mismatch, and missing compacted state fail closed.
Accepted ledger order determines sources even when timestamps tie. No artificial concurrent-success/timestamp-winner race is introduced; unrelated Work never supplies state.
Compare fair-prefix planning, committed grant, stored assignment validation, activation resolution, and recovered dispatch: all retain the same frozen references, including historical reads after newer yields.
Retry after a cancelled activation preserves the last verified state; prior settled Claims cannot publish new effects.
Race two different result request IDs for one Claim: exactly one value can be accepted. Retrying its accepted stable request returns the original receipt, including after Result and compaction.
Race ClaimResult publication against a verified Result proposal. ClaimResult-first invalidates stale output-inventory evidence; Result-first rejects fresh data publication. Test both CAS orders and bounded retry exhaustion without phantom state.
Race Result against DeliveryFailure in both orders: only the first accepted settlement remains, and failure never advances resume state or permits an automatic rerun.
Recover an old accepted Result after a later verified yield: recovery returns the old receipt without regressing resume references or changing the current Claim.
Reject Work cancellation during pending yielded delivery; after verified settlement, cancellation cannot be undone by a delayed callback or duplicate request.
Grant a new Claim while the prior native reservation remains bound when capacity permits. Old Release cannot cancel the new Claim, refund charges, or grant fresh old-Claim effect authority.
Batch members cannot exchange checkpoints, selectors, completion authority, or delivery evidence.
Yields preserve fairness and leave DAG successors/backing Issues open; only verified terminal completion releases them.
Full replay, compacted replay, repeated compaction, duplicate finish recovery, and checkpoint tampering agree in both languages. Retain result/descriptor sources used by older supported assignments after the Work has newer state; reject forged source links even if snapshot hashes are self-consistent.
Race a prepared compaction/grant against Result publication; stale CAS retries rederive from the accepted prefix without losing a resume update, using the existing real Git-ancestry checks.
Compatible worker evolution preserves frozen source validation; an incompatible result schema/limit declaration cannot reinterpret old bytes or make an already-bound Claim use current-route authority.
All old protocol versions and keep_open inputs fail explicitly without empty-queue fallback.
Exactly 16384 canonical UTF-8 JSON bytes is accepted; 16385 is rejected. Cover multibyte text, escaping, lower configured limits, over-cap declarations, and unchanged state after rejection. No compressed or multi-message bypass is accepted.
Compile a workflow with frontmatter result.schema; assert the generated MCP tool advertises that exact nested schema and protected publication configuration retains it. Omit the result declaration entirely in the branch-only case.
Exercise actual MCP tools/call requests over both supported transports: wrong nested types, missing required properties, undeclared properties, invalid enums, array/string/numeric bounds, invalid Claim selectors, and oversized values return agent-visible errors with useful paths/constraints.
Verify invalid MCP calls append no output and consume no accepted result slot; a corrected retry succeeds. Tampered staged intents are rejected again at trusted publication, independently of MCP validation.
A large working-branch result uses an independently verified published commit/tree matching trusted prepared evidence. Cover signed/unsigned supported paths, stale expected heads, wrong head repository, handler-only/forged evidence, and unaccounted auxiliary mutations. Required undelivered branch writes block a verified yield.
A moved branch tip cannot retarget the frozen commit. Deletion, missing commit/tree, and lost read permission produce explicit input-resolution errors rather than fallback to the tip; no perpetual object-retention guarantee is claimed.
Check proper per-checkout credentials and sparse/shallow preparation; branch-only resolution must not trigger an authenticated remote operation in the agent/MCP server.
Measure 4 KiB descriptors, 48 KiB assignments, aggregate ledger limits, and closure/recovery headroom at boundaries. Include a result-enabled Claim that emits nothing, one that emits its full bound, and a worker with no result capability.
Checkpoint persistence/inline reads create no dedicated result branches, use no repo-memory, and require no extra inline-storage credential. Working-branch writes/readback retain their separate scoped trusted publication and checkout credentials.

Validation and handoff

Use existing tooling; no new test framework or dependency is planned.

Regenerate schemas with python3 specs/work-queue/generate_contract.py; check drift with its --check mode.
Run narrowly selected queue Go tests and targeted Vitest files together per runner, including cross-language parity and the full coroutine lifecycle.
Run compiler/integration, actual MCP server/transport, and CLI-log tests covering the affected surfaces; run the applicable bounded model configurations with the repository's existing harness.
Run make build && make fmt after significant Go edits, and make fmt-cjs && make lint-cjs for JavaScript changes. Run the existing JavaScript typecheck.
Recompile changed workflow sources and regenerate locks with make recompile; verify compiler drift through the final repository gate.
Before an intermediate save, use make agent-report-progress-no-test. Run make agent-report-progress once at the final save, including impacted test-unit, then commit/push through the available repository progress mechanism with the required coauthor trailer.

Do not trigger hosted workflows or modify any live ledger as part of this implementation. Local tests establish behavior and persistence in fixtures/disposable repositories, not live deployment.

Scope boundaries

No repo-memory integration, dedicated result branches, external Git-tree inline-result publication, automatic bulk-result storage/deserializer, native process/session continuation, new scheduler, downstream streaming/outbox protocol, checkpoint garbage collector, live-ledger migration/reset, direct agent Git publication, or alternate return channel. Scoped branch-publication verification and required trusted checkout preparation are included, not a new storage service.

The queue log is the authority for resume references and optional inline result data capped at 16 KiB; references confer no effect permissions. Working branches may hold all state or just larger data, subject to their own availability and read authority. Activation files transport verified queue data. Inline result data is optional for every outcome; omission preserves the last verified inline value with its provenance, while output references advance to the latest verified yield. Verified Result settlement remains mandatory for successful delivery.

@pelikhan

@pelikhan is this ready to turn to a PR?

no we need to make sure the work queues are operational first.

Assigned to @SivaKesava1 as a recent contributor; no strong area signal, so this balances ownership across the two recent human contributors.

Generated by 🔧 Daily Assign Issue To User · copilot · auto · ~14 AIC · ⌖ ~1 AIC · ⊞ 7.7K · ◷

Implementation plan: durable queue coroutines

Status and scope

Planning only for #69585. Do not implement, dispatch workflows, modify queue ledgers, or start a rollout as part of this request.

This refresh preserves the decisions recorded in the existing "Durable queue coroutines" issue artifact and matching design comment. Those decisions supersede the issue body's earlier mandatory memory-snapshot example. The later comment that work queues must be operational first remains an implementation prerequisite; this planning review does not establish operational readiness.

The proposed implementation makes a Work a durable logical coroutine and each Claim one activation. Explicit yielded finishes an activation without finishing the Work. completed is terminal. Both require independently verified safe-output delivery; neither files, artifacts, stdout, nor final chat supply an alternate return mechanism.

Current implementation findings

The inspected checkout still uses protocol version 3, successful completed finishes with optional keep_open, and protected Git-tree publication for declarative memory. Worker assignments carry DAG predecessor result_refs, not frozen same-Work continuation references. The ESLint worker restores workflow-specific memory history rather than a generic ledger-selected same-Work result.

Existing delivery barriers, authenticated Claim scope, checked queue publication, immutable assignments, and checkpoint ancestry verification provide the mechanisms to extend. The generic built-in delivery verifier still rejects branch updates without a pinned commit/tree verifier. Branch-only continuation therefore needs actual independent verification, not just documentation of a handler-returned SHA.

No tests or model checks were run for this planning refresh. Historical TLC results in the earlier artifact cover the old models and must not be presented as verification of the proposed result/resume protocol.

Contract to preserve

Replace keep_open with outcome: "yielded"; retain terminal completed and existing cancelled retry/effect-suppression behavior. Preserve the existing omitted-outcome default of completed.
Introduce a current-only version 4 for affected queue transactions, assignments, intents, activation snapshots, and checkpoint formats. Reject old versions and keep_open explicitly. Do not migrate, reset, delete, or silently bootstrap over unsupported ledgers. Leave unrelated versioned formats unchanged unless their shape changes.
Inline result is optional for every outcome. Declare it under tools.work-queue.result.schema and stage it with set_work_queue_result({claim_handle, result}), retaining explicitly configured custom tool names when declarations are updated.
Accept at most one result object per Claim, bounded by 16384 bytes of canonical UTF-8 JSON. Default max-bytes to that ceiling and allow lower positive limits only. Preserve strict JSON rules and the supported declarative schema subset. No truncation, splitting, compression bypass, or silent external spill.
Store optional result data in a claim-scoped ClaimResult operation in the existing queue log. Do not add a result argument to the finish tool, a memory service, or dedicated result branches.
A worker may keep all state on an authorized working/PR branch and yield without a result declaration or call. A no-write yield is also valid when the immutable output contract permits it.
Pending or failed delivery blocks another Claim. Only verified yielded Result settlement updates continuation state and releases that barrier. A yield never satisfies DAG successors or closes the backing Issue.
Keep two separately sourced continuation references: resume_ref.yield_result identifies the latest verified yielded Result; optional resume_ref.result identifies the last verified inline result from a yielded Claim. Omitting result preserves its prior value and provenance. Latest-yield output descriptors do not implicitly accumulate older outputs.
Freeze continuation references at the accepted Claim grant, independently of predecessor result_refs. Later yields, route changes, retries, or compaction must not retarget a historical assignment.
Result data and references confer no effect authority. Old accepted requests remain recoverable, but settled Claims cannot publish fresh effects.

Implementation todos

1. Defining the v4 contract and breaking cutover

Update the TypeSpec contract first, regenerate schemas, and change Go/JavaScript producers, readers, fixtures, and structural validation together. Define explicit successful Completion outcomes, ClaimResult request/operation shapes, and closed continuation-reference metadata.

Bind schema identity and byte limits to the approved immutable logical contract, not agent-supplied metadata or the currently deployed route. Distinguish QueueCommit IDs from native Git object SHAs. Preserve original Claim selection and exact accepted-request recovery.

Primary contract source:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/specs/work-queue/transactions.tsp

2. Implementing matching lifecycle and continuation projections

Extend both native engines to replace retained completion with explicit yield semantics. Reuse existing scheduling, delivery barriers, fairness charging, cancellation, reservation release, and CAS publication rules.

Store pending result records separately from verified Work continuation state. A verified yielded Result must atomically advance continuation references and remove the delivery fence. Terminal completion and cancellation do not advance resume state. Successful yields preserve the existing retry-reset behavior.

Derive frozen references from the accepted prefix immediately before the grant, using the same logic for fair-prefix packing, committed assignments, recovery, and inspection. Copy reference values rather than sharing mutable projection objects. Keep native reservation accounting independent: a new Claim can coexist with the old run's reservation only when normal capacity permits.

Primary implementation roots:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/pkg/workqueue/

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/actions/setup/js/

3. Adding the queue-backed result safe-output

Replace the declarative external-memory configuration and generated Git-tree adapter with the optional native result output. Remove obsolete storage fields rather than retaining two backends.

Advertise the exact nested schema through MCP and validate schema constraints, strict JSON, Claim selector, canonical size, and cardinality before appending an intent or consuming an accepted-call budget. Use a shared validation path for stdio and HTTP. Invalid calls must return useful property-path/constraint errors without echoing rejected data; a corrected call remains possible.

Repeat validation in trusted publication. After Completion, publish any emitted ClaimResult under the original Claim's pending-delivery authority and independently read back its exact accepted bytes, digest, schema, ownership, and Completion. No result call means no result record and no implicit output minimum.

Enforce one result slot per Claim in native replay/publication, not merely MCP counting. Include result publication in the protected output/control inventory and final settlement CAS recheck:

ClaimResult accepted first invalidates stale settlement evidence.
Result accepted first rejects fresh result publication.
An exact previously accepted request remains recoverable after settlement.
First accepted Result or DeliveryFailure wins; failure never replaces verified continuation state.

Current configuration entry point to replace:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/pkg/workflow/work_queue_memory.go

Current trusted finish integration:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/actions/setup/js/finish_work_queue_claim.cjs

4. Supporting verified branch-only continuation

Implement scoped independent verification for the supported working/PR-branch publication path. Reuse trusted prepared mutation evidence, repository/ref/path authorization, guarded expected-head publication, and immutable commit/tree/blob readback. Distinguish the PR's base repository from its head repository.

Account for signing/squashing and every auxiliary mutation on supported paths. Handler success, a local SHA, a moving branch ref, or a URL alone is insufficient. Unsupported or incomplete evidence must leave delivery unverified.

Supply exact recorded commit references to the next Claim. Trusted setup must use the declared checkout/read authority and correct per-checkout credentials, including sparse/shallow constraints. Keep the agent and MCP server credential-free. Missing objects, permission failures, or stale heads are explicit errors, never empty state or fallback to the current branch tip.

This is not a new storage service or general verification of arbitrary PR updates. Pinned identities do not guarantee indefinite object availability, branch exclusivity, process quiescence, or exactly-once external effects.

Current delivery entry point:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/actions/setup/js/work_queue_delivery.cjs

5. Preserving replay, compaction, and capacity guarantees

Retain/deduplicate result data and provenance required by supported frozen assignments, pending delivery, and accepted-request recovery, not only the newest Work pointer. Restore references against exact verified settlement sources and preserve real Git-parent/history verification.

Keep full replay, checkpoint-plus-suffix replay, repeated compaction, and historical assignment recovery equivalent in both engines. Reject self-consistent forged checkpoint links as well as altered bytes, foreign ownership, missing records, and incompatible schema provenance.

Reserve worst-case declared result capability costs before accepting admissions/grants, including record envelopes and closure/recovery headroom. A result-enabled worker reserves its configured bound even when it emits nothing; workers without the capability avoid that reserve. Do not promise infinitely many yields on a bounded ledger.

Keep Result descriptors within 4 KiB and complete assignments within 48 KiB by carrying references, not result bodies. Fair packing must stop at an unfit winning Work rather than skip it or charge an unaccepted Claim.

6. Exposing resume context and updating all consumers

Resolve each assignment's frozen references from authenticated activation replay. Expose continuation context through the existing worker work_queue_read surface for selected assigned Work, without duplicating bodies in dispatch inputs or general listings.

Distinguish fresh Work, branch-only resumed Work, and older carried-forward result data. Validate referenced inline data using its source schema/limit and owning verified yielded Result, not a newer route's declaration.

Update finish schemas, prompt generation, ingestion, authorization checks, Issue projection, summaries, CLI logs/inspection, compiler configuration, documentation, examples, and fixtures. Generated guidance must say result is optional and continuation is logical, not LLM/process restoration.

Update the ESLint worker without replacing its cross-Work historical-analysis mission with same-Work resume state. Preserve independent reporting inputs and terminal behavior; do not claim continuity across the unsupported ledger cutover. Remove legacy restore helpers only when no independent consumers remain.

Regenerate affected workflow locks through the compiler, respecting upstream-managed sources. Add a breaking-change changeset describing explicit rejection of old ledgers and the absence of migration/reset.

Current activation integration:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/actions/setup/js/write_work_queue_snapshot.cjs

Current workflow consumer:

/Users/peli/.copilot/repos/copilot-worktrees/gh-aw/pelikhan-issue-69585-add-durable-coroutine-style-yield-and-re-333db3/.github/workflows/eslint-refiner.md

7. Establishing acceptance evidence before implementation handoff

Add cross-language conformance and integration coverage for:

Two successive yields with results A/B, then terminal completion; exact same-Work restoration at each grant.
Branch-only and no-write yields without result declarations/calls; completion without result; existing cancellation/retry behavior.
Result A followed by a yield omitting result: latest output reference advances while A retains its original provenance. Pending/failed B cannot replace A.
Pending/failed output delivery fences, independent batch settlement, unchanged fairness, open backing Issues, and blocked DAG successors until verified terminal completion.
Duplicate/conflicting finishes, two distinct result request IDs, late old-Claim writes, historical request recovery, and both CAS orders for result publication versus settlement.
Result-versus-failure ordering, pending-delivery cancellation rejection, delayed callbacks, native reservation overlap, and old Release isolation.
Historical grants after newer yields, compatible/incompatible route evolution, repeated compaction, tampered provenance, and stale compaction/grant CAS retries.
Actual MCP calls over stdio/HTTP: nested schema failures, invalid selectors, unchanged staging/cardinality after rejection, corrected retries, and independent rejection of tampered intents.
Exactly 16384 versus 16385 canonical UTF-8 bytes, escaping/multibyte data, lower configured limits, over-cap declarations, descriptor/assignment bounds, and admission/closure headroom.
Branch verification with guarded heads, exact prepared content, wrong head repository, signed/unsigned supported paths, auxiliary effects, moved refs, missing objects, permission loss, and trusted per-checkout credentials.
Explicit rejection of old protocol versions and removed fields with no empty-queue fallback.

Extend the existing bounded queue models with optional result, explicit yield outcomes, separate provenance, frozen grants, unique result slots, settlement exclusivity, and continuation/checkpoint invariants. Add deliberate mutation controls with exact expected failures. Preserve applicable fairness, deployment, lifecycle, Issue, expiration, and delivery-recovery coverage.

Model checks do not replace runtime tests of JSON/MCP validation, cryptography, Git readback, physical compaction, or credentials. Record only checks actually performed during implementation.

Dependencies and validation

Protocol definition precedes lifecycle and result-output implementation. Branch verification can be developed independently of inline result once the contract is fixed. Continuation activation depends on lifecycle semantics and verified result publication; end-to-end acceptance additionally depends on branch verification and persistence/capacity work.

Use the existing schema generator and its drift check, targeted Go/Vitest tests, actual MCP transport tests, compiler/CLI tests, and bounded model harness. During implementation, apply repository formatting, lint, JavaScript typechecking, workflow recompilation, and the final impacted-test progress gate. No new framework is planned.

Do not run hosted workflows or mutate a live ledger for this feature without separate authorization and applicable repository execution gates. Operational queue readiness must be established separately before implementation/rollout proceeds.

Explicit non-goals

No implementation in this request; no live deployment; no legacy migration/reset; no repo-memory or dedicated result branches; no automatic bulk-data serialization; no process/session restoration; no new scheduler, outbox, streaming protocol, or alternate return channel.

Work-queue status and readiness for #69585

Assessment as of 2026-11-06: the basic native queue lifecycle is demonstrably working, but operational readiness is not fully established. This coroutine request remains planned, not implemented. I would not yet consider the earlier "make sure the work queues are operational first" prerequisite cleared.

This assessment checked live GitHub evidence and source pinned to main commit 8f3224323bdfd3ae8574c99c5da8a65d450577bb, rather than relying on this session's older checkout.

What is working

A real dispatcher-to-worker lifecycle completed successfully. The scheduled dispatcher and its worker passed on revision 8bb82a7563d5e61a80cfe3343e9c02a119daa459. The worker's separate verification job asserted a completed, verified queue Result, not merely a staged finish or successful agent job. Conclusion logs also confirmed native reservation release. This smoke uses an effect_contract of none; it does not validate every external safe-output path.
Compilation integration is green on the inspected main revision. Run 42259898964 passed the activation/agent/safe-output compilation integration test. This is compiler evidence, not a live scheduling or delivery stress test.
Recent fixes address concrete operational failures. #75352 reduced checkpoint-history verification overhead; #75369 stopped Issue-mirroring failures from incorrectly failing an otherwise committed queue lifecycle; #75384 fixed output_id/depends-on metadata rejection in delivery verification and closed #75274. The successful smoke above ran after these merges, but its no-effect contract does not independently validate PR/discussion delivery.

Remaining operational concerns

Area	Observed evidence	Meaning
Main-branch stress	The six most recent sampled `main` stress runs, spanning November 5-6, all failed. The three latest inspected failures share `publication_unresolved: queue CAS retries exhausted without a committed stable request`. In the latest main run, lifecycle diagnostics reported 238 completed Work and 224 verified Results out of 1,024 requested Work, with 150 CAS conflicts. Admission saturation did not run.	A recurring concurrent-publication/recovery failure remains unexplained or unresolved at the main workload. It is not evidence that every queue operation is broken, but it prevents calling the full stress path healthy.
Misleading green comparison	The newer green stress run is a pull-request run. The workflow uses 16 lifecycle items for PRs versus 1,024 on main.	That smaller run does not supersede the failing full main workload.
Issue projection	The successful worker's conclusion still logged uncertain/held projection locks, a pending target, and "Issue synchronization remains pending."	The warning-only change correctly separates bookkeeping from queue success, but does not demonstrate that mirror synchronization itself has recovered. Open mirrored Issues alone are not authoritative queue state.
Formal workflow	In the latest formal-verification workflow, both `formal_checks (QueueOrdering)` and `formal_checks (FairDAGGitHub)` passed. The overall run failed at `Upload analysis handoff` because the expected handoff files were absent.	Do not misclassify that red workflow as a model invariant failure. Equally, passing existing bounded models does not verify the proposed coroutine/result protocol.
Current queue inventory	Read-only `gh aw work-queue ... stats --json` did not return before collection was stopped. A separate 30-second `gh aw health smoke-work-queue-worker` collection was partial and reported queue evidence unavailable because its deadline expired. Both used installed CLI v1.0.1.	Current global Claim, reservation, and pending-delivery counts remain unverified. These collection limitations do not establish corruption, an empty queue, or a specific queue defect.

State of the coroutine request

The feature is still absent from the inspected main implementation:

Requested behavior	Current source
Explicit `outcome: "yielded"` replacing `keep_open`	The finish tool still accepts only `completed`/`cancelled` plus `keep_open`.
V4 queue-backed optional result data	Native types remain version 3. Declarative memory still uses a Git-tree adapter and a 256 KiB ceiling, rather than the planned 16 KiB queue-backed result.
Frozen, same-Work continuation provenance and automatic resolution	Assignment construction still includes predecessor `result_refs`, with no `resume_ref`. The replay contract has no `ClaimResult` operation.
Independently verified working-branch continuation	The built-in verifier still explicitly requires a pinned commit/tree verifier for branch updates.

The merged artifact-backed engine continuation feature, #75097, is related but does not implement this request. It restores engine session state across workflow runs; it does not supply Work-scoped yield settlement, frozen same-Work result provenance, or verified regrant semantics. It must not be treated as an alternative safe-output return channel.

Recommended disposition: keep #69585 open and its implementation plan intact. Before treating the operational prerequisite as satisfied, resolve or explicitly account for full-workload CAS exhaustion, investigate pending Issue-projection locks, and obtain an authoritative bounded queue inventory. The later coroutine implementation still requires the coordinated protocol/compiler/runtime/compaction changes and acceptance coverage in that plan.

This was a read-only assessment of source, existing runs/logs, downloaded diagnostics, and queue diagnostic commands. No implementation, new test execution, workflow dispatch, queue mutation, or rollout was performed. Run samples are bounded, not an exhaustive repository health audit.

Add durable coroutine-style yield and return to native work queues #69585

Description

Goal

Yield/resume contract

What exists today

Activity

pelikhan commented on Oct 17, 2026

elefthei commented on Oct 18, 2026

pelikhan commented on Oct 18, 2026

Durable queue coroutines

Problem and approach

Naming

Confirmed decisions

Contract and invariants

Protocol review: inconsistencies found and corrections

Precise resume_ref guarantees to implement

TLC verification performed

Implementation todos

1. Replacing the native protocol contract

2. Implementing matching Go and JavaScript lifecycle behavior

3. Publishing inline result safe-outputs into the queue log

Frontmatter schema and agent-visible validation

Publication and settlement

4. Supplying verified same-Work state from queue replay

5. Updating all public and operational surfaces

6. Verifying coroutine behavior and persistence

Validation and handoff

Scope boundaries

elefthei commented on Nov 3, 2026

pelikhan commented on Nov 4, 2026

github-actions commented on Nov 4, 2026

pelikhan commented on Nov 6, 2026

Implementation plan: durable queue coroutines

Status and scope

Current implementation findings

Contract to preserve

Implementation todos

1. Defining the v4 contract and breaking cutover

2. Implementing matching lifecycle and continuation projections

3. Adding the queue-backed result safe-output

4. Supporting verified branch-only continuation

5. Preserving replay, compaction, and capacity guarantees

6. Exposing resume context and updating all consumers

7. Establishing acceptance evidence before implementation handoff

Dependencies and validation

Explicit non-goals

pelikhan commented on Nov 6, 2026

Work-queue status and readiness for #69585

What is working

Remaining operational concerns

State of the coroutine request

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Precise `resume_ref` guarantees to implement