Skip to content

Make work queue Issue synchronization failures warning-only - #75369

Merged
pelikhan merged 1 commit into
mainfrom
pelikhan/work-queue-smoke
Nov 6, 2026
Merged

pelikhan merged 1 commit into
mainfrom
pelikhan/work-queue-smoke

Conversation

@pelikhan

@pelikhan pelikhan commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

A worker can finish its Claim with independently verified delivery and release its native reservation, yet still fail because the conclusion job cannot synchronize its tracking Issue. Issue projection is bookkeeping and should not turn a committed queue transition into a workflow failure.

Changes

  • Emit warnings for pending Issue synchronization and coordination-lock cleanup in worker conclusion and scheduled maintenance.
  • Preserve pending diagnostics, exact-request recovery, and uncertain-lock fencing. Claim delivery and Result verification failures remain fatal.
  • Add regression coverage for warning behavior and projector credential wiring, document required token permissions, and regenerate affected workflow locks and maintenance.

Validation

  • make agent-report-progress passed, including impacted tests, linting, and generated-workflow drift checks.
  • Focused Go and JavaScript reconciliation/Issue projection tests and JavaScript typechecking passed.
  • All 359 workflows compiled successfully; existing repository warnings remain.
  • Observer smoke run passed. The native worker recorded Completion, a verified Result, and reservation Release. The authoritative queue subsequently had zero open Claims and zero native reservations.

Operational notes

The live runs preceded this patch. The worker's overall run failed on pending Issue projection despite successful Claim verification; the dispatcher separately encountered a transient native-run title mismatch that trusted activation later recovered. This change does not repair uncertain locks or alter dispatch correlation, and was not rerun live.

Projector jobs and hook-specific GitHub App tokens retain contents: write, issues: write, and actions: read. Issue updates and lock cleanup use the same client. The default token expression is GH_AW_GITHUB_TOKEN || GITHUB_TOKEN; masked run logs cannot verify a custom PAT's actual scopes.

Keep Issue projection and coordination cleanup nonfatal after committed queue transitions. Preserve pending diagnostics, fatal Claim verification, and scoped projector credentials.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review November 6, 2026 08:31
Copilot AI auto review requested due to automatic review settings November 6, 2026 08:31
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

PR Specification Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from PR Specification Reviewer · gpt56 · ~2 AIC · ⌖ ~1 AIC · ⊞ 1.6K · ◷

Generated by PR Specification Reviewer for #75369

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: gpt56 max · ~1 AIC · ⌖ ~2 AIC · ⊞ 7.7K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt56 max · ~4 AIC · ⌖ ~1 AIC · ⊞ 118 · ◷

Generated by Ponytail Reviewer for #75369

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: haiku55 xhigh · ~1 AIC · ⌖ ~1 AIC · ⊞ 6.3K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-11-06T08:34:34Z
review_event: REQUEST_CHANGES
top_themes: [warning-only projection masks authentication and journal-integrity failures]
files_reviewed: [pkg/workflow/work_queue_issues.go, actions/setup/js/work_queue_issues.cjs, actions/setup/js/reconcile_work_queue_run.cjs, pkg/workflow/maintenance_workflow_work_queue_compaction.go]
comment_count: 1

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning-only transition is not safely scoped: the conclusion projector now masks authentication, identity, and journal-integrity failures as pending synchronization, allowing a security-relevant failure to finish green.

Highlights
  • Preserve fatal handling for non-retryable trust/integrity errors.
  • Downgrade only explicitly typed, safe Issue synchronization conditions.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

@@ -160,6 +160,6 @@ func (c *Compiler) buildWorkQueueIssuesStep(data *WorkflowData) []string {
" script: |\n",
" const { main } = require('${{ runner.temp }}/gh-aw/actions/work_queue_issues.cjs');\n",
" const result = await main({ core, githubClient: github, context });\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes authentication, identity, and journal-integrity failures into warnings because main() catches every exception and returns it as pending, allowing a security-relevant projection failure to finish the conclusion job successfully.

💡 Preserve fatal projection errors

work_queue_issues.cjs catches errors from publisher authentication, queue validation, and checked-journal reads in the outer catch and converts them to a pending result, so this caller can no longer distinguish retryable Issue synchronization from trust/integrity failures. Keep a typed fatal/error result (or rethrow non-retryable codes) and only downgrade the explicitly safe pending cases; this preserves the stated guarantee that delivery and verification/integrity failures remain fatal.

@pelikhan
pelikhan merged commit 8f4f48c into main Nov 6, 2026
164 of 182 checks passed
@pelikhan
pelikhan deleted the pelikhan/work-queue-smoke branch November 6, 2026 08:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The ADR still specifies failure for pending projections during conclusion and maintenance, contradicting the warning-only policy documented in the work-queue reference.

1 open issue
What changed in this PR

Updates gh-aw’s work-queue bookkeeping so pending Issue synchronization does not fail worker conclusion or scheduled maintenance after queue transitions have committed.

Changes:

  • Replaces pending-sync failures with warnings while preserving fatal delivery, verification, and unresolved reconciliation errors.
  • Adds regression assertions for warning behavior and projector credential wiring.
  • Documents projector permissions and regenerates affected workflows.
File Description
pkg/​workflow/​work_queue_issues.go Generates warning-only Issue projection handling.
pkg/​workflow/​work_queue_issues_test.go Checks warnings and projector token wiring.
pkg/​workflow/​maintenance_workflow_work_queue_compaction.go Generates deferred synchronization warnings after compaction.
pkg/​workflow/​maintenance_workflow_work_queue_compaction_test.go Checks warning-only maintenance generation.
docs/​src/​content/​docs/​reference/​work-queue.md Documents warning behavior and token permissions.
actions/​setup/​js/​reconcile_work_queue_run.test.cjs Checks pending-sync warnings and deferral.
actions/​setup/​js/​reconcile_work_queue_run.cjs Warns instead of throwing for pending synchronization.
.github/​workflows/​smoke-work-queue-worker.lock.yml Uses warning-only projection.
.github/​workflows/​smoke-work-queue-dispatcher.lock.yml Uses warning-only projection.
.github/​workflows/​repo-tree-map.lock.yml Uses warning-only projection.
.github/​workflows/​eslint-refiner.lock.yml Uses warning-only projection.
.github/​workflows/​eslint-monster.lock.yml Uses warning-only projection.
.github/​workflows/​eslint-miner.lock.yml Uses warning-only projection.
.github/​workflows/​eslint-factory-dispatcher.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-pydantic-ai.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-pi.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-opencode.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-kiro.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-goose.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-gemini.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-factory.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-dispatcher.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-deepseek-harness.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-cursor.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-crush.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-copilot.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-codex.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-claude.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-atomic.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-aider.lock.yml Uses warning-only projection.
.github/​workflows/​engine-conformance-agy.lock.yml Uses warning-only projection.
.github/​workflows/​deep-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-token-consumption-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-team-evolution-insights.lock.yml Uses warning-only projection.
.github/​workflows/​daily-secrets-analysis.lock.yml Uses warning-only projection.
.github/​workflows/​daily-report-dispatcher.lock.yml Uses warning-only projection.
.github/​workflows/​daily-repo-chronicle.lock.yml Uses warning-only projection.
.github/​workflows/​daily-regulatory.lock.yml Uses warning-only projection.
.github/​workflows/​daily-observability-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-issues-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-firewall-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-evals-report.lock.yml Uses warning-only projection.
.github/​workflows/​daily-compiler-quality.lock.yml Uses warning-only projection.
.github/​workflows/​artifacts-summary.lock.yml Uses warning-only projection.
.github/​workflows/​agentics-maintenance.yml Emits synchronization warnings after maintenance compaction.

Review effort: Max (auto)

The compiler generates `agentic-workqueue.yml` for Issue events. Closing an exactly bound Issue can cancel unfinished Work after fresh identity and label checks; it cannot revoke completed delivery, stop a worker, or establish Result.

Projection hooks require `contents: write`, `issues: write`, and `actions: read`, with credentials kept outside agent execution. Failed projection does not roll back queue admission or justify rerunning a worker. Ambiguous Issue/comment creation remains pending rather than blindly creating duplicates; automatic recovery of unreceipted creation is not implemented.
Projection hooks require `contents: write`, `issues: write`, and `actions: read`, with credentials kept outside agent execution. Issue updates and coordination-lock cleanup use the same projector token; custom tokens must grant those permissions for every projected repository. Failed projection emits a warning rather than failing worker conclusion or scheduled maintenance; it does not roll back queue admission or justify rerunning a worker. Claim delivery and Result verification failures remain fatal. Ambiguous Issue/comment creation remains pending rather than blindly creating duplicates; automatic recovery of unreceipted creation is not implemented.
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants