Skip to content

fix: recognize community submission types and report workflow outcomes - #4829

Merged
KSchlobohm merged 1 commit into
github:mainfrom
KSchlobohm:kschlobohm-oct-3-upstream-intake
Oct 3, 2026
Merged

KSchlobohm merged 1 commit into
github:mainfrom
KSchlobohm:kschlobohm-oct-3-upstream-intake

Conversation

@KSchlobohm

Copy link
Copy Markdown
Contributor

Description

Replace exact title-prefix gates in extension, preset, and bundle submission workflows with shared title/body intake instructions. Accept case, whitespace, optional-colon, and submission-title variants. Clear type-specific body fields take precedence; incidental dependency/component words do not establish a type.

Wrong-type and unclear issues receive an explanation and stop without validation, catalog edits, PRs, or label changes. Maintainers own relabeling; this does not automatically route submissions.

Require an explicit outcome, reason, owner/next action, and exact run-attempt link. Add a shared deterministic submission_outcome job for missing outcomes and incomplete agent/safe-output processing. It recognizes PAT-authored safe-output comments through the exported comment ID, deduplicates only explicit outcomes linked to the same attempt, and requires zero failed/deferred/cancelled item counters to confirm completion. Missing counters remain unconfirmed; incomplete processing is Blocked, retaining a confirmed published PR link where available.

Existing artifact validation and label-application behavior are unchanged. This excludes the separately fixed ZIP-fixture issue and the label-application fix already upstream.

User-reported context: #4670 describes a mislabeled preset; #4729, #4803, and #4802 supplied title-variant examples. These are context, not issues this PR closes.

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample project (if applicable)

Used this worktree's own virtual environment rather than the template's bare uv run pytest. Exact local commands/results on Windows, Python 3.11.5:

  • uv sync --extra test: passed; created the missing worktree virtual environment.
  • gh aw compile add-community-extension add-community-preset add-community-bundle --no-check-update: passed, 3 compiled, 0 warnings, using the existing v0.88.7 compiler. Removed only its duplicate .gitattributes addition.
  • .\.venv\Scripts\python.exe -m pytest tests\test_submission_outcomes.py tests\test_github_workflows.py -q: 144 passed, 16 skipped.
  • .\.venv\Scripts\specify.exe --help: passed.
  • git diff --check HEAD: passed.

Before/after regression evidence: the three parametrized test_submission_reporting_is_wired_into_compiled_workflow cases fail against unchanged upstream e1fa857a with missing shared imports; all pass with this patch. An isolated temporary baseline tree was used without modifying the working patch.

Hosted fork evidence at source revision 08bafe9a (all five runs concluded success, with one explicit outcome per case and successful reporting jobs):

Case Result Run
KSchlobohm/spec-kit#15, Extension submission: Validated; draft KSchlobohm/spec-kit#55, extension catalog only Run
KSchlobohm/spec-kit#21, [Preset] without colon Validated; draft KSchlobohm/spec-kit#56, preset catalog only Run
KSchlobohm/spec-kit#22, [Bundle Submission] Recognized; explicit Failed validation for outdated v0.5.1 fixture, no PR Run
KSchlobohm/spec-kit#30, preset labeled extension Wrong submission type; maintainer next action, no validation, relabeling, or PR Run
KSchlobohm/spec-kit#19, neutral title/body Needs clarification; no relabeling or PR Run

Fixture titles/bodies/labels were restored after those tests. Agent failures, missing outcomes, PAT deduplication, and item-level safe-output failures have automated local coverage, not hosted failure-injection coverage. Type recognition is agent-driven; local deterministic tests do not prove every prompt variant.

No slash commands or project scaffolding changed, so sample-project testing is not applicable. The full Python suite and upstream hosted CI were not run for this isolated patch. Earlier fork CI at 08bafe9a encountered the pre-existing timestamp-dependent ZIP-fixture checksum test failure; later passing fork CI included a separate fix and is not evidence of full CI for this PR.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Upstream preparation used GitHub Copilot App powered by GPT-6.1 Sol (gpt-6.1-sol) in autonomous mode, with runtime-default reasoning effort (not explicitly overridden). It applied the existing fork change, regenerated workflow YAML with gh-aw, inspected scope, ran local checks, and drafted this PR description. The original fork work also used GitHub Copilot App, with GPT-6.1 Sol and GPT-5.6 Sol Fast used during the session, for workflow instructions, reporting code, tests, documentation, and review-driven fixes under human direction; the exact reasoning settings for that earlier work were not recorded.

Assisted-by: GitHub Copilot App (model: GPT-6.1 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@KSchlobohm
KSchlobohm requested a review from mnriem as a code owner October 3, 2026 15:45
Copilot AI balanced review requested due to automatic review settings October 3, 2026 15:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Deterministic reporting is well tested, but agent-driven type classification still requires final human review.

Review effort: Balanced
Findings: None

What changed in this PR

Improves community submission recognition and guarantees explicit workflow outcomes.

Changes:

  • Shares flexible extension, preset, and bundle intake guidance.
  • Adds deterministic fallback reporting for missing or incomplete outcomes.
  • Adds regression tests, documentation, and regenerated workflows.
File Description
.github/​workflows/​shared/​catalog-submission.md Defines shared intake and outcome reporting.
.github/​workflows/​add-community-extension.md Imports shared extension intake guidance.
.github/​workflows/​add-community-extension.lock.yml Compiles extension reporting changes.
.github/​workflows/​add-community-preset.md Imports shared preset intake guidance.
.github/​workflows/​add-community-preset.lock.yml Compiles preset reporting changes.
.github/​workflows/​add-community-bundle.md Imports shared bundle intake guidance.
.github/​workflows/​add-community-bundle.lock.yml Compiles bundle reporting changes.
tests/​test_submission_outcomes.py Covers reporting success and failure paths.
docs/​guides/​agentic-sdlc.md Documents intake and fallback behavior.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@KSchlobohm
KSchlobohm merged commit de0cbd7 into github:main Oct 3, 2026
15 checks passed
KSchlobohm added a commit that referenced this pull request Oct 3, 2026
Create space for additional hosted validation before proceeding with or reintroducing the changes from #4829.

Assisted-by: GitHub Copilot App (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants