Repository navigation
E2E status report — 2026-11-09 (gh-aw matrix) #28748
Description
Run: https://github.com/githubnext/gh-aw-test/actions/runs/42554770262 · Trigger: schedule · Generated: 2026-11-09T08:25:38Z · Outcome: failure
Matrix summary
❌ Matrix incomplete: 0 failing test entries (91 failed test run(s)), 0 infrastructure error(s), 0 entries with no tests executed
| entry | ref | mode | samples | exit code | passed | failed | status |
|---|---|---|---|---|---|---|---|
| main | main | source | false | 0 | 1 | 0 | ✅ pass |
| main | main | source | true | ? | 80 | 29 | ⚪ unknown |
| release | v1.0.1 | extension | true | ? | 80 | 31 | ⚪ unknown |
| pre-release | v1.2.1 | extension | true | ? | 66 | 31 | ⚪ unknown |
❌ Test errors — samples mode
✅ Test successes — samples mode
✅ Test successes — inference mode
| test | v1.0.1 | v1.2.1 | main |
|---|---|---|---|
| test-copilot-create-issue | — | — | ✅ |
🤖 Agent triage prompt
Copy the block below into a coding agent (e.g. Copilot) to investigate the failures:
You are triaging failures from the gh-aw-test E2E suite.
Run: https://github.com/githubnext/gh-aw-test/actions/runs/42554770262
Repository under test: github/gh-aw (the gh-aw CLI/compiler).
Test harness repository: githubnext/gh-aw-test (this repo; runner is e2e.sh).
Focus: gh-aw main (commit 984d37677904), sampling = true (--use-samples; AI engine was NOT called).
Failed tests (main, samples=true):
- test-copilot-close-discussion run:
- test-copilot-close-issue run:
- test-copilot-assign-to-agent run: https://github.com/githubnext/gh-aw-test/actions/runs/42557306655
- test-copilot-mark-pull-request-as-ready-for-review run: https://github.com/githubnext/gh-aw-test/actions/runs/42558715115
- test-copilot-resolve-pull-request-review-thread run: https://github.com/githubnext/gh-aw-test/actions/runs/42558719603
- test-copilot-push-to-pull-request-branch-using-dispatch run: https://github.com/githubnext/gh-aw-test/actions/runs/42558718132
- test-copilot-submit-pull-request-review-locked run: https://github.com/githubnext/gh-aw-test/actions/runs/42558719748
- test-copilot-create-pull-request-auto-merge run:
- test-copilot-dispatch-workflow-issue-comment-ref run:
- test-copilot-dispatch-workflow run: https://github.com/githubnext/gh-aw-test/actions/runs/42560158330
- test-copilot-missing-tool run: https://github.com/githubnext/gh-aw-test/actions/runs/42560155419
- test-copilot-noop run: https://github.com/githubnext/gh-aw-test/actions/runs/42560155560
- test-copilot-report-incomplete run: https://github.com/githubnext/gh-aw-test/actions/runs/42560156390
- test-copilot-inline-sub-agents run: https://github.com/githubnext/gh-aw-test/actions/runs/42560156413
- test-copilot-upload-code-coverage run: https://github.com/githubnext/gh-aw-test/actions/runs/42560156932
- test-copilot-update-release run: https://github.com/githubnext/gh-aw-test/actions/runs/42560158273
- test-copilot-dispatch-workflow-ref-override run: https://github.com/githubnext/gh-aw-test/actions/runs/42560158699
- test-copilot-submit-pull-request-review-commit-id run: https://github.com/githubnext/gh-aw-test/actions/runs/42560166838
- test-copilot-create-pull-request-labels run:
- test-copilot-playwright-cli run: https://github.com/githubnext/gh-aw-test/actions/runs/42561734661
- test-copilot-network-engine-domain-opt-in run: https://github.com/githubnext/gh-aw-test/actions/runs/42561734875
- test-copilot-cooldown run: https://github.com/githubnext/gh-aw-test/actions/runs/42561734542
- test-copilot-sandbox-runtime-profile run: https://github.com/githubnext/gh-aw-test/actions/runs/42561734906
- test-copilot-add-labels-target-restriction run: https://github.com/githubnext/gh-aw-test/actions/runs/42561739660
- test-copilot-push-to-pull-request-branch-no-fallback run: https://github.com/githubnext/gh-aw-test/actions/runs/42561744383
- test-copilot-repo-memory run: https://github.com/githubnext/gh-aw-test/actions/runs/42563231701
- test-copilot-assign-to-agent-with-config run: https://github.com/githubnext/gh-aw-test/actions/runs/42563232897
- test-copilot-nosandbox-create-repository-code-scanning-alert run: https://github.com/githubnext/gh-aw-test/actions/runs/42563232648
- test-copilot-nosandbox-create-discussion run: https://github.com/githubnext/gh-aw-test/actions/runs/42563232736
Goal: for EACH failed test above, access the GitHub
Actions logs for the run above (and the per-entry artifacts e2e-<label>-samples-<bool>,
which contain e2e-test-*.log, e2e-output.log and fails.txt), determine the root
cause, and categorize the failure as exactly one of:
1. TRANSIENT — flaky/infra/network/rate-limit/timing; not a real defect.
Action: note it and recommend a re-run (./e2e.sh rerun).
2. TEST-FRAMEWORK BUG — a defect in this repo's harness (e2e.sh), a workflow
source file (.github/workflows/test-*.md), a sample, or CI config.
Action: propose a concrete fix (file + change) in githubnext/gh-aw-test.
3. GH-AW BUG — a defect in github/gh-aw itself (compiler output, runtime
engine behaviour, safe-output handling, etc.).
Action: open an issue in github/gh-aw with a minimal repro, the failing
test name, the gh-aw ref/mode/samples combination, and links to the
relevant log lines. Check for an existing open issue first and link it
instead of filing a duplicate.
Steps:
- Use 'gh run view <run-id> --log' and 'gh run download <run-id>' to fetch logs/artifacts.
- Read AGENTS.md in githubnext/gh-aw-test for harness conventions before proposing fixes.
- Group failures by suspected root cause; the same gh-aw bug may explain several.
- Produce a table: test | category | root cause | recommended action | issue/PR link.
- Only open github/gh-aw issues for category 3, and only after confirming no duplicate exists.
Activity