E2E status report — 2026-11-08 (gh-aw matrix) #28548

You are triaging failures from the gh-aw-test E2E suite.
Run: https://github.com/githubnext/gh-aw-test/actions/runs/42441403908
Repository under test: github/gh-aw (the gh-aw CLI/compiler).
Test harness repository: githubnext/gh-aw-test (this repo; runner is e2e.sh).

Focus: gh-aw main (commit 2a5905e6879a), sampling = true (--use-samples; AI engine was NOT called).

Failed tests (main, samples=true):
  - test-copilot-add-discussion-comment  run: 
  - test-copilot-replace-label  run: 
  - test-copilot-update-issue  run: 
  - test-copilot-assign-to-user  run: 
  - test-copilot-set-issue-type  run: 
  - test-copilot-command  run: 
  - test-copilot-push-to-pull-request-branch-using-dispatch  run: https://github.com/githubnext/gh-aw-test/actions/runs/42443813734
  - test-copilot-resolve-pull-request-review-thread  run: https://github.com/githubnext/gh-aw-test/actions/runs/42443814635
  - test-copilot-reply-to-pull-request-review-comment  run: https://github.com/githubnext/gh-aw-test/actions/runs/42443815342
  - test-copilot-create-pull-request-auto-merge  run: 
  - test-copilot-dispatch-workflow-issue-comment-ref  run: 
  - test-copilot-submit-pull-request-review-commit-id  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444895128
  - test-copilot-noop  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444888501
  - test-copilot-missing-tool  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444888355
  - test-copilot-missing-data  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444888911
  - test-copilot-call-workflow  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444889145
  - test-copilot-upload-code-coverage  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444889089
  - test-copilot-network-isolation  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444888874
  - test-copilot-report-incomplete  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444888938
  - test-copilot-dispatch-workflow  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444890820
  - test-copilot-update-release  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444889936
  - test-copilot-dispatch-workflow-ref-override  run: https://github.com/githubnext/gh-aw-test/actions/runs/42444890740
  - test-copilot-create-pull-request-labels  run: 
  - test-copilot-sandbox-runtime-profile  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446004061
  - test-copilot-network-engine-domain-opt-in  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446003959
  - test-copilot-playwright-cli  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446003998
  - test-copilot-mcp-large-json-payload  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446004214
  - test-copilot-sandbox-exclude-env  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446004217
  - test-copilot-add-discussion-comment-dispatch  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446006633
  - test-copilot-push-to-pull-request-branch-no-fallback  run: https://github.com/githubnext/gh-aw-test/actions/runs/42446010479
  - test-copilot-cooldown  run: https://github.com/githubnext/gh-aw-test/actions/runs/42447028700
  - test-copilot-assign-to-agent-with-config  run: https://github.com/githubnext/gh-aw-test/actions/runs/42447511598
  - test-copilot-repo-memory  run: https://github.com/githubnext/gh-aw-test/actions/runs/42447512283
  - test-copilot-nosandbox-create-repository-code-scanning-alert  run: https://github.com/githubnext/gh-aw-test/actions/runs/42447512257
  - test-copilot-nosandbox-push-to-pull-request-branch  run: 
  - test-copilot-nosandbox-add-labels  run: 

Goal: for EACH failed test above, access the GitHub
Actions logs for the run above (and the per-entry artifacts e2e-<label>-samples-<bool>,
which contain e2e-test-*.log, e2e-output.log and fails.txt), determine the root
cause, and categorize the failure as exactly one of:

  1. TRANSIENT — flaky/infra/network/rate-limit/timing; not a real defect.
     Action: note it and recommend a re-run (./e2e.sh rerun).
  2. TEST-FRAMEWORK BUG — a defect in this repo's harness (e2e.sh), a workflow
     source file (.github/workflows/test-*.md), a sample, or CI config.
     Action: propose a concrete fix (file + change) in githubnext/gh-aw-test.
  3. GH-AW BUG — a defect in github/gh-aw itself (compiler output, runtime
     engine behaviour, safe-output handling, etc.).
     Action: open an issue in github/gh-aw with a minimal repro, the failing
     test name, the gh-aw ref/mode/samples combination, and links to the
     relevant log lines. Check for an existing open issue first and link it
     instead of filing a duplicate.

Steps:
  - Use 'gh run view <run-id> --log' and 'gh run download <run-id>' to fetch logs/artifacts.
  - Read AGENTS.md in githubnext/gh-aw-test for harness conventions before proposing fixes.
  - Group failures by suspected root cause; the same gh-aw bug may explain several.
  - Produce a table: test | category | root cause | recommended action | issue/PR link.
  - Only open github/gh-aw issues for category 3, and only after confirming no duplicate exists.