Repository navigation
[CI / CD] System to perform tasks on a test repo, end to end #40
Description
Activity
Keeping this open after the CNCF transfer — it's still the largest testing gap. The repo dogfoods itself via
.github/workflows/{issueComment,onPr,lgtm-merger}.yml, which gives partial coverage. A nightly workflow against a sandbox repo exercising/lgtm,/approve,/assignand the cron merge would close the gap. Low priority.- added a commit that references this issue
on Sep 27, 2026 Closed PR #210 is a first cut at this: branch
test/e2e-sandbox, drivere2e/sandbox.mts. It runs the committeddist/index.jsagainst a real sandbox repo and covers label-sync, the author's own/lgtmbeing refused,/assign,/approvewith OWNERS,/lgtmmerging on the comment event,/hold, and the lgtm cron merge.Two things have to come from a maintainer before it can land:
- The workflow file.
.github/workflows/e2e.yml(the YAML is in test: nightly end-to-end run of the bundle against a sandbox repository #210) needs to be pushed by someone whose credentials have theworkflowscope. Hive's token doesn't have it. - The sandbox. A dedicated sandbox repo, two accounts with tokens (
E2E_REVIEWER_TOKEN,E2E_AUTHOR_TOKEN), and theE2E_REPOSITORYvariable. Two accounts are needed because the author can't/lgtmtheir own PR.
The full scenario has not run against a real sandbox yet. Only the failure paths and a read-only label-sync dry-run against this repo were exercised.
— hive: backend=omp model=github-copilot/claude-opus-5.5 effort=medium
🐝 Hive Agent:
contributor| SHA:aaeafff- The workflow file.
- added a commit that references this issue
on Sep 27, 2026 Closing as superseded. The gap this was opened for in 2021 — nothing exercises the action end to end, only unit tests — is now covered by the bundle acceptance harness in
__tests__/bundle/: 72 files / 452 tests run the committeddist/index.jsas a child process against a fake GitHub API server and currently reach 99.3 % ofsrc/lines end to end (npm run test:coverage:e2e). That catches the "new changes break existing operations" half of this issue on every PR, offline and without secrets.The other half — detecting upstream GitHub API drift by running against a real sandbox repository — is a different, infra-bound deliverable: #210 has a working first cut (
e2e/sandbox.mts+ a nightly workflow) but needs a dedicated sandbox repo, two bot accounts (E2E_REVIEWER_TOKEN/E2E_AUTHOR_TOKEN) and theE2E_REPOSITORYvariable from a maintainer. If we decide to fund that, it should be a fresh, narrowly-scoped issue pointing at #210 rather than this one.Triage context: #414.
✨ Feature ✨
We should build a system to do prow operations end to end on a test repo.
This would be obviously beneficial since it would create a way to determine if the API has changed, if new changes to the prow bot break the existing operations and functionaliy, etc.
As it stands today, we run unit tests, but nothing end to end on an actual GitHub repository.