docs(e2e): document overlay fullsend triage in the CI Medic guide - #5296
docs(e2e): document overlay fullsend triage in the CI Medic guide#5296subhashkhileri wants to merge 3 commits into
Conversation
Give CI medics a workflow for overlay nightly failures: fullsend classifies them, files GitHub issues, and opens PRs that medics validate and merge. Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5296 +/- ##
==========================================
- Coverage 63.71% 63.68% -0.04%
==========================================
Files 121 121
Lines 2285 2288 +3
Branches 525 503 -22
==========================================
+ Hits 1456 1457 +1
- Misses 827 830 +3
+ Partials 2 1 -1
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
Tracked under epic RHIDP-12914: https://redhat.atlassian.net/browse/RHIDP-16515 |
| - Check [Jira](https://redhat.atlassian.net/jira/dashboards/21388#v=1&d=21388&rf=acef7fac-ada0-4363-b3fb-9aad7ae021f0&static=f0579c09-f63e-45aa-87b9-05e042eee707&g=60993:view@0a7ec296-c2fd-4ddc-b7cb-64de0540e8ba) for existing issues with the **`ci-fail`** label. | ||
| - If it's a **new issue**, create a bug and assign it to the responsible team or person. The AI triager can also create Jira bugs directly. | ||
| - If the failure **blocks PRs**, mark the test as skipped (`test.fixme`) until it is fixed. | ||
| - **Overlay** (`redhat-developer/rhdh-plugin-export-overlays`): Open the **E2E AI Triage Summary** link in the Slack alert. Fullsend has already classified each workspace failure, reused or created GitHub issues, and opened fix PRs where it can. Your job is to validate that work -- see [Overlay Nightly Failures (Fullsend)](#overlay-nightly-failures-fullsend). |
There was a problem hiding this comment.
I guess you'll add the automated sending of the comments later? I see the issue for today's nightly (redhat-developer/rhdh-plugin-export-overlays#3382), but don't see it automatically sent to https://redhat-internal.slack.com/archives/C07D05S8U82/p1787637084192769
There was a problem hiding this comment.
the change was done yesterday, so it should appear today.
There was a problem hiding this comment.
Ah, nice. I can see it now :D https://redhat-internal.slack.com/archives/C07D05S8U82/p1787724724911539
| |---|---|---| | ||
| | **Prow job name** | `periodic-ci-redhat-developer-rhdh-{BRANCH}-e2e-...` | `periodic-ci-redhat-developer-rhdh-plugin-export-overlays-{BRANCH}-e2e-...` | | ||
| | **Automated triage** | **AI Test Triager** (`@Nightly Test Alerts`) -- inline Slack analysis | **Fullsend only** -- the **E2E AI Triage Summary** GitHub comment in the alert. The AI Test Triager is not configured for overlay. | | ||
| | **Tracking** | Jira with the **`ci-fail`** label | GitHub issues titled `[fullsend] E2E: ...` | |
There was a problem hiding this comment.
Should this be Automated tracking instead? We still want to create ci-fail issues for bugs that aren't automatically fixed by Fullsend, right?
|
|
||
| PR check jobs use the `pull-ci-` prefix instead of `periodic-ci-`. | ||
|
|
||
| **Overlay jobs** insert `plugin-export-overlays` after the org/repo segment: |
There was a problem hiding this comment.
Maybe we can rework this part and explain that the part after periodic-ci- is the org/repository. Or even add a link to https://docs.ci.openshift.org/how-tos/naming-your-ci-jobs/
There was a problem hiding this comment.
I meant that we can rewrite the whole Job Naming Convention section, with the explanation of the name breakdown. But this is enough, too.
Keep Jira ci-fail in the medic path for overlay bugs fullsend does not close, and describe job names as org/repo after the periodic-ci- prefix. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/lgtm |
Explain OpenShift CI names as job-type, org, and repo first so Core and overlay jobs share one breakdown instead of a Core-only pattern. Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
/lgtm |
2 participants
Summary
#rhdh-e2e-alerts/#rhdh-e2e-alerts-1-10include an E2E AI Triage Summary GitHub comment from fullsend.test_fix,product_bug,environment,infra_flake): validate issues/PRs, merge obvious test fixes, ping workspace CODEOWNERS otherwise,/fs-fixwhen the approach is wrong, file Jira and merge skip PRs for product bugs, and comment on GitHub issues when the analysis is off so the next run has the right context.@Nightly Test Alerts) is not configured for overlay — fullsend is the only automated triage for those jobs.Test plan
docs/e2e-tests/CI-medic-guide.mdOverview + Overlay Nightly Failures section as a CI medic would on Day 1Made with Cursor