Add ACS SMOKE test pipeline for OPP interop - #82838
Conversation
Add stackrox-opp-readiness and stackrox-opp-smoke step-registry components for running ACS SMOKE tests as part of the OPP interop pipeline. Phase 1a: stackrox-opp-readiness step - Polling loop with 30s interval / 5min timeout - Namespace discovery via Central and SecuredCluster CRs - Central route, API health, cluster connectivity checks - Sensor pod status with OOMKilled detection - Default policy count validation (>80) - Credential extraction with xtrace protection - Writes ROX_ADMIN_PASSWORD, CENTRAL_URL, CENTRAL_NS, SC_NS to SHARED_DIR Phase 1b: stackrox-opp-smoke step - Sparse-clones stackrox/stackrox and stackrox/scanner repos - Runs ./gradlew testSMOKE from qa-tests-backend/ - Copies JUnit XML results to ARTIFACT_DIR - Uses acs-smoke-runner image (UBI9 + OpenJDK 17) Phase 1c: OPP 4.22 config update - Adds dockerfile_literal image build for acs-smoke-runner - Inserts readiness and smoke refs into interop-opp-aws test chain after cucushift-installer-check-cluster-health Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The generate-registry-metadata tool rewrites metadata.json files to canonical format with path and owners (derived from OWNERS files), replacing the manual labels/description fields. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: amp-rh The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe AWS interop workflow builds an ACS smoke-test image, validates StackRox OPP readiness, and runs the ACS backend smoke suite with published reports. ChangesACS OPP smoke-test integration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant AWSInterop
participant ReadinessGate
participant OpenShift
participant SmokeRunner
participant ARTIFACT_DIR
AWSInterop->>ReadinessGate: validate StackRox OPP readiness
ReadinessGate->>OpenShift: check Central, clusters, sensors, and policies
OpenShift-->>ReadinessGate: return readiness status
ReadinessGate->>SmokeRunner: provide SHARED_DIR credentials and endpoints
SmokeRunner->>OpenShift: run ACS backend smoke tests
SmokeRunner->>ARTIFACT_DIR: publish test reports
🚥 Pre-merge checks | ✅ 14 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (14 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh (1)
40-41: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winPin the cloned repositories to a specific ref for reproducible smoke runs.
git clone --depth 1 --filter=blob:none --sparsepulls the defaultmainbranch ofstackrox/stackroxandstackrox/scannerwith no commit or tag pin. Any change on those repos'mainbranches (e.g., a brokenqa-tests-backendcommit) will affect every future run of this step, independent of changes in this repository, making failures harder to attribute and reproduce.Pin to a known-good tag or commit SHA, and consider making it overridable via an environment variable for maintenance.
Also applies to: 49-50
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh` around lines 40 - 41, Update the StackRox repository clone commands in the smoke-test setup to check out a known-good tag or commit SHA instead of the default branch, applying the same pin to the scanner clone. Allow the pinned ref to be overridden through an environment variable for maintenance while retaining the known-good value as the default.ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml (1)
51-57: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winPin the base image tag and verify the downloaded client binaries.
The
acs-smoke-runnerimage buildsFROM registry.access.redhat.com/ubi9/openjdk-17:latest, an unpinned tag, and extractsoc/kubectlfrom a remote tarball with no checksum or signature verification. Both reduce build reproducibility: the image content can change silently between CI runs, and a compromised or corrupted download would not be detected.Pin the base image to a specific tag or digest, and validate the downloaded tarball against a published checksum before extracting it.
🔒 Proposed hardening for the Dockerfile literal
- dockerfile_literal: | - FROM registry.access.redhat.com/ubi9/openjdk-17:latest + FROM registry.access.redhat.com/ubi9/openjdk-17:1.21 USER root RUN microdnf install -y git && microdnf clean all - RUN curl -sL https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-linux.tar.gz \ - | tar xz -C /usr/local/bin oc kubectl + RUN curl -sLo /tmp/oc-client.tar.gz https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-linux.tar.gz \ + && curl -sL https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/sha256sum.txt | grep openshift-client-linux.tar.gz | sha256sum -c - \ + && tar xz -f /tmp/oc-client.tar.gz -C /usr/local/bin oc kubectl \ + && rm -f /tmp/oc-client.tar.gz🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml` around lines 51 - 57, Update the acs-smoke-runner dockerfile_literal to use a specific immutable UBI OpenJDK base tag or digest instead of latest, and download the OpenShift client checksum from the corresponding published source. Verify the tarball against that checksum before extracting oc and kubectl, failing the build on mismatch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/stackrox/opp-readiness/stackrox-opp-readiness-commands.sh`:
- Around line 80-81: Remove the echo of CENTRAL_URL from the readiness command
flow to prevent the Central route host from appearing in CI logs. Keep writing
CENTRAL_URL to SHARED_DIR as the existing downstream mechanism, without adding
another URL-bearing log statement.
- Around line 98-107: Replace the unconditional `set -x 2>/dev/null || true`
restoration commands in `check_clusters_connected` and the corresponding blocks
around lines 112-117, 166-175, and 184-186 with restoration that keeps xtrace
disabled. Preserve the existing sensitive-command protection and error-return
behavior without enabling tracing for the remainder of the script.
In
`@ci-operator/step-registry/stackrox/opp-readiness/stackrox-opp-readiness-ref.yaml`:
- Line 9: Increase the step timeout configured near the stackrox-opp-readiness
step beyond 10m0s so the seven sequential wait_for checks in
stackrox-opp-readiness-commands.sh can exhaust their retry budgets and emit
their own timeout diagnostics before ci-operator terminates the pod.
In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh`:
- Line 31: Remove the Central URL value from the logging statement in the smoke
command script by deleting the echo or replacing ${CENTRAL_URL} with a redacted
placeholder, while preserving other smoke-test logging.
- Around line 27-29: Remove the unconditional tracing enablement after reading
ROX_ADMIN_PASSWORD in the stackrox-opp-smoke command flow. Preserve tracing as
disabled, matching the behavior of stackrox-opp-readiness-commands.sh, and avoid
invoking set -x in this script.
---
Nitpick comments:
In
`@ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml`:
- Around line 51-57: Update the acs-smoke-runner dockerfile_literal to use a
specific immutable UBI OpenJDK base tag or digest instead of latest, and
download the OpenShift client checksum from the corresponding published source.
Verify the tarball against that checksum before extracting oc and kubectl,
failing the build on mismatch.
In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh`:
- Around line 40-41: Update the StackRox repository clone commands in the
smoke-test setup to check out a known-good tag or commit SHA instead of the
default branch, applying the same pin to the scanner clone. Allow the pinned ref
to be overridden through an environment variable for maintenance while retaining
the known-good value as the default.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 6f2aed87-8e32-442a-837a-76c046a31b39
📒 Files selected for processing (9)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/step-registry/stackrox/opp-readiness/OWNERSci-operator/step-registry/stackrox/opp-readiness/stackrox-opp-readiness-commands.shci-operator/step-registry/stackrox/opp-readiness/stackrox-opp-readiness-ref.metadata.jsonci-operator/step-registry/stackrox/opp-readiness/stackrox-opp-readiness-ref.yamlci-operator/step-registry/stackrox/opp-smoke/OWNERSci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.shci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-ref.metadata.jsonci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-ref.yaml
- Remove sensitive URL/namespace values from CI logs - Remove incorrect set -x re-enablement (tracing was never on) - Increase readiness step timeout from 10m to 40m - Add non-root USER 1001 to acs-smoke-runner Dockerfile Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Make stackrox/scanner clone refs overridable via env vars - Pin ubi9/openjdk-17 base image to tag 1.21 - Add SHA-256 checksum verification for oc client download Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh (2)
71-72: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInitialize
TEST_EXITbefore running Gradle.If the job environment already defines
TEST_EXIT, a successful Gradle run preserves that stale value because the||branch is skipped. The step can report failure after a successful smoke test.Initialize
TEST_EXIT=0before Line 71, or use an explicitif/else.Proposed fix
+TEST_EXIT=0 ./gradlew testSMOKE -i --no-daemon || TEST_EXIT=$? -TEST_EXIT="${TEST_EXIT:-0}"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh` around lines 71 - 72, Initialize TEST_EXIT to 0 before the ./gradlew testSMOKE command, so a successful run cannot preserve a stale environment value; retain the existing assignment of the Gradle failure status when the command fails.
77-88: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winKeep artifact collection from changing or hiding the test result.
mkdir -pat Line 86 runs underset -ewithout a guard. An artifact-directory failure can exit the script before it returns the capturedTEST_EXIT. Thecpcommands suppress all errors, so report publication can fail without a warning.Handle
mkdirandcpfailures explicitly, log warnings, and always return the captured test status.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh` around lines 77 - 88, Update the artifact collection block after the smoke test to explicitly handle failures from mkdir and both cp operations, logging warnings when JUnit or HTML report publication fails instead of suppressing errors. Ensure these failures cannot terminate the script under set -e, and preserve the final return of the captured TEST_EXIT status regardless of artifact-collection outcomes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh`:
- Around line 71-72: Initialize TEST_EXIT to 0 before the ./gradlew testSMOKE
command, so a successful run cannot preserve a stale environment value; retain
the existing assignment of the Gradle failure status when the command fails.
- Around line 77-88: Update the artifact collection block after the smoke test
to explicitly handle failures from mkdir and both cp operations, logging
warnings when JUnit or HTML report publication fails instead of suppressing
errors. Ensure these failures cannot terminate the script under set -e, and
preserve the final return of the captured TEST_EXIT status regardless of
artifact-collection outcomes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 9e6271c7-827a-47b7-bb0b-248711493638
📒 Files selected for processing (2)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/pj-rehearse AI-generated. Review for accuracy. |
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
|
@redhat-chai-bot: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
- Declare STACKROX_REF and SCANNER_REF in smoke ref YAML - Guard artifact collection against masking test exit code - Increase memory request to 6Gi with JVM heap cap - Add retry wrapper for sparse git clones
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh`:
- Around line 37-49: Update retry_clone and its callers to remove the known
StackRox and Scanner clone destination directories before each retry attempt,
including the additional clone call sites noted in the review. Ensure cleanup
occurs only before rerunning a failed clone and preserves the existing retry
behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: c9b25810-6c54-47f0-843e-76c3a5e6d696
📒 Files selected for processing (2)
ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.shci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-ref.yaml
| retry_clone() { | ||
| local max_attempts=3 | ||
| local attempt=1 | ||
| while [[ $attempt -le $max_attempts ]]; do | ||
| if "$@"; then | ||
| return 0 | ||
| fi | ||
| echo "[smoke] Clone attempt $attempt/$max_attempts failed, retrying in 10s..." | ||
| sleep 10 | ||
| attempt=$((attempt + 1)) | ||
| done | ||
| echo "[smoke] ERROR: Clone failed after $max_attempts attempts" | ||
| return 1 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Remove failed clone directories before each retry.
If git clone fails after it creates its destination, the next attempt uses the same non-empty directory and fails immediately. Clean the known StackRox and Scanner clone destinations before rerunning each clone command, or clone into a new temporary directory for each attempt.
Also applies to: 55-57, 66-66
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci-operator/step-registry/stackrox/opp-smoke/stackrox-opp-smoke-commands.sh`
around lines 37 - 49, Update retry_clone and its callers to remove the known
StackRox and Scanner clone destination directories before each retry attempt,
including the additional clone call sites noted in the review. Ensure cleanup
occurs only before rerunning a failed clone and preserves the existing retry
behavior.
The RUN command downloaded the OC client tarball as /tmp/oc-client.tar.gz but sha256sum.txt references openshift-client-linux.tar.gz. When sha256sum -c reads from stdin, it looks for a file named openshift-client-linux.tar.gz which does not exist, causing the verification to always fail. Fix by cd-ing into /tmp and using curl -LO to save the file with its original name (openshift-client-linux.tar.gz), matching what sha256sum expects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/pj-rehearse AI-generated. Review for accuracy. |
|
@redhat-chai-bot: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
/pj-rehearse |
|
@amp-rh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@amp-rh: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Superseded by new PR from amp-rh fork with code fixes addressing review feedback. |
2 participants
Summary
Adds ACS (StackRox) SMOKE test integration to the OPP interop test suite,
as part of the INTEROP-8003 epic. This implements Phase 1 — step-registry
refs and OPP config wiring — using the existing stackrox qa-tests-backend
Gradle test suite.
Changes
Phase 1a — Readiness gate (
stackrox-opp-readiness)dynamically via Central/SecuredCluster CRs (
--all-namespaces)/v1/metadata) → securedcluster connected (
/v1/clusters) → sensor pods Running (withOOMKilled detection) → default policies loaded (>80)
ROX_ADMIN_PASSWORDfromcentral-htpasswdsecret(xtrace disabled during credential handling)
$SHARED_DIRPhase 1b — SMOKE test runner (
stackrox-opp-smoke)acs-smoke-runnerimage (UBI9/openjdk-17 + git + oc)stackrox/scanner (proto/scanner)
./gradlew testSMOKE -i --no-daemon$ARTIFACT_DIRregardless oftest exit code
Phase 1c — OPP config wiring
acs-smoke-runnerdockerfile_literal image to the OPP 4.22ci-operator config
interop-opp-awstest chain aftercucushift-installer-check-cluster-health, beforeacm-tests-clc-createJira
INTEROP-8003
/cc @cspi-qe-ocp-lp
Summary by CodeRabbit
stackrox-opp-readinessto validate ACS health, connectivity, sensor pods, policies, and credentials.stackrox-opp-smoketo runqa-tests-backendSMOKE tests and publish JUnit and HTML reports.acs-smoke-runnerimage with UBI OpenJDK 17, Git,oc, andkubectltooling.