Fail ocp_testing when any stage was marked UNSTABLE

[tusharjadhav3302]

Summary

Monolithic shiftstack Zuul jobs were reporting SUCCESS (green) when testing stages had failing tests. Test stages already caught failures in rescue blocks, but the playbook exited 0 because nothing consumed the accumulated UNSTABLE state at job level.

This PR adds a test-only unstable flag written by testing-stage rescues. Job failure and yellow Zuul UI are handled by companion changes in ci-framework and ci-framework-jobs (links below).

Changes in this PR

• Add mark_test_stage_unstable.yml — writes ~/artifacts/test_stage_unstable_flag with a warning message.
• Switch testing-stage rescues to use it:
  • openstack_test (includes lb_tests)
  • egressip_tests
  • cpms_test
  • cinder_csi_tests / manila_csi_tests
  • conformance_test
• Add test_stage_unstable_flag to configs/global.yml and clean it up at the start of ocp_testing.yaml.
• No end-of-playbook fail in ocp_testing.yaml — the inner playbook completes all stages; ci-framework fails the outer Zuul job after artifact rsync.
• Soft verification checks and install workarounds continue using stage_unstable_flag only (unchanged, no job fail).

Companion changes (merge together)

Repo	Link	Role
ci-framework	openstack-k8s-operators/ci-framework#4111	Fail shiftstack testconfig when test_stage_unstable_flag exists in rsynced artifacts
ci-framework-jobs	https://gitlab.cee.redhat.com/ci-framework/ci-framework-jobs/-/merge_requests/3556	failure-message: unstable on periodic-osasinfra-shiftstack_ci-osp_verification for yellow Zuul UI

End-to-end flow

Testing stage fails
  → rescue → mark_test_stage_unstable.yml → test_stage_unstable_flag
  → ocp_testing.yaml completes (exit 0 inside pod)
  → ci-framework rsyncs artifacts → sees flag → outer run fails
  → Zuul job with failure-message: unstable → yellow UNSTABLE

Expected job behavior

Scenario	Zuul result
Any testing stage has failing tests	unstable / yellow (with companion PRs)
All testing stages pass	SUCCESS / green
Soft verification only (pods, alerts, etc.)	SUCCESS / green (no test flag, no outer fail)
Hard verification or install failure	Fails mid-playbook (also unstable/yellow on monolithic jobs with failure-message: unstable*)

*Zuul applies one failure-message per job; install/verification hard failures on monolithic jobs also display as unstable when that setting is present.

Test plan

• Land all three PRs/MRs together.
• TP check with shiftstack-qa ref pinned to this branch and a known failing test stage → job unstable/yellow, not green.
• Passing periodic or TP run → green.
• Soft verification issue only (no test failures) → green.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dlaw4608 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tusharjadhav3302]

Test Project MR : https://gitlab.cee.redhat.com/ci-framework/ci-framework-testproject/-/merge_requests/2642