fix: harden runtime scheduling and notifications - #237
Conversation
Co-Authored-By: Codex <noreply@openai.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7acbb38cc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Co-Authored-By: Codex <noreply@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3befc852a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| CLOUD_RUN_REGION: ${{ vars.CLOUD_RUN_REGION }} | ||
| CLOUD_RUN_SERVICE: ${{ vars.CLOUD_RUN_SERVICE }} | ||
| CLOUD_RUN_SERVICES: ${{ vars.CLOUD_RUN_SERVICES }} | ||
| CLOUD_RUN_SERVICE_TARGETS_JSON: ${{ vars.CLOUD_RUN_SERVICE_TARGETS_JSON }} |
There was a problem hiding this comment.
Although the workflow now exports the repository-level EXECUTION_REPORT_GCS_URI, per-service configurations may validly set that value under each target's env (the deployment plan resolves target/default values before the global fallback). When targets write to different buckets or the global variable is unset, this heartbeat scans only the global URI—or falls back to strategy-runs—so valid reports for those targets are never found and the scheduled workflow emits false missing-report alerts. Extract all enabled targets' effective report URIs from CLOUD_RUN_SERVICE_TARGETS_JSON as well.
Useful? React with 👍 / 👎.
| schedule_skip_reason = _heartbeat_skip_reason_for_schedule(since, now) | ||
| if schedule_skip_reason: | ||
| print(f"Execution report heartbeat skipped for {name}: {schedule_skip_reason}") | ||
| runtime_targets = load_runtime_targets(os.environ) |
There was a problem hiding this comment.
When CLOUD_RUN_SERVICE_TARGETS_JSON uses a supported strategy alias, deployment canonicalizes it via resolve_strategy_definition before placing it in RUNTIME_TARGET_JSON, so persisted reports contain the canonical profile. This call leaves the new profile_resolver unset, causing match_payload_target() to compare the original alias against the canonical report value and falsely mark that target missing. Pass the same canonical profile resolver used by the deployment plan when loading heartbeat targets.
Useful? React with 👍 / 👎.
| ( | ||
| f"{alias}-scheduler", | ||
| f"{alias}-probe-scheduler", | ||
| f"{alias}-precheck-scheduler", | ||
| ) |
There was a problem hiding this comment.
In sync-cloud-run-env.yml's non-migrated branch, firstrade-monitor-dispatcher-scheduler remains the active five-minute monitor job, and reconcile_cloud_runtime.py explicitly preserves it until the direct monitor schedulers exist. Restricting the derived pattern to these service-specific main/probe/precheck names filters that dispatcher out, so scheduler-only failures such as authentication or unreachable-URL errors produce no runtime-guard alert while DIRECT_MONITOR_MIGRATION_COMPLETE is not enabled. Include the dispatcher in the derived job set until the migration is confirmed.
Useful? React with 👍 / 👎.
1 participant
Summary
Validation