Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 14 additions & 15 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
name: Runtime

on:
workflow_run:
workflows: [CI]
types: [completed]
branches: [main]
workflow_dispatch:
inputs:
validate_only:
Expand All @@ -27,13 +23,6 @@ concurrency:

jobs:
deploy:
if: >
github.event_name == 'workflow_dispatch' ||
(
github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push'
)
runs-on: self-hosted
timeout-minutes: 60
environment: binance-runtime
Expand Down Expand Up @@ -67,8 +56,6 @@ jobs:
fi
- name: 1. Checkout latest code
uses: actions/checkout@v6
with:
ref: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_sha || github.sha }}

- name: 2. Authenticate to Google Cloud
uses: google-github-actions/auth@v3
Expand Down Expand Up @@ -171,6 +158,7 @@ jobs:
BINANCE_API_KEY: ${{ secrets.BINANCE_API_KEY }}
BINANCE_API_SECRET: ${{ secrets.BINANCE_API_SECRET }}
STRATEGY_PROFILE: ${{ vars.STRATEGY_PROFILE || 'crypto_live_pool_rotation' }}
RUNTIME_TARGET_JSON: ${{ vars.RUNTIME_TARGET_JSON }}
TG_TOKEN: ${{ secrets.TG_TOKEN }}
GLOBAL_TELEGRAM_CHAT_ID: ${{ vars.GLOBAL_TELEGRAM_CHAT_ID }}
NOTIFY_LANG: ${{ vars.NOTIFY_LANG }}
Expand Down Expand Up @@ -271,9 +259,20 @@ jobs:
"sha: ${GITHUB_SHA}" \
"actor: ${GITHUB_ACTOR}" \
"url: ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}")
curl -fsS -X POST "https://api.telegram.org/bot${TG_TOKEN}/sendMessage" \
RESPONSE="$(curl -fsS -X POST "https://api.telegram.org/bot${TG_TOKEN}/sendMessage" \
--data-urlencode "chat_id=${GLOBAL_TELEGRAM_CHAT_ID}" \
--data-urlencode "text=${MESSAGE}" >/dev/null
--data-urlencode "text=${MESSAGE}")"
python3 - "${RESPONSE}" <<'PY'
import json
import sys

try:
payload = json.loads(sys.argv[1])
except json.JSONDecodeError as exc:
raise SystemExit("Telegram returned invalid JSON") from exc
if not isinstance(payload, dict) or payload.get("ok") is not True:
raise SystemExit("Telegram did not acknowledge workflow failure notification")
PY
env:
TG_TOKEN: ${{ secrets.TG_TOKEN }}
GLOBAL_TELEGRAM_CHAT_ID: ${{ vars.GLOBAL_TELEGRAM_CHAT_ID }}
9 changes: 8 additions & 1 deletion application/cycle_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from quant_platform_kit.common.runtime_reports import persist_runtime_report
from quant_platform_kit.strategy_lifecycle.performance_monitor import try_record_platform_execution
from runtime_logging import RuntimeLogContext, emit_runtime_log
from runtime_support import finalize_notification_delivery


def execute_strategy_cycle(
Expand Down Expand Up @@ -224,6 +225,7 @@ def execute_strategy_cycle(
pass
finally:
report["log_lines"] = list(log_buffer)
finalize_notification_delivery(report)
try_record_platform_execution(
str(getattr(runtime, "strategy_profile", "") or ""),
{
Expand Down Expand Up @@ -269,10 +271,15 @@ def run_live_cycle(
exit_fn=None,
):
runtime = runtime_builder()
runtime_target = getattr(runtime, "runtime_target", None)
log_context = RuntimeLogContext(
platform="binance",
deploy_target=os.getenv("LOG_DEPLOY_TARGET", "vps"),
service_name=os.getenv("SERVICE_NAME", "binance-platform"),
service_name=(
getattr(runtime_target, "service_name", None)
or os.getenv("SERVICE_NAME")
or "binance-platform"
),
strategy_profile=str(getattr(runtime, "strategy_profile", "") or os.getenv("STRATEGY_PROFILE", "crypto_live_pool_rotation")),
run_id=str(getattr(runtime, "run_id", "") or ""),
extra_fields={
Expand Down
8 changes: 6 additions & 2 deletions docs/operator_runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ The monthly execution pool is locked to the accepted upstream `version` / `as_of
- GitHub Actions no longer owns the hourly cadence for runtime execution in this repo.
- Production cadence should come from one external scheduler, for example VPS cron calling the GitHub Actions dispatch API.
- The VPS dispatch guard retries bounded transient failures such as network errors and GitHub `500`/`502`/`503`/`504`, but still alerts immediately for configuration and permission failures.
- Repository variable changes are consumed by the next externally scheduled dispatch; they do not reconfigure the VPS scheduler cadence.
- Avoid overlapping dispatches from multiple schedulers or from a second manual run while the current runtime job is still in progress.

## Degraded Mode Ladder
Expand Down Expand Up @@ -93,6 +94,7 @@ Use the generic `STRATEGY_ARTIFACT_*` names for crypto strategy artifacts.

Primary settings:

- `RUNTIME_TARGET_JSON`: canonical runtime target written by QuantRuntimeSettings; when present, `STRATEGY_PROFILE` and `BINANCE_DRY_RUN` must match it or the run fails closed
- `STRATEGY_PROFILE`: live profile selector; current supported value is `crypto_live_pool_rotation`
- `STRATEGY_ARTIFACT_FIRESTORE_COLLECTION`: upstream artifact collection, default `strategy`
- `STRATEGY_ARTIFACT_FIRESTORE_DOCUMENT`: upstream artifact document, default `CRYPTO_LIVE_POOL_ROTATION_LIVE_POOL`
Expand Down Expand Up @@ -147,8 +149,10 @@ Operator action:

Expected behavior:

- Telegram send failures should not stop the trading cycle
- The cycle may still finish while alert delivery is degraded
- Telegram transport and response-body acknowledgement are both validated
- A delivery failure does not roll back completed trading actions, but the execution report and workflow are marked failed
- Persisted notification receipts contain only delivery metadata and message hashes, never tokens, chat IDs, or message text
- A failed periodic status delivery does not advance its report bucket, so a later cycle can retry

Operator action:

Expand Down
31 changes: 28 additions & 3 deletions live_services.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import hashlib

import requests

from notify_i18n_support import build_telegram_message, translate as t
Expand Down Expand Up @@ -46,10 +48,33 @@ def save_trade_state(data, *, normalize_fn, collection="strategy", document="MUL


def send_tg_msg(token, chat_id, text):
message = build_telegram_message(text)
receipt = {
"sink": "telegram",
"delivery_status": "failed",
"transport_acknowledged": False,
"compact_text_sha256": hashlib.sha256(message.encode("utf-8")).hexdigest(),
"compact_text_length": len(message),
}
if not token or not chat_id:
return
return {**receipt, "error_type": "missing_target"}
url = f"https://api.telegram.org/bot{token}/sendMessage"
try:
requests.post(url, data={"chat_id": chat_id, "text": build_telegram_message(text)}, timeout=10)
except Exception:
response = requests.post(
url,
data={"chat_id": chat_id, "text": message},
timeout=10,
)
if int(getattr(response, "status_code", 500)) >= 400:
return {**receipt, "error_type": "http_error"}
payload = response.json()
if not isinstance(payload, dict) or payload.get("ok") is not True:
return {**receipt, "error_type": "telegram_rejected"}
return {
**receipt,
"delivery_status": "sent",
"transport_acknowledged": True,
}
except Exception as exc:
print(t("telegram_send_failed"))
return {**receipt, "error_type": type(exc).__name__}
8 changes: 5 additions & 3 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,7 @@ def append_log(log_buffer, message):


def send_tg_msg(token, chat_id, text):
live_send_tg_msg(token, chat_id, text)
return live_send_tg_msg(token, chat_id, text)


def _runtime_error_notification_message(exc):
Expand All @@ -454,11 +454,13 @@ def _notify_runtime_error(exc):
print("Binance runtime error notification skipped: no Telegram target configured.")
return False
try:
send_tg_msg(token, chat_id, _runtime_error_notification_message(exc))
receipt = send_tg_msg(token, chat_id, _runtime_error_notification_message(exc))
except Exception as send_exc:
print(f"Binance runtime error Telegram send failed: {send_exc}")
return False
return True
if isinstance(receipt, dict):
return receipt.get("transport_acknowledged") is True
return receipt is True

# ==========================================
# 2. Earn and balance helpers
Expand Down
10 changes: 7 additions & 3 deletions reporting/status_reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,11 +67,15 @@ def maybe_send_periodic_btc_status_report(
f"{separator}\n"
f"💡 {hint}"
)
if notifier_fn is None:
send_tg_msg_fn(tg_token, tg_chat_id, text)
delivery = send_tg_msg_fn(tg_token, tg_chat_id, text) if notifier_fn is None else notifier_fn(text)
if isinstance(delivery, dict):
acknowledged = delivery.get("transport_acknowledged") is True
else:
notifier_fn(text)
acknowledged = delivery is not False
if not acknowledged:
return False
state["last_btc_status_report_bucket"] = report_bucket
return True


def append_portfolio_report(
Expand Down
61 changes: 56 additions & 5 deletions runtime_config_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
from datetime import datetime, timezone
from typing import Any, Callable

from quant_platform_kit.common.runtime_target import (
RuntimeTarget,
build_runtime_target,
resolve_runtime_target_from_env,
)

from notify_i18n_support import build_strategy_display_name, build_translator, get_notify_lang
from runtime_support import ExecutionRuntime
from strategy_registry import (
Expand Down Expand Up @@ -43,14 +49,12 @@ class CycleExecutionSettings:
strategy_display_name: str
strategy_display_name_localized: str
strategy_domain: str
runtime_target: RuntimeTarget


def load_cycle_execution_settings() -> CycleExecutionSettings:
notify_lang = get_notify_lang()
strategy_definition = resolve_strategy_definition(
os.getenv("STRATEGY_PROFILE"),
platform_id=BINANCE_PLATFORM,
)
runtime_target, strategy_definition = _resolve_runtime_target()
strategy_metadata = resolve_strategy_metadata(
strategy_definition.profile,
platform_id=BINANCE_PLATFORM,
Expand All @@ -70,7 +74,53 @@ def load_cycle_execution_settings() -> CycleExecutionSettings:
strategy_display_name=strategy_metadata.display_name,
strategy_display_name_localized=strategy_display_name_localized,
strategy_domain=strategy_definition.domain,
runtime_target=runtime_target,
)


def _resolve_runtime_target():
raw_runtime_target = str(os.getenv("RUNTIME_TARGET_JSON") or "").strip()
raw_strategy_profile = str(os.getenv("STRATEGY_PROFILE") or "").strip()
if raw_runtime_target:
runtime_target = resolve_runtime_target_from_env(
env=os.environ,
expected_platform_id=BINANCE_PLATFORM,
)
strategy_definition = resolve_strategy_definition(
runtime_target.strategy_profile,
platform_id=BINANCE_PLATFORM,
)
if raw_strategy_profile:
legacy_definition = resolve_strategy_definition(
raw_strategy_profile,
platform_id=BINANCE_PLATFORM,
)
if legacy_definition.profile != strategy_definition.profile:
raise ValueError(
"STRATEGY_PROFILE does not match RUNTIME_TARGET_JSON.strategy_profile"
)
if "BINANCE_DRY_RUN" in os.environ:
legacy_dry_run = get_env_bool("BINANCE_DRY_RUN", default=True)
if legacy_dry_run != runtime_target.dry_run_only:
raise ValueError(
"BINANCE_DRY_RUN does not match RUNTIME_TARGET_JSON.dry_run_only"
)
return runtime_target, strategy_definition

strategy_definition = resolve_strategy_definition(
raw_strategy_profile or None,
platform_id=BINANCE_PLATFORM,
)
runtime_target = build_runtime_target(
platform_id=BINANCE_PLATFORM,
strategy_profile=strategy_definition.profile,
dry_run_only=get_env_bool("BINANCE_DRY_RUN", default=True),
deployment_selector=os.getenv("DEPLOYMENT_SELECTOR") or "default",
account_selector=os.getenv("ACCOUNT_SELECTOR") or "default",
account_scope=os.getenv("ACCOUNT_SCOPE") or "default",
service_name=os.getenv("SERVICE_NAME") or "binance-platform",
)
return runtime_target, strategy_definition


def build_live_runtime(
Expand All @@ -83,7 +133,7 @@ def build_live_runtime(
runtime_now = now_utc or datetime.now(timezone.utc)
cycle_settings = load_cycle_execution_settings()
return ExecutionRuntime(
dry_run=get_env_bool("BINANCE_DRY_RUN", default=True),
dry_run=cycle_settings.runtime_target.dry_run_only,
now_utc=runtime_now,
strategy_profile=cycle_settings.strategy_profile,
strategy_domain=cycle_settings.strategy_domain,
Expand All @@ -96,4 +146,5 @@ def build_live_runtime(
state_loader=state_loader,
state_writer=state_writer,
notifier=notifier,
runtime_target=cycle_settings.runtime_target,
)
Loading