Emit immutable compiled execution capability receipts (capability extension v1)

CHANGELOG.md

@@ -5,6 +5,29 @@ Release notes for `pineforge-codegen`. From 1.0.0 on, versions of codegen and
 supported as exact pairs; on the 0.x line they are independent. See the
 [pairing rule](README.md#engine-pairing).
 
+## Unreleased
+
+### Compiled execution capabilities
+
+- Emit the optional generated-strategy C ABI capability extension:
+  `strategy_capabilities_api_version()` and `strategy_capabilities_receipt()`,
+  versioned by the paired engine's `PF_CAPABILITIES_API_VERSION`. Its immutable
+  canonical JSON uses the checked-settings buffer protocol without changing the
+  base C ABI. Regenerate C++ and relink against the paired next-release engine.
+- Record `strategy()` execution declarations, including positional arguments in
+  Pine signature order, and analyzed request/feed, FX and intrabar requirements.
+  Name nonliteral arguments and unpinned runtime-lowered request sites in
+  `unresolved`; retain every request kind, symbol, timeframe and lookahead.
+  Record endpoint/realtime builtin use even in plots, labels and tables. The
+  receipt proves declarations only, not arbitrary live-versus-batch equivalence.
+- The paired close-only runner refuses intrabar/fill-policy declarations,
+  `process_orders_on_close`, every `request.*` site regardless of timeframe,
+  unsupported clock/feed/FX requirements, `varip`, unresolved declarations and
+  the six endpoint/realtime builtins before its ledger exists, naming the
+  requirement instead of silently changing the computation. Legacy libraries
+  without the extension warn and run, including their requests, without proving
+  eligibility. Default batch computation is unchanged.
+
 ## Release note policy
 
 - Keep a section for each released version, including prereleases. Use the exact

docs/PUBLIC_CONTRACT.md

@@ -1,5 +1,48 @@
 # Public contract for 1.0
 
+## Optional compiled execution capabilities
+
+Availability: **since the next release**, with the paired engine's capability
+extension and receipt-based runner admission policy.
+
+The receipt proves declarations only, not general live-versus-batch equivalence.
+Its `strategy()` positional arguments follow Pine signature order independently
+of batch configuration extraction. Every unresolved argument is named in
+`unresolved`. Runtime-lowered unpinned request sites retain their request kind,
+symbol and timeframe in `requests` with `feed: "unpinned"`, and in `unresolved`.
+The close-only runner retains its conservative refusal of every nonempty
+request array and `process_orders_on_close=true`; confirmed-bar runtime fixes
+and their equivalence tests do not automatically widen that admission policy.
+Legacy libraries without a receipt warn and run, including their requests.
+Realtime
+builtins are conservatively named and refused: generated `barstate.isrealtime`
+is `false`, and `timenow` reads the bar timestamp in both warmup and realtime.
+`barstate.islast`, `barstate.islastconfirmedhistory`, `last_bar_index` and
+`last_bar_time` are also named in `unresolved`: a stream's delivered endpoint
+and last-bar flags differ from the completed batch's final bar.
+All six builtin uses are refused including display-only use (plots, labels,
+tables): the receipt records their use conservatively rather than certifying
+that display-only code cannot influence strategy execution.
+
+Paired development headers defining `PF_CAPABILITIES_API_VERSION` add
+`strategy_capabilities_api_version()` (version 1) and
+`strategy_capabilities_receipt(handle, json, capacity, required, error, error_capacity)`.
+The receipt uses the checked-settings status and buffer protocol: NULL/0 queries
+return `PF_SETTINGS_BUFFER_TOO_SMALL`, `required` includes the NUL, and short
+buffers return no partial JSON. Canonical JSON is emitted from `strategy()`
+declarations and analyzed security/request sites, not guessed from a handle's
+runtime behavior. It is immutable across settings changes and fresh/reused runs.
+
+Version 1 has `version`, `declarations`, `requests`, `requirements` and `unresolved`
+keys. Declarations include calculation cadence, close execution, magnifier,
+standard-OHLC fill mode, limit verification, currency and clock settings. Requests
+name their function, symbol, timeframe, gaps, lookahead, Heikin-Ashi transform and
+feed source. Requirements name historical-only data and intrabar persistence;
+nonliteral contexts remain explicit rather than silently assuming defaults.
+See the paired engine's `docs/strategy-capabilities.md` for the exact schema and
+live-runner refusal policy. No batch dispatch, strategy calculation, matching,
+margin or numeric behavior changes. Old paired headers emit no capability extension.
+
 ## Optional generated settings extension
 
 From 1.1.0 on, C++ compiled against an engine providing

pineforge_codegen/codegen/capabilities.py

@@ -0,0 +1,186 @@
+"""Immutable execution metadata; it does not alter generated computation."""
+
+import json
+import math
+
+from ..ast_nodes import BoolLiteral, FuncCall, Identifier, MemberAccess, NumberLiteral, StrategyDecl, StringLiteral, UnaryOp, VarDecl
+from ..external_requests import CAPABILITY_UNPINNED_ANNOTATION
+from ..limits import iter_ast_nodes
+
+
+STRATEGY_PARAMETERS = (
+    "title", "shorttitle", "overlay", "format", "precision", "scale", "pyramiding",
+    "calc_on_order_fills", "calc_on_every_tick", "max_bars_back",
+    "backtest_fill_limits_assumption", "default_qty_type", "default_qty_value",
+    "initial_capital", "currency", "slippage", "commission_type", "commission_value",
+    "process_orders_on_close", "close_entries_rule", "margin_long", "margin_short",
+    "explicit_plot_zorder", "max_lines_count", "max_labels_count", "max_boxes_count",
+    "calc_bars_count", "risk_free_rate", "use_bar_magnifier", "fill_orders_on_standard_ohlc",
+    "max_polylines_count", "dynamic_requests", "behind_chart", "calc_on_every_history_tick",
+)
+
+
+DECLARATION_DEFAULTS = {
+    "calc_on_every_tick": False,
+    "calc_on_order_fills": False,
+    "process_orders_on_close": False,
+    "use_bar_magnifier": False,
+    "fill_orders_on_standard_ohlc": False,
+    "backtest_fill_limits_assumption": 0,
+    "currency": "currency.NONE",
+    "timeframe": "",
+    "timeframe_gaps": True,
+    "dynamic_requests": True,
+    "calc_on_every_history_tick": False,
+}
+
+
+def _expression(node):
+    if isinstance(node, (StringLiteral, NumberLiteral, BoolLiteral)):
+        return node.value
+    if isinstance(node, Identifier):
+        return node.name
+    if isinstance(node, MemberAccess):
+        return f"{_expression(node.object)}.{node.member}"
+    if isinstance(node, FuncCall):
+        return f"{_expression(node.callee)}({','.join(str(_expression(arg)) for arg in node.args)})"
+    return None
+
+
+def _literal(node):
+    if isinstance(node, (StringLiteral, NumberLiteral, BoolLiteral)):
+        return node.value, True
+    if isinstance(node, UnaryOp) and node.op in ("+", "-"):
+        value, valid = _literal(node.operand)
+        if valid and type(value) in (int, float):
+            return value if node.op == "+" else -value, True
+    if isinstance(node, MemberAccess):
+        value = _expression(node)
+        if value and value.split(".")[0] in ("currency", "format", "scale", "strategy"):
+            return value, True
+    return _expression(node), False
+
+
+def capabilities_document(emitter) -> str:
+    declarations = dict(DECLARATION_DEFAULTS)
+    unresolved = []
+    requests = []
+    lower_symbols = {}
+    intrabar = False
+    recorded = False
+    for node, _depth in iter_ast_nodes(emitter.ctx.ast):
+        if isinstance(node, StrategyDecl):
+            arguments = {STRATEGY_PARAMETERS[index] if index < len(STRATEGY_PARAMETERS)
+                         else f"strategy() positional argument {index + 1}": argument
+                         for index, argument in enumerate(node.args)}
+            unresolved.extend(arguments.keys() & node.kwargs.keys())
+            arguments.update(node.kwargs)
+            for name, argument in arguments.items():
+                value, valid = _literal(argument)
+                default = DECLARATION_DEFAULTS.get(name)
+                if name in DECLARATION_DEFAULTS:
+                    valid &= type(value) is type(default)
+                    if name == "backtest_fill_limits_assumption":
+                        valid = (type(value) is int and value >= 0) or (
+                            type(value) is float and math.isfinite(value) and value >= 0 and value == int(value))
+                        if valid:
+                            value = int(value)
+                    declarations[name] = value
+                if not valid or name not in STRATEGY_PARAMETERS and name not in DECLARATION_DEFAULTS:
+                    unresolved.append(name)
+        elif isinstance(node, VarDecl):
+            intrabar |= node.is_varip
+        elif isinstance(node, FuncCall):
+            name, namespace = emitter._resolve_callee(node.callee)
+            if namespace == "request" and name == "security_lower_tf":
+                expression = node.args[2] if len(node.args) > 2 else node.kwargs.get("expression")
+                lower_symbols[id(expression)] = node.args[0] if node.args else node.kwargs.get("symbol")
+            parts = (node.annotations or {}).get("pf_recorded_key")
+            if parts is not None:
+                recorded = True
+                requests.append({
+                    "function": f"{namespace}.{name}",
+                    "symbol": _expression(node.args[0]) if node.args else None,
+                    "timeframe": "",
+                    "lookahead": f"barmerge.lookahead_{parts.get('lookahead', 'off')}",
+                    "gaps": f"barmerge.gaps_{parts.get('gaps', 'off')}",
+                    "heikinashi": False,
+                    "feed": "recorded",
+                })
+        if isinstance(node, Identifier) and node.name == "timenow":
+            unresolved.append("timenow (bar timestamp approximation)")
+        if isinstance(node, Identifier) and node.name in ("last_bar_index", "last_bar_time"):
+            unresolved.append(f"{node.name} (stream endpoint is not batch-equivalent)")
+        if isinstance(node, MemberAccess) and _expression(node) == "barstate.isrealtime":
+            unresolved.append("barstate.isrealtime (compiled historical-only approximation)")
+        if isinstance(node, MemberAccess) and _expression(node) in (
+                "barstate.islast", "barstate.islastconfirmedhistory"):
+            unresolved.append(f"{_expression(node)} (stream endpoint is not batch-equivalent)")
+
+    auxiliary = False
+    for site in emitter._security_calls:
+        symbol_node = site.get("symbol_node")
+        if site.get("is_lower_tf_array"):
+            symbol_node = lower_symbols.get(id(site.get("expr_node")), symbol_node)
+        if isinstance(symbol_node, FuncCall) and site.get("heikinashi") and symbol_node.args:
+            symbol_node = symbol_node.args[0]
+        symbol = _expression(symbol_node) if symbol_node is not None else "syminfo.tickerid"
+        timeframe = _expression(site.get("tf_node"))
+        lookahead = _expression(site.get("lookahead_node")) or "barmerge.lookahead_off"
+        gaps = _expression(site.get("gaps_node")) or "barmerge.gaps_off"
+        if isinstance(symbol_node, Identifier) and symbol_node.name not in emitter._global_mutable_infos:
+            symbol = _expression((getattr(emitter.ctx, "global_expr_map", {}) or {}).get(
+                symbol_node.name, symbol_node))
+        feed = "chart" if symbol in ("syminfo.tickerid", "syminfo.ticker", "") else "auxiliary"
+        auxiliary |= feed == "auxiliary"
+        if not isinstance(site.get("tf_node"), StringLiteral) and timeframe != "timeframe.period":
+            unresolved.append(f"request.security[{site['sec_id']}].timeframe")
+        if lookahead not in ("barmerge.lookahead_off", "barmerge.lookahead_on"):
+            unresolved.append(f"request.security[{site['sec_id']}].lookahead")
+        requests.append({
+            "function": "request.security_lower_tf" if site.get("is_lower_tf_array") else "request.security",
+            "symbol": symbol,
+            "timeframe": timeframe,
+            "lookahead": lookahead,
+            "gaps": gaps,
+            "heikinashi": bool(site.get("heikinashi")),
+            "feed": feed,
+        })
+    for index, site in enumerate((emitter.ctx.ast.annotations or {}).get(CAPABILITY_UNPINNED_ANNOTATION, ())):
+        requests.append({
+            "function": site["function"],
+            "symbol": _expression(site["symbol_node"]),
+            "timeframe": _expression(site["tf_node"]) if site["function"] in (
+                "request.security", "request.security_lower_tf") else "",
+            "lookahead": _expression(site["lookahead_node"]) or "barmerge.lookahead_off",
+            "gaps": _expression(site["gaps_node"]) or "barmerge.gaps_off",
+            "heikinashi": False,
+            "feed": "unpinned",
+        })
+        unresolved.append(f"{site['function']}[unpinned:{index}]")
+    requirements = {
+        "auxiliary_security_feeds": auxiliary,
+        "native_security_feeds": False,
+        "fx_curve": declarations["currency"] != "currency.NONE",
+        "recorded_series": recorded,
+        "historical_probe_overrides": False,
+        "intrabar_persistence": intrabar,
+    }
+    return json.dumps({"version": 1, "declarations": declarations, "requests": requests,
+                       "requirements": requirements, "unresolved": sorted(set(unresolved))},
+                      sort_keys=True, separators=(",", ":"), allow_nan=False)
+
+
+def emit_capabilities_exports(emitter, lines: list[str]) -> None:
+    literal = json.dumps(capabilities_document(emitter))
+    lines.extend([
+        "#ifdef PF_SETTINGS_API_VERSION",
+        "#ifdef PF_CAPABILITIES_API_VERSION",
+        "    uint32_t strategy_capabilities_api_version(void) { return PF_CAPABILITIES_API_VERSION; }",
+        "    int strategy_capabilities_receipt(void* s, char* json, size_t capacity, size_t* required, char* error, size_t error_capacity) {",
+        "        if (required) *required = 0;",
+        "        return ::pineforge::checked_settings::boundary(error, error_capacity, [&] {",
+        '            ::pineforge::checked_settings::require(s != nullptr, "null strategy");',
+        f"            ::pineforge::checked_settings::receipt({literal}, json, capacity, required);",
+        "        });", "    }", "#endif", "#endif",
+    ])

pineforge_codegen/codegen/emit_top.py

@@ -1998,6 +1998,8 @@ def _emit_extern_c(self, lines: list[str]) -> None:
         lines.append("    }")
         from .checked_settings import emit_settings_exports
         emit_settings_exports(lines)
+        from .capabilities import emit_capabilities_exports
+        emit_capabilities_exports(self, lines)
         if self._declares_bar_magnifier():
             # TradingView runs a script that declares use_bar_magnifier = true
             # on its bar magnifier; the host reads this export to run it on

pineforge_codegen/codegen/helpers.py

@@ -282,6 +282,7 @@ def cpp_code_only(cpp: str) -> str:
     configure_security_evaluators snapshot_script_state restore_script_state
     commit_script_state set_strategy_override set_input
     set_magnifier_volume_weighted strategy_declares_bar_magnifier fill_report run
+    strategy_capabilities_api_version strategy_capabilities_receipt
     precalculate
     strategy_entry strategy_close strategy_close_all strategy_exit
     strategy_exit_cancel_bracket strategy_cancel strategy_cancel_all strategy_order

pineforge_codegen/external_requests.py

@@ -85,6 +85,19 @@
 # RequestRef}``: the run stops there only when no data is installed for the
 # request that carries the same ref (``REQUEST_REF_ANNOTATION``).
 UNPINNED_ANNOTATION = "pf_request_unpinned"
+CAPABILITY_UNPINNED_ANNOTATION = "pf_capability_unpinned_requests"
+
+
+def _record_unpinned_capability(program, request):
+    notes = program.annotations = dict(program.annotations or {})
+    sites = notes.setdefault(CAPABILITY_UNPINNED_ANNOTATION, [])
+    sites.append({
+        "function": f"request.{request.callee.member}",
+        "symbol_node": request.args[0] if request.args else request.kwargs.get("symbol"),
+        "tf_node": request.args[1] if len(request.args) > 1 else request.kwargs.get("timeframe"),
+        "lookahead_node": request.kwargs.get("lookahead"),
+        "gaps_node": request.kwargs.get("gaps"),
+    })
 REQUEST_REF_ANNOTATION = "pf_request_ref"
 # On a request whose symbol can select the chart's or another symbol's: it
 # read the chart before it read a feed, and keeps that lowering where no feed
@@ -785,10 +798,12 @@ def lower_no_data_requests(program: Program) -> Program:
             backed.append(node)
             continue
         if lowering == ABSENT_LOWERING:
+            _record_unpinned_capability(program, node)
             node.annotations = {**node.annotations, UNPINNED_ANNOTATION: unpinned_message(node)}
             continue
         swaps[id(node)] = _na_of(node, funcs)
         if lowering == "unpinned":
+            _record_unpinned_capability(program, node)
             unpinned.append(node)
     if unpinned or backed:
         index = ScriptIndex(program)
@@ -853,6 +868,7 @@ def unpin_requests(program: Program, reasons: dict[int, str]) -> None:
                 "results."))
             continue
         lowered = _na_of(request, funcs)
+        _record_unpinned_capability(program, request)
         marker = (request.annotations or {}).get(UNPINNED_ANNOTATION)
         if isinstance(marker, dict):
             lowered.annotations = {**(lowered.annotations or {}),

tests/_legacy_cpp.py

@@ -1,11 +1,13 @@
-"""Compare frozen lowering bytes without the additive checked-settings ABI."""
+"""Compare frozen lowering bytes without additive settings/capabilities ABIs."""
 import re
 
 
 def legacy_cpp(cpp: str) -> str:
     cpp = cpp.replace(
         '#if __has_include(<pineforge/checked_settings.hpp>)\n'
         '#include <pineforge/checked_settings.hpp>\n#endif\n', '')
+    cpp = re.sub(r'^#ifdef PF_CAPABILITIES_API_VERSION\n.*?^#endif\n', '',
+                 cpp, flags=re.MULTILINE | re.DOTALL)
     cpp = re.sub(r'^#ifdef PF_SETTINGS_API_VERSION\n.*?^#endif\n', '',
                  cpp, flags=re.MULTILINE | re.DOTALL)
     cpp = re.sub(r'^#ifndef PF_SETTINGS_API_VERSION\n.*?^#endif\n', '',