Add opt-in recorded outputs and validate output manifests

CHANGELOG.md

@@ -5,6 +5,33 @@ README's *Releases* section and on the GitHub releases page. From 1.0.0 the
 version number follows semantic versioning over the surfaces the
 [public contract](docs/pages/public-contract.md) lists.
 
+## Unreleased
+
+- **Recorded outputs.** A module can record what it computes on each bar
+  besides its trades: doubles in named slots per bar, doubles kept once per
+  run, and events with an optional message, one row per bar the module
+  publishes (`docs/outputs.md`). A generated module or a C++ host writes them
+  through new protected `BacktestEngine` members (`declare_outputs`,
+  `output_run_begin`, `output_bar`, `output_value`, `output_event`,
+  `output_constant`); a caller reads them through nine new runtime exports of
+  `pineforge.h`, group `pf_outputs` (`strategy_outputs_set_enabled`, the
+  `strategy_outputs_*` readers, the size-prefixed `pf_output_event_v1_t`),
+  present in every module built on this engine, and a recording module adds
+  `strategy_outputs_api_version`, `strategy_outputs_manifest` and
+  `strategy_signal_safety_receipt`. `docker/run_json.py --outputs` writes the
+  record as the report's `outputs` block. Every refusal carries a run-failure
+  code: the new `outputs_rejected` (reasons `not_declared`, `run_in_progress`,
+  `recalculated_after_clear`), the new `outputs_limit` (reasons
+  `too_many_events`, `too_many_rows`), `strategy_library_incompatible` with the
+  new reasons `outputs_api_mismatch` and `outputs_manifest_invalid`, and
+  `engine_invariant` for a broken recorder precondition (all additions to
+  `docker/run_failure_codes.json`). Recording is off until a caller
+  turns it on, and nothing a run computes depends on it: trades, reports,
+  metrics and every hash are byte-identical, `pf_report_t` and every existing
+  symbol are unchanged, and `PF_ABI_VERSION` stays 4. The new members enter
+  `engine_script_run_v19` without an epoch; a C++ object compiled against an
+  earlier v19 header is rebuilt with the library.
+
 ## 1.4.0 — 2026-10-08
 
 **[1.4.0](https://github.com/pineforge-4pass/pineforge-engine/releases/tag/v1.4.0)**,

README.md

@@ -33,7 +33,10 @@ The separate PineForge compiler, [`pineforge-codegen`](https://github.com/pinefo
 - **Open runtime.** The engine and native live runner are Apache-2.0. The separately distributed [PineForge compiler](https://github.com/pineforge-4pass/pineforge-codegen-oss/blob/main/LICENSE) is source-available under the PineForge Source License 1.2 from its release 1.3.0 (release 1.2.0 keeps the PineForge Source License 1.1, and releases up to 1.1.0 keep the PolyForm Noncommercial terms they shipped with); personal trading is free, and commercial use requires a separate license; the capital in an account a proprietary-trading firm or funded-trader program provides or allocates, including a challenge, evaluation or simulated account, is investment capital. Public reference strategies, benchmarks and validation tooling are available in their respective repositories; the community-script test set is not redistributed.
 - **Fast.** In-process, no interpreter: median **36× faster than PyneCore** on the 200 strategies both engines time, measured at engine `35db01c8` on 2026-09-29 on an AWS c7a.8xlarge with PyneCore timed as a subprocess, its interpreter start-up included (PineForge runs a median 807k bars/s per strategy over its 201 slots with the bar magnifier on; [method](benchmarks/results/speed.md)). Parameter sweeps re-run a loaded `.so` with new inputs — no recompile, no fork.
 - **Deterministic to the bit.** Two runs with the same inputs produce identical trade lists. Same on Linux and macOS.
-- **Yours to embed.** 124 `PF_API` declarations across two headers — `pineforge.h`'s 81 (64 runtime exports and up to 17 per-strategy functions a generated module defines) and `native_c_api.h`'s 43 native-host declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
+- **Yours to embed.** 136 `PF_API` declarations across two headers — `pineforge.h`'s 93 (73 runtime exports and up to 20 per-strategy functions a generated module defines) and `native_c_api.h`'s 43 native-host declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
+
+Recorded outputs and their additional API declarations described here are
+unreleased work targeted for **1.5.0**; they are not part of 1.4.0.
 
 **[1.4.0](https://github.com/pineforge-4pass/pineforge-engine/releases/tag/v1.4.0)
 was released on 2026-10-08**, paired with
@@ -501,11 +504,13 @@ API, the script ABI epoch and the pairing with codegen — is the
 [public contract](docs/pages/public-contract.md); what a 0.x user must act on
 is in [CHANGELOG.md](CHANGELOG.md).
 
-A newly generated strategy `.so` exposes 80 compiled-strategy `PF_API` declarations
-(64 runtime implementations plus sixteen generated exports) plus 43 native-host
-declarations: 123 `PF_API` exports in total; a script that declares
-`use_bar_magnifier = true` also exports `strategy_declares_bar_magnifier`, bringing
-the inventories to 81 and 124. Older modules lack the six opt-in
+The counts below include recorded outputs and describe the unreleased API targeted for 1.5.0; recorded outputs are not part of 1.4.0.
+
+A newly generated strategy `.so` exposes 89 compiled-strategy `PF_API` declarations
+(73 runtime implementations plus sixteen generated exports) plus 43 native-host
+declarations: 132 `PF_API` exports in total; `use_bar_magnifier = true` adds
+`strategy_declares_bar_magnifier` and a module that records [outputs](docs/outputs.md) its
+three outputs exports, bringing the inventories to at most 93 and 136. Older modules lack the six opt-in
 [checked-settings exports](docs/checked-settings.md). Older modules can also lack
 the two opt-in [execution-capability exports](docs/strategy-capabilities.md).
 In an optimized build `nm -gU` also shows libc++'s

docker/README.md

@@ -266,6 +266,33 @@ docker run --rm \
   or an index naming no symbol: the key is absent and the report is what it was
   before this variable existed, apart from `elapsed_seconds`.
 
+### Recorded outputs (the harness's `--outputs`)
+
+This section describes unreleased support targeted for **1.5.0**. Released
+1.4.0 images do not provide recorded outputs.
+
+A library compiled to record outputs keeps, besides its trades, values per bar,
+values per run and events with an optional message (`docs/outputs.md` in the
+engine repository). `run_json.py --outputs` turns recording on and writes the
+record as the report's `outputs` key, just before `fingerprint`, with
+`"outputs": true` in `applied_runtime`: the bars' times, each listed series,
+the run constants, each horizontal level's price and every event (sequence,
+output, bar, phase, value, message). Its keys are listed in the engine's
+report-schema page. Without the flag the report is unchanged. On a library that
+records no outputs the run fails before it starts: one line
+`{"engine":"pineforge","error":"--outputs: this library records no outputs (compile the script as an indicator, or with outputs on)","code":"outputs_rejected","args":{"reason":"not_declared"}}`
+on stdout, harness exit 1. With `--bench` the timed runs record too.
+
+Recording preserves the typed provenance rules described below. The runtime
+fact `"outputs": true` remains a boolean in both `applied_runtime` and the
+fingerprinted provenance; input and override normalization runs before the
+fingerprint is built. Recording does not certify a foreign source or replace
+the native settings receipt. Without `--outputs`, no output API is called and
+no `"outputs": false` runtime field is added. A provenance normalization
+failure keeps the existing `fingerprint: null` behavior, including a recorded
+output block when recording succeeded; an invalid output manifest instead
+produces the structured failure line and no partial report.
+
 The engine catches run errors (TF mismatch, unsupported emulation flags,
 unknown-input-TF, etc.) into `strategy_get_last_error()` and, with the
 1.4.0 runtime, its code/args getters. The failure line below describes both

docker/run_failure_codes.json

@@ -249,6 +249,19 @@
     "null admission history event",
     "null strategy",
     "null strategy, key or value",
+    "outputs: bar opened at {open_ms} is before the last recorded bar",
+    "outputs: declare_outputs repeated with other counts",
+    "outputs: declare_outputs while a run is in progress",
+    "outputs: declared counts must not be negative",
+    "outputs: event sequence overflow",
+    "outputs: output {output} is out of range",
+    "outputs: output_bar before any output_run_begin",
+    "outputs: output_constant before any output_run_begin",
+    "outputs: output_event with no open bar",
+    "outputs: output_value with no open bar",
+    "outputs: run constant {index} changed within a run",
+    "outputs: run constant {index} is out of range",
+    "outputs: series slot {slot} is out of range",
     "params_json is reserved; use checked setters",
     "quantity fraction requires a positive finite denominator",
     "quantity grid requires a finite positive step",
@@ -453,6 +466,51 @@
    "retryable": true,
    "since": "1.4.0"
   },
+  "outputs_limit": {
+   "args": {
+    "max": {
+     "kind": "integer"
+    },
+    "reason": {
+     "kind": "vocab",
+     "values": [
+      "too_many_events",
+      "too_many_rows"
+     ]
+    }
+   },
+   "class": "strategy_limit",
+   "description": "A recorded-outputs capacity limit was reached during the run.",
+   "english": [
+    "outputs: the event queue is full",
+    "outputs: the row count is at its limit"
+   ],
+   "retryable": false,
+   "since": "1.5.0"
+  },
+  "outputs_rejected": {
+   "args": {
+    "reason": {
+     "kind": "vocab",
+     "values": [
+      "not_declared",
+      "run_in_progress",
+      "recalculated_after_clear"
+     ]
+    }
+   },
+   "class": "input",
+   "description": "Recorded outputs were asked of a strategy library that records none, or the recording cannot be honoured as asked.",
+   "english": [
+    "--outputs: this library records no outputs (compile the script as an indicator, or with outputs on)",
+    "--outputs: {message}",
+    "outputs: bar {bar} was recalculated after its events were cleared",
+    "outputs: recording cannot change during a run",
+    "outputs: this module declares no outputs"
+   ],
+   "retryable": false,
+   "since": "1.5.0"
+  },
   "pine_array_error": {
    "args": {
     "collection": {
@@ -1117,13 +1175,26 @@
       "abi_mismatch",
       "symbol_missing",
       "setter_missing",
-      "settings_api_mismatch"
+      "settings_api_mismatch",
+      "outputs_api_mismatch",
+      "outputs_manifest_invalid"
      ]
     }
    },
    "class": "engine_fault",
    "description": "The compiled strategy library does not match the harness.",
    "english": [
+    "--outputs: event {index} could not be read",
+    "--outputs: series slot {slot} could not be read",
+    "--outputs: the bar times could not be read",
+    "--outputs: the library lacks {export}, which recording needs; rebuild.",
+    "--outputs: the library returned no outputs manifest: {error}",
+    "--outputs: the library's outputs API version is {version}, the harness reads 1; rebuild.",
+    "--outputs: the outputs manifest is not JSON: {error}",
+    "--outputs: the outputs manifest is not a JSON object",
+    "--outputs: the outputs manifest lists no output {index}, which an event names",
+    "--outputs: the outputs manifest names series slot {slot}, which the library does not record",
+    "--outputs: the outputs manifest's {key} is not a list of entries with {fields}",
     "cannot load the strategy library: {error}",
     "checked settings API mismatch: the strategy library must export strategy_settings_api_version() == 1, strategy_create_checked, strategy_set_input_checked and strategy_set_override_checked, or none of them; rebuild.",
     "pineforge ABI mismatch: .so reports {abi}, harness expects 4; rebuild.",