release: promocionar develop a main (2e2a5527) — every coverage difference between the engines is registered - #796
Merged
Conversation
…tered per rule, both ways (GT-716 AC3) `68-validate-engine-verdict-parity.mjs` holds the engines to agreement on the rules BOTH decide and prints what only one decides as two counts gated on nothing. ADR-0041 never promised equal reach; this does not ask for it. It asks that every coverage difference be a diff somebody reads. - `73-validate-engine-coverage-parity.mjs`, a sibling of 68 reusing its `deriveOutcomes` precedence: both engines run on the repository root and on a satellite the guard creates with `evolith init` in a temporary directory (`--core` pointed at the root), because the sign flips between the two — here the native engine decides 138 ADR-conformance rules no policy names; there the policies that decide anything read facets nobody supplied. - Every rule decided by exactly one engine is held to `engine-coverage-parity.baseline.json`: per scenario, per direction, per rule, with the reason the OTHER engine gave. Measured first — the class its report states and, for the OPA side, the facets its skip row names — then derived: an id no reachable policy emits, or a policy reading facets `opa-input-builder.ts` never emits (absent on a bare run whatever their provenance: `layers` is observed in nature and supplied in practice, `input.repository` is produced by nothing). A native-side class stated on the OPA side is filed as `opa-gave-no-reason` (the enforcer-routed HXA-01/02/04/05 skip without a class of their own), not as handler debt. - The ratchet closes both ways: an unregistered rule fails, a registered rule both engines now decide fails, an entry whose class changed fails. `--write` regenerates the file for review; `--json` publishes the counts. Measured 2026-09-20 — repository: 220 native-only (164 no-policy-in-bundle, 52 supplied-facet-absent, 4 opa-gave-no-reason) and 8 opa-only (7 unimplemented-native, 1 handler declined); init satellite: 50 native-only (35 / 11 / 4) and 4 opa-only (2 needs-runtime, 1 needs-external-system, 1 handler declined). ≈17 s + ≈3 s, in the `Test` job after 68. Falsifier, both outputs recorded: the criterion's own probe (drop an import from `main.rego`) cannot run since GT-675 — the bundle build refuses a reachable policy nobody imports — so the equivalent is renaming `OBS-EVD-03` in `telemetry-evidence.rego`: the guard went red on both scenarios naming the id (`opaOnly OBS-EVD-03 … no longer coverage-only`), and green once restored. Guard 42 classifies it INSTRUMENTED (89 guards); guard 43 observed it red on the empty fixture (65/65); 11 unit tests. `known-limitations` and the OPA README now say CI registers coverage differences per rule rather than merely allowing them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
…er-rule ratchet now Ticks criterion 3 in both catalogs with the measurement and the recorded falsifier (the criterion's own probe cannot run since GT-675; the equivalent one — renaming OBS-EVD-03 in its policy — turned the guard red on both scenarios naming the id), appends the closure to the board row and adds a `Last Updated` line. Counters unchanged (688 / 715). 08, 01, 04 and 46 green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
…cked artifacts flipped six rules between machines The first CI run of the coverage ratchet disagreed with the laptop that wrote its baseline: EM-Y-01 and QT-01 were decided natively here (a `coverage/` directory from a local jest run) and skipped there; DRIFT-01 was decided here (git history) and skipped on a shallow clone; MCP-01..03 the other way round; and OBS-EVD-01..03 changed class because the CLI's bundled corpus copy predated `facets.json` and classified them by the old defaults. Both scenarios now read the Core from an export of the tracked files (`git ls-files`, local modifications included) plus the compiled `policy.wasm`: no coverage directories, no dists, no `.git`, on every machine alike. A rule whose native verdict needs one of those is skipped identically everywhere, which is the fact the baseline should carry. The export also corrected a measurement this branch had recorded: the 138 ADR-conformance rules were never "decided by native alone" — the working-tree run resolved their decision-record references against the CLI's bundled copy and failed all 138 falsely; against the export they are documentation on both engines, decided by neither. Repository: 82 native-only (52 supplied-facet- absent, 26 no-policy-in-bundle, 4 opa-gave-no-reason) and 7 opa-only; init satellite: 49 and the same 7. Catalog, board and the guard's header say so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
feat(harness): every coverage difference between the engines is registered per rule, both ways (GT-716 AC3)
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes
develop(2e2a5527) tomain: the merge of #795 — GT-716 criterion 3, every coverage difference between the engines registered per rule, both ways.What lands on
main:73-validate-engine-coverage-parity.mjsruns both engines on an export of the tracked tree and on a satellite fresh fromevolith init, and holds every rule only one engine decides toengine-coverage-parity.baseline.json— per scenario, per direction, per rule, with the reason the other engine gave; an unregistered rule, a stale entry or a changed class fails. Measured: repository 82 native-only / 7 opa-only, init satellite 49 / 7.known-limitationsand the OPA README say CI registers coverage differences rather than merely allowing them. Once this lands,pages.ymlrebuildsgh-pagesfrommain.Commits carried:
🤖 Generated with Claude Code