diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index f14c6c91..09071b88 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -113,6 +113,16 @@ jobs: node --test .harness/scripts/ci/68-validate-engine-verdict-parity.test.mjs node .harness/scripts/ci/68-validate-engine-verdict-parity.mjs --verbose --json + # GT-716 AC3 — what only ONE engine decides is registered per rule, with the + # reason the other engine gave, in both directions and on both scenarios (this + # repository, and a satellite fresh from `init`). 68 gates the verdicts both + # engines reach; this gates the reach itself becoming a diff nobody read. + # Same job for the same reason: it needs dist/main.js and policy.wasm. + - name: Every coverage difference between the engines is registered (GT-716 AC3) + run: | + node --test .harness/scripts/ci/73-validate-engine-coverage-parity.test.mjs + node .harness/scripts/ci/73-validate-engine-coverage-parity.mjs --json + # GT-588, criterion 3 — the transparency rule is RUN, not merely declared. # # `AUD-TRANSP-01…04` and their negative tests existed while diff --git a/.harness/scripts/ci/73-validate-engine-coverage-parity.mjs b/.harness/scripts/ci/73-validate-engine-coverage-parity.mjs new file mode 100644 index 00000000..b6672662 --- /dev/null +++ b/.harness/scripts/ci/73-validate-engine-coverage-parity.mjs @@ -0,0 +1,441 @@ +#!/usr/bin/env node + +/** + * GT-716 AC3 — the coverage difference between the two engines is a per-rule + * ratchet, in both directions, on both scenarios. + * + * ## What 68 leaves on the table + * + * `68-validate-engine-verdict-parity.mjs` holds the engines to agreement on the + * rules BOTH decide, and prints what only one of them decides as `coverageOnly` — + * two counts, gated on nothing. ADR-0041 never promised equal reach, and this guard + * does not ask for it. It asks that every rule one engine decides and the other + * does not be REGISTERED, with the measured reason the other engine gave, so that + * a coverage change is a diff somebody reads rather than a number nobody does. + * + * ## The comparison, stated precisely + * + * For each scenario, each engine runs once (`evolith validate --engine --format + * json`) and every rule id gets one outcome through 68's `deriveOutcomes` (skipped / + * not-applicable / non-executable / errored are "did not decide"; failed and passed + * are "decided"). A rule decided by exactly one engine is a coverage-only rule, and + * its entry carries WHY the other engine did not decide it — the evaluability class + * the report states (`needs-supplied-facts`, `no-policy-in-bundle`, …) and, for the + * OPA side, the facets its policy reads that a bare run does not supply. + * + * Two scenarios, because they do not skip the same rules (GT-716): this repository, + * where native handlers decide the DoD, compliance-baseline, manifesto and taxonomy + * rules whose policies read a context nobody supplied, and a satellite fresh from + * `evolith init`, where almost everything either engine decides is decided by one of + * them. A ratchet on one would let the other drift. + * + * ## The baseline is a ratchet in both directions + * + * `engine-coverage-parity.baseline.json` is written by `--write` and compared by + * default. An id that is coverage-only and not registered fails (a new divergence); + * an id that is registered and is no longer coverage-only fails (a stale entry — the + * handler or policy landed and the entry must go with it); an entry whose reason + * changed class fails (the same id, a different debt). A fix cannot land without its + * entry, and an entry cannot outlive the difference it describes. + * + * ## The tree that is measured is the COMMITTED one + * + * The first CI run of this guard disagreed with the laptop that wrote its baseline: + * EM-Y-01 and QT-01 were decided natively here (a `coverage/` directory from a local + * jest run) and skipped there; DRIFT-01 was decided here (git history) and skipped on + * a shallow clone; MCP-01..03 the other way round. None of that is the corpus — it is + * whatever happens to sit in the working tree. So both scenarios read the Core from an + * EXPORT of the tracked files (`git ls-files`, local modifications included) plus the + * compiled `policy.wasm`: no coverage directories, no dists, no `.git`, on every + * machine alike. A rule whose native verdict needs one of those is skipped identically + * everywhere, which is the fact the baseline should carry. + * + * ## Anti-vacuous pass + * + * Both engine runs of both scenarios go through `assertScannedPerSource`; a missing + * dist, an unbuilt bundle, an export with no corpus or an `init` that produced nothing + * fails loudly instead of reporting "no differences". + * + * Usage: + * node .harness/scripts/ci/73-validate-engine-coverage-parity.mjs + * node .harness/scripts/ci/73-validate-engine-coverage-parity.mjs --verbose + * node .harness/scripts/ci/73-validate-engine-coverage-parity.mjs --json + * node .harness/scripts/ci/73-validate-engine-coverage-parity.mjs --write # regenerate the baseline (review the diff) + * + * Exit codes: + * 0 - every coverage-only rule is registered with its reason, and every entry still holds + * 1 - an unregistered rule, a stale entry, a changed reason, or an engine that produced nothing + */ + +import { copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { execFileSync, spawnSync } from 'node:child_process'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { REPO_ROOT } from '../lib/paths.mjs'; +import { assertScannedPerSource, ZeroCoverageError } from '../lib/coverage.mjs'; +import { facetOfInputPath, readCorpusFacts, readVocabulary } from '../lib/rule-facts.mjs'; +import { deriveOutcomes, outcomeOf } from './68-validate-engine-verdict-parity.mjs'; + +const HERE = dirname(fileURLToPath(import.meta.url)); + +export const BASELINE_PATH = resolve(HERE, 'engine-coverage-parity.baseline.json'); +const CLI_ENTRY = 'src/sdk/cli/dist/main.js'; +const WASM_CANDIDATES = ['src/rulesets/opa/policy.wasm', 'src/sdk/cli/rulesets/opa/policy.wasm']; +const ENGINES = ['native', 'opa']; +const DECIDED = new Set(['passed', 'failed']); +export const SCENARIOS = ['repository', 'init-satellite']; + +/** The evaluability classes a report can state about a rule it did not decide. */ +const CLASS_IN_TEXT = /\b(unimplemented-native|needs-external-system|needs-runtime|needs-supplied-facts|documentation-only|underspecified|no-policy-in-bundle|supplied-facet-absent)\b/; + +/** Classes the NATIVE triage produces; stated on the OPA side they mean the OPA path said nothing of its own. */ +const NATIVE_CLASSES = new Set(['unimplemented-native', 'needs-external-system', 'needs-runtime', 'needs-supplied-facts', 'documentation-only', 'underspecified']); + +/** What closing each kind of entry costs — the follow-up the baseline carries. */ +export const FOLLOW_UP = Object.freeze({ + 'opa-gave-no-reason': 'Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class).', + 'no-policy-in-bundle': 'Author the Rego twin, or record that the rule is native-only.', + 'supplied-facet-absent': 'Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy.', + 'unimplemented-native': 'Write the native handler — the rule declares an observed fact.', + 'needs-supplied-facts': 'The caller supplies the posture; the native engine cannot obtain it.', + 'needs-external-system': 'An adapter over the external system, through the enforcer seam.', + 'needs-runtime': 'An adapter that observes the running system, through the enforcer seam.', + 'documentation-only': 'Author a check or retire the rule; nothing can run it as written.', + underspecified: 'Author the check the rule never got, or drop the blocking flag.', + 'handler-declined': 'The native handler found nothing to judge here; a fixture with the subject would decide it.', + undecided: 'The report states no class for the skip — make the engine say why.', +}); + +/** + * Rule ids decided by exactly one engine, with the OTHER engine's outcome. + * Exported for the unit tests; the precedence is 68's. + */ +export function coverageOnly(nativeOutcomes, opaOutcomes, universe) { + const nativeOnly = []; + const opaOnly = []; + for (const id of [...universe].sort()) { + const n = outcomeOf(nativeOutcomes, id); + const o = outcomeOf(opaOutcomes, id); + if (DECIDED.has(n) && !DECIDED.has(o)) nativeOnly.push({ ruleId: id, other: o }); + else if (DECIDED.has(o) && !DECIDED.has(n)) opaOnly.push({ ruleId: id, other: n }); + } + return { nativeOnly, opaOnly }; +} + +/** The class a report states for a rule it did not decide, or null. */ +export function classFromReport(data, ruleId) { + for (const issue of data?.issues ?? []) { + if (String(issue?.ruleId ?? '') !== ruleId) continue; + const m = CLASS_IN_TEXT.exec(`${issue.title ?? ''} ${issue.description ?? ''} ${issue.message ?? ''}`); + if (m) return m[1]; + } + return null; +} + +/** The facets the OPA evaluator named in its skip message, when the report carries the row. */ +const READS_IN_TEXT = /reads ((?:`input\.[^`]+`(?:, )?)+), and this run supplied/; + +/** + * The facets `opa-input-builder.ts` emits on every run, read from its source: the + * `satellite:` and `core:` blocks plus the two paths. A facet a policy reads that is + * not among them is absent on a bare run whatever its provenance says — GT-694's + * `layers` is observed in nature and supplied in practice, and `repository-taxonomy` + * reads an `input.repository` nothing produces at all. + */ +export function builderEmits(root) { + const src = readFileSync(resolve(root, 'src/packages/core-domain/src/application/validators/evaluators/opa-input-builder.ts'), 'utf8'); + const emitted = new Set(['satellitePath', 'corePath']); + for (const container of ['satellite', 'core']) { + const start = src.indexOf(` ${container}: {`); + if (start < 0) continue; + const end = src.indexOf('\n }', start); + for (const m of src.slice(start, end).matchAll(/^\s{8}(\w+):/gm)) emitted.add(`${container}.${m[1]}`); + } + return emitted; +} + +/** + * The reason the OPA engine did not decide a rule. Measured first — the class and + * facets the report states in its skip row — then derived from the bundle's own + * manifest: an id no reachable policy emits, or a policy that reads facets the input + * builder never emits (absent on a bare run, whatever their provenance). + */ +export function opaReason(ruleId, manifest, corpus, vocabulary, opaReport = null, emitted = null) { + const stated = opaReport ? classFromReport(opaReport, ruleId) : null; + if (stated === 'supplied-facet-absent') { + const row = (opaReport.issues ?? []).find((i) => String(i?.ruleId ?? '') === ruleId); + const m = READS_IN_TEXT.exec(`${row?.description ?? ''} ${row?.message ?? ''}`); + const facets = m ? [...m[1].matchAll(/`input\.([^`]+)`/g)].map((x) => x[1]).sort() : []; + return { class: stated, why: `The report states the policy reads ${facets.join(', ') || 'a facet'} this run did not supply.`, facets }; + } + if (stated && NATIVE_CLASSES.has(stated)) { + // The OPA path returned a skip with no class of its own (an enforcer route that + // failed, a strategy that declined) and the reporter fell back to the + // declaration's class. Say that, rather than file OPA's silence as handler debt. + const row = (opaReport.issues ?? []).find((i) => String(i?.ruleId ?? '') === ruleId); + const said = (row?.description ?? '').replace(/^\[[A-Z ]+\] This rule[^.]*\. /, '').replace(/ A blocking rule that skips.*$/, '').trim(); + return { class: 'opa-gave-no-reason', why: `The OPA path skipped without stating why — the reporter fell back to the declaration's \`${stated}\`; the row says: ${said.slice(0, 160) || '(nothing)'}` }; + } + if (stated) return { class: stated, why: `The report states \`${stated}\`.` }; + if (!manifest.declared.has(ruleId)) { + return { class: 'no-policy-in-bundle', why: 'No reachable policy in the compiled bundle emits this id.' }; + } + const facets = [...new Set((manifest.inputPaths.get(ruleId) ?? []).map(facetOfInputPath).filter(Boolean))]; + const absent = facets + .filter((f) => (emitted ? !emitted.has(f) : vocabulary.get(f)?.provenance && vocabulary.get(f).provenance !== 'observed')) + .sort(); + if (absent.length > 0) { + return { class: 'supplied-facet-absent', why: `The policy reads ${absent.join(', ')}, which the input builder does not emit on a bare run.`, facets: absent }; + } + const declared = corpus.get(ruleId)?.facts ?? []; + return { class: 'undecided', why: `The bundle declares the id and reads ${facets.join(', ') || 'no input'}; declared facts: ${declared.join(', ') || 'none'}.` }; +} + +/** The reason the native engine did not decide a rule: the class it stated, or the declaration's. */ +export function nativeReason(ruleId, data, snapshot, corpus) { + const stated = classFromReport(data, ruleId); + const cls = stated ?? (snapshot[ruleId] === 'native-handler' ? 'handler-declined' : snapshot[ruleId]) ?? 'undecided'; + const facts = corpus.get(ruleId)?.facts ?? []; + return { class: cls, why: `${stated ? 'The report states' : 'The declaration gives'} \`${cls}\`; declared facts: ${facts.join(', ') || 'none'}.` }; +} + +/** + * Compare measured entries against a baseline scenario. + * @returns {{unregistered: object[], stale: object[], changed: object[]}} + */ +export function reconcileCoverage(measured, baselineScenario) { + const out = { unregistered: [], stale: [], changed: [] }; + for (const direction of ['nativeOnly', 'opaOnly']) { + const have = new Map((measured[direction] ?? []).map((e) => [e.ruleId, e])); + const want = new Map(Object.entries(baselineScenario?.[direction] ?? {})); + for (const [id, entry] of have) { + const registered = want.get(id); + if (!registered) out.unregistered.push({ direction, ruleId: id, class: entry.reason.class, why: entry.reason.why }); + else if (registered.class !== entry.reason.class) out.changed.push({ direction, ruleId: id, from: registered.class, to: entry.reason.class }); + } + for (const [id, registered] of want) { + if (!have.has(id)) out.stale.push({ direction, ruleId: id, class: registered.class }); + } + } + return out; +} + +/** Render measured entries in the baseline's shape. */ +export function toBaselineScenario(measured) { + const render = (entries) => + Object.fromEntries(entries.map((e) => [e.ruleId, { class: e.reason.class, why: e.reason.why, followUp: FOLLOW_UP[e.reason.class] ?? FOLLOW_UP.undecided }])); + return { nativeOnly: render(measured.nativeOnly), opaOnly: render(measured.opaOnly) }; +} + +function runCli(args, cwd) { + const proc = spawnSync(process.execPath, [resolve(REPO_ROOT, CLI_ENTRY), ...args], { + cwd, + encoding: 'utf8', + maxBuffer: 64 * 1024 * 1024, + }); + if (proc.error) throw new Error(`could not spawn the CLI (${args.join(' ')}): ${proc.error.message}`); + return proc; +} + +function runEngine(engine, cwd, extra = []) { + const proc = runCli(['validate', '--engine', engine, '--format', 'json', ...extra], cwd); + let parsed; + try { + parsed = JSON.parse(proc.stdout); + } catch { + const tail = String(proc.stdout ?? '').slice(-400) || '(empty stdout)'; + throw new Error(`engine '${engine}' did not emit a JSON report (exit ${proc.status}). Last stdout: ${tail}`); + } + if (!parsed?.data) throw new Error(`engine '${engine}' emitted a report with no \`data\` envelope.`); + return parsed.data; +} + +/** + * The Core as committed: every tracked file (with local modifications), nothing + * untracked, plus the compiled bundle the evaluator needs. See the header. + */ +function exportCore(root) { + const dir = mkdtempSync(join(tmpdir(), 'evolith-coverage-parity-core-')); + const listed = execFileSync('git', ['ls-files', '-z'], { cwd: root, maxBuffer: 256 * 1024 * 1024 }); + const archive = execFileSync('tar', ['-c', '--null', '-T', '-', '-f', '-'], { cwd: root, input: listed, maxBuffer: 1024 * 1024 * 1024 }); + execFileSync('tar', ['-x', '-f', '-', '-C', dir], { input: archive, maxBuffer: 1024 * 1024 * 1024 }); + const wasm = WASM_CANDIDATES.find((r) => existsSync(resolve(root, r))); + for (const rel of WASM_CANDIDATES) { + mkdirSync(dirname(resolve(dir, rel)), { recursive: true }); + copyFileSync(resolve(root, wasm), resolve(dir, rel)); + } + if (!existsSync(join(dir, 'src', 'rulesets', 'schema', 'facets.json'))) { + throw new Error(`the export at ${dir} has no corpus vocabulary — \`git ls-files\` produced an incomplete tree`); + } + return dir; +} + +/** A satellite exactly as `evolith init` leaves it, in a temporary directory. */ +function initSatellite() { + const dir = mkdtempSync(join(tmpdir(), 'evolith-coverage-parity-')); + const proc = runCli(['init', '--name', 'coverage-parity-sat', '--yes'], dir); + if (proc.status !== 0 || !existsSync(join(dir, 'evolith.yaml'))) { + throw new Error(`\`evolith init\` did not produce a satellite in ${dir} (exit ${proc.status}): ${String(proc.stderr ?? '').slice(-300)}`); + } + return dir; +} + +async function readManifest(root) { + const rel = WASM_CANDIDATES.find((r) => existsSync(resolve(root, r))); + const { loadPolicy } = await import('@open-policy-agent/opa-wasm'); + const policy = await loadPolicy(readFileSync(resolve(root, rel))); + const declared = new Set((policy.evaluate({}, 'evolith/manifest/declared_rule_ids')?.[0]?.result ?? []).map(String)); + const raw = policy.evaluate({}, 'evolith/manifest/rule_input_paths')?.[0]?.result ?? {}; + const inputPaths = new Map(Object.entries(raw).map(([id, paths]) => [id, (paths ?? []).map(String)])); + return { declared, inputPaths }; +} + +function preflight(root) { + const missing = []; + if (!existsSync(resolve(root, CLI_ENTRY))) missing.push(`${CLI_ENTRY} (build it: npm run build --workspace src/sdk/cli)`); + if (!WASM_CANDIDATES.some((rel) => existsSync(resolve(root, rel)))) missing.push(`${WASM_CANDIDATES[0]} (build it: npm run build:policy)`); + if (!existsSync(resolve(root, 'src/rulesets/standards/native-evaluability-snapshot.json'))) missing.push('src/rulesets/standards/native-evaluability-snapshot.json'); + return missing; +} + +function measureScenario(name, runs, manifest, snapshot, corpus, vocabulary, emitted) { + const outcomes = Object.fromEntries(ENGINES.map((e) => [e, deriveOutcomes(runs[e])])); + assertScannedPerSource( + { native: outcomes.native.size, opa: outcomes.opa.size }, + { what: `rule outcomes (${name})` }, + ); + const universe = new Set([...outcomes.native.keys(), ...outcomes.opa.keys()]); + const { nativeOnly, opaOnly } = coverageOnly(outcomes.native, outcomes.opa, universe); + return { + nativeOnly: nativeOnly.map((e) => ({ ...e, reason: opaReason(e.ruleId, manifest, corpus, vocabulary, runs.opa, emitted) })), + opaOnly: opaOnly.map((e) => ({ ...e, reason: nativeReason(e.ruleId, runs.native, snapshot, corpus) })), + }; +} + +async function main() { + const argv = process.argv.slice(2); + const verbose = argv.includes('--verbose'); + const asJson = argv.includes('--json'); + const write = argv.includes('--write'); + const root = REPO_ROOT; + + console.log('⚖️ Engine coverage parity — what only one engine decides, per rule, both scenarios (GT-716 AC3)'); + + const missing = preflight(root); + if (missing.length > 0) { + console.error('❌ the two engines cannot both be run, so nothing was compared:'); + for (const m of missing) console.error(` - missing ${m}`); + process.exit(1); + } + + const manifest = await readManifest(root); + const snapshot = JSON.parse(readFileSync(resolve(root, 'src/rulesets/standards/native-evaluability-snapshot.json'), 'utf8')).classes ?? {}; + const corpus = readCorpusFacts(resolve(root, 'src/rulesets'), root); + const vocabulary = readVocabulary(root); + const emitted = builderEmits(root); + + const measured = {}; + let satellite = null; + let core = null; + try { + core = exportCore(root); + for (const scenario of SCENARIOS) { + const runs = {}; + const started = Date.now(); + if (scenario === 'init-satellite') satellite = initSatellite(); + for (const engine of ENGINES) { + runs[engine] = scenario === 'repository' + ? runEngine(engine, core, ['--core', core]) + : runEngine(engine, satellite, ['--core', core]); + } + measured[scenario] = measureScenario(scenario, runs, manifest, snapshot, corpus, vocabulary, emitted); + measured[scenario].durationMs = Date.now() - started; + } + } catch (err) { + if (err instanceof ZeroCoverageError) { + console.error(`❌ ${err.message}`); + process.exit(1); + } + console.error(`❌ ${err.message}`); + process.exit(1); + } finally { + if (satellite) rmSync(satellite, { recursive: true, force: true }); + if (core) rmSync(core, { recursive: true, force: true }); + } + + if (write) { + const baseline = { + $comment: [ + 'GT-716 AC3 — every rule ONE engine decides and the other does not, per scenario, with the reason the other engine gave.', + 'Written by `73-validate-engine-coverage-parity.mjs --write` and compared by default: an unregistered rule, a stale entry or a', + 'changed class fails. ADR-0041 never promised equal coverage; this file makes every coverage difference a diff somebody reads.', + '`why` is measured — the class the report states, the facets the policy reads — and `followUp` says what would REMOVE the entry.', + ], + measuredOn: new Date().toISOString().slice(0, 10), + method: 'evolith validate --engine {native,opa} --format json, on an export of the tracked tree (git ls-files + policy.wasm) and on a satellite fresh from `evolith init` with --core pointed at that export; outcomes per 68-validate-engine-verdict-parity.mjs.', + scenarios: Object.fromEntries(SCENARIOS.map((s) => [s, toBaselineScenario(measured[s])])), + }; + writeFileSync(BASELINE_PATH, JSON.stringify(baseline, null, 2) + '\n'); + for (const s of SCENARIOS) { + console.log(` ${s}: native-only ${measured[s].nativeOnly.length}, opa-only ${measured[s].opaOnly.length} (${measured[s].durationMs} ms)`); + } + console.log(`✓ baseline written to ${BASELINE_PATH.replace(`${root}/`, '')} — review the diff before committing it.`); + return; + } + + const baseline = existsSync(BASELINE_PATH) ? JSON.parse(readFileSync(BASELINE_PATH, 'utf8')) : { scenarios: {} }; + const report = { schemaVersion: '1.0', scenarios: {} }; + let failed = false; + + for (const s of SCENARIOS) { + const { unregistered, stale, changed } = reconcileCoverage(measured[s], baseline.scenarios?.[s]); + report.scenarios[s] = { + nativeOnly: measured[s].nativeOnly.length, + opaOnly: measured[s].opaOnly.length, + unregistered: unregistered.map((e) => e.ruleId), + stale: stale.map((e) => e.ruleId), + changed: changed.map((e) => e.ruleId), + durationMs: measured[s].durationMs, + }; + console.log( + ` ${s}: native-only ${measured[s].nativeOnly.length}, opa-only ${measured[s].opaOnly.length}; ` + + `${unregistered.length} unregistered, ${stale.length} stale, ${changed.length} changed class (${measured[s].durationMs} ms).`, + ); + if (verbose) { + for (const e of measured[s].nativeOnly) console.log(` · native-only ${e.ruleId}: opa ${e.reason.class}`); + for (const e of measured[s].opaOnly) console.log(` · opa-only ${e.ruleId}: native ${e.reason.class}`); + } + if (unregistered.length > 0) { + failed = true; + console.error(`❌ ${s}: ${unregistered.length} rule(s) are decided by ONE engine and not registered:`); + for (const e of unregistered) console.error(` - ${e.direction} ${e.ruleId}: the other engine says ${e.class} — ${e.why}`); + } + if (stale.length > 0) { + failed = true; + console.error(`❌ ${s}: ${stale.length} registered entry(ies) are no longer coverage-only — remove them (or re-run with --write and review):`); + for (const e of stale) console.error(` - ${e.direction} ${e.ruleId} (registered as ${e.class})`); + } + if (changed.length > 0) { + failed = true; + console.error(`❌ ${s}: ${changed.length} entry(ies) changed class — the same id, a different debt:`); + for (const e of changed) console.error(` - ${e.direction} ${e.ruleId}: ${e.from} → ${e.to}`); + } + } + + if (asJson) console.log(`ENGINE_COVERAGE_PARITY ${JSON.stringify(report)}`); + + if (failed) { + console.error(' A coverage difference is legitimate (ADR-0041); an unregistered one is not. Register it with its reason, or fix it.'); + process.exit(1); + } + console.log('✓ 73-validate-engine-coverage-parity: every coverage-only rule is registered with its reason, in both directions, on both scenarios.'); +} + +if (process.argv[1] && resolve(process.argv[1]) === resolve(fileURLToPath(import.meta.url))) { + main().catch((err) => { + console.error(`❌ ${err instanceof Error ? err.stack ?? err.message : String(err)}`); + process.exit(1); + }); +} diff --git a/.harness/scripts/ci/73-validate-engine-coverage-parity.test.mjs b/.harness/scripts/ci/73-validate-engine-coverage-parity.test.mjs new file mode 100644 index 00000000..b098ea60 --- /dev/null +++ b/.harness/scripts/ci/73-validate-engine-coverage-parity.test.mjs @@ -0,0 +1,122 @@ +/** + * GT-716 AC3 — unit tests for the ratchet itself. The live guard needs both engines + * built and an `init`; the properties that decide whether an entry is NEW, STALE or + * CHANGED are asserted here against hand-built outcomes and baselines. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { deriveOutcomes } from './68-validate-engine-verdict-parity.mjs'; +import { + FOLLOW_UP, + classFromReport, + coverageOnly, + nativeReason, + opaReason, + reconcileCoverage, + toBaselineScenario, +} from './73-validate-engine-coverage-parity.mjs'; + +const outcomes = (data) => deriveOutcomes(data); + +test('a rule decided by one engine and skipped by the other is coverage-only, in the right direction', () => { + const native = outcomes({ skippedRuleIds: ['A-01'], issues: [{ ruleId: 'B-01' }] }); // A skipped, B failed + const opa = outcomes({ skippedRuleIds: ['B-01'], issues: [{ ruleId: 'A-01' }] }); // B skipped, A failed + const { nativeOnly, opaOnly } = coverageOnly(native, opa, new Set(['A-01', 'B-01', 'C-01'])); + assert.deepEqual(nativeOnly, [{ ruleId: 'B-01', other: 'skipped' }]); + assert.deepEqual(opaOnly, [{ ruleId: 'A-01', other: 'skipped' }]); + // C-01 is mentioned by neither report: decided and clean on both sides — not coverage-only. +}); + +test('a rule neither engine decides is not coverage-only either — it is a gap, not a difference', () => { + const native = outcomes({ skippedRuleIds: ['X-01'] }); + const opa = outcomes({ nonExecutableRuleIds: ['X-01'] }); + assert.deepEqual(coverageOnly(native, opa, new Set(['X-01'])), { nativeOnly: [], opaOnly: [] }); +}); + +test('a skip that ALSO carries a "did not run" issue stays a skip — 68\'s precedence is reused, not re-implemented', () => { + const native = outcomes({ skippedRuleIds: ['KI-R01'], issues: [{ ruleId: 'KI-R01', title: 'Blocking rule did not run' }] }); + const opa = outcomes({ issues: [{ ruleId: 'KI-R01', description: 'Knowledge candidate must declare a source' }] }); + assert.deepEqual(coverageOnly(native, opa, new Set(['KI-R01'])).opaOnly, [{ ruleId: 'KI-R01', other: 'skipped' }]); +}); + +test('the class a report states for a skip is read from its issue row', () => { + const data = { issues: [{ ruleId: 'MTN-01', description: 'This rule is declared `blocking: true` and was NOT evaluated (needs-supplied-facts). Declared facts: …' }] }; + assert.equal(classFromReport(data, 'MTN-01'), 'needs-supplied-facts'); + assert.equal(classFromReport(data, 'OTHER-01'), null); +}); + +test('the OPA reason comes from the bundle manifest: undeclared id, or facets a bare run does not supply', () => { + const manifest = { + declared: new Set(['GIT-01', 'INH-06']), + inputPaths: new Map([['GIT-01', ['input.satellite.git.branchNameInvalid']], ['INH-06', ['input.satellite.files']]]), + }; + const vocabulary = new Map([['satellite.git', { provenance: 'external' }], ['satellite.files', { provenance: 'observed' }]]); + const corpus = new Map([['INH-06', { facts: ['satellite.files'] }]]); + assert.equal(opaReason('SEC-INJ-01', manifest, corpus, vocabulary).class, 'no-policy-in-bundle'); + const git = opaReason('GIT-01', manifest, corpus, vocabulary); + assert.equal(git.class, 'supplied-facet-absent'); + assert.deepEqual(git.facets, ['satellite.git']); + // declared, reads only observed facets, and still not decided: the guard says so rather than guessing + assert.equal(opaReason('INH-06', manifest, corpus, vocabulary).class, 'undecided'); +}); + +test('the OPA reason prefers what the report states, facets included; the builder\'s emitted set beats provenance as the fallback', () => { + const manifest = { declared: new Set(['HXA-03', 'TAX-01']), inputPaths: new Map([['HXA-03', ['input.satellite.layers.core']], ['TAX-01', ['input.repository.files']]]) }; + const vocabulary = new Map([['satellite.layers', { provenance: 'observed' }], ['repository', { provenance: 'observed' }]]); + const report = { issues: [{ ruleId: 'HXA-03', description: "Not evaluated: the policy deciding 'HXA-03' reads `input.satellite.layers`, and this run supplied no such fact." }] }; + const stated = opaReason('HXA-03', manifest, new Map(), vocabulary, report, new Set(['satellite.files'])); + assert.equal(stated.class, 'supplied-facet-absent'); + assert.deepEqual(stated.facets, ['satellite.layers']); + // no row (a non-blocking skip): `repository` is observed in nature, but the builder never emits it + const inferred = opaReason('TAX-01', manifest, new Map(), vocabulary, { issues: [] }, new Set(['satellite.files'])); + assert.equal(inferred.class, 'supplied-facet-absent'); + assert.deepEqual(inferred.facets, ['repository']); +}); + +test('the native reason prefers what the report states, then the declaration, and names a handler that declined', () => { + const corpus = new Map([['ACL-02', { facts: ['adapter'] }], ['DEP-03', { facts: ['repository'] }]]); + const snapshot = { 'ACL-02': 'needs-supplied-facts', 'DEP-03': 'native-handler' }; + const stated = { issues: [{ ruleId: 'ACL-02', description: 'NOT evaluated (needs-supplied-facts).' }] }; + assert.equal(nativeReason('ACL-02', stated, snapshot, corpus).class, 'needs-supplied-facts'); + assert.equal(nativeReason('ACL-02', { issues: [] }, snapshot, corpus).class, 'needs-supplied-facts'); + assert.equal(nativeReason('DEP-03', { issues: [] }, snapshot, corpus).class, 'handler-declined'); + assert.equal(nativeReason('NEW-01', { issues: [] }, snapshot, corpus).class, 'undecided'); +}); + +test('the ratchet closes both ways, and a changed class is neither new nor stale but wrong', () => { + const measured = { + nativeOnly: [{ ruleId: 'A-01', reason: { class: 'no-policy-in-bundle', why: '' } }, { ruleId: 'N-01', reason: { class: 'no-policy-in-bundle', why: '' } }], + opaOnly: [{ ruleId: 'B-01', reason: { class: 'needs-runtime', why: '' } }], + }; + const baseline = { + nativeOnly: { 'A-01': { class: 'no-policy-in-bundle' }, 'S-01': { class: 'no-policy-in-bundle' } }, + opaOnly: { 'B-01': { class: 'unimplemented-native' } }, + }; + const r = reconcileCoverage(measured, baseline); + assert.deepEqual(r.unregistered.map((e) => e.ruleId), ['N-01']); + assert.deepEqual(r.stale.map((e) => e.ruleId), ['S-01']); + assert.deepEqual(r.changed.map((e) => `${e.ruleId}:${e.from}→${e.to}`), ['B-01:unimplemented-native→needs-runtime']); +}); + +test('an empty baseline is not an excuse — every coverage-only rule is unregistered', () => { + const measured = { nativeOnly: [{ ruleId: 'A-01', reason: { class: 'no-policy-in-bundle', why: '' } }], opaOnly: [] }; + assert.equal(reconcileCoverage(measured, undefined).unregistered.length, 1); +}); + +test('the baseline shape carries class, why and a follow-up per entry', () => { + const rendered = toBaselineScenario({ + nativeOnly: [{ ruleId: 'A-01', reason: { class: 'no-policy-in-bundle', why: 'w' } }], + opaOnly: [{ ruleId: 'B-01', reason: { class: 'mystery', why: 'w' } }], + }); + assert.equal(rendered.nativeOnly['A-01'].followUp, FOLLOW_UP['no-policy-in-bundle']); + assert.equal(rendered.opaOnly['B-01'].followUp, FOLLOW_UP.undecided); +}); + +test('a native-side class stated on the OPA side is OPA giving no reason of its own — filed as such, not as handler debt', () => { + const manifest = { declared: new Set(['HXA-01']), inputPaths: new Map([['HXA-01', ['input.satellite.layers']]]) }; + const report = { issues: [{ ruleId: 'HXA-01', description: "[MUST] This rule is declared `blocking: true` and was NOT evaluated (unimplemented-native). Enforcer 'dependency-cruiser' failed to run: not installed A blocking rule that skips is reported exactly like one that passed." }] }; + const r = opaReason('HXA-01', manifest, new Map(), new Map(), report, new Set()); + assert.equal(r.class, 'opa-gave-no-reason'); + assert.match(r.why, /dependency-cruiser/); + assert.match(FOLLOW_UP['opa-gave-no-reason'], /state why it declined/); +}); diff --git a/.harness/scripts/ci/engine-coverage-parity.baseline.json b/.harness/scripts/ci/engine-coverage-parity.baseline.json new file mode 100644 index 00000000..a337a265 --- /dev/null +++ b/.harness/scripts/ci/engine-coverage-parity.baseline.json @@ -0,0 +1,749 @@ +{ + "$comment": [ + "GT-716 AC3 — every rule ONE engine decides and the other does not, per scenario, with the reason the other engine gave.", + "Written by `73-validate-engine-coverage-parity.mjs --write` and compared by default: an unregistered rule, a stale entry or a", + "changed class fails. ADR-0041 never promised equal coverage; this file makes every coverage difference a diff somebody reads.", + "`why` is measured — the class the report states, the facets the policy reads — and `followUp` says what would REMOVE the entry." + ], + "measuredOn": "2026-09-20", + "method": "evolith validate --engine {native,opa} --format json, on an export of the tracked tree (git ls-files + policy.wasm) and on a satellite fresh from `evolith init` with --core pointed at that export; outcomes per 68-validate-engine-verdict-parity.mjs.", + "scenarios": { + "repository": { + "nativeOnly": { + "ACL-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "ACL-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "ACL-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-02": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-VAL-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CB-VAL-02": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CLI-RR-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.releaseReadiness this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CLI-RR-02": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.releaseReadiness this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CLI-RR-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.releaseReadiness this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CLI-RR-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.releaseReadiness this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "CLI-RR-05": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.releaseReadiness, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-07": { + "class": "supplied-facet-absent", + "why": "The policy reads context, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-09": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-10": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DORA-01": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.scorecards, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-D-01": { + "class": "supplied-facet-absent", + "why": "The policy reads duplicateCodeRatio, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-D-02": { + "class": "supplied-facet-absent", + "why": "The policy reads duplicateConfigCount, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-K-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads maxCyclomaticComplexity this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-K-02": { + "class": "supplied-facet-absent", + "why": "The policy reads prematureAbstractionSignals, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-01": { + "class": "supplied-facet-absent", + "why": "The policy reads classLineCount, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-02": { + "class": "supplied-facet-absent", + "why": "The policy reads openClosedViolations, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads liskovViolations this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-04": { + "class": "supplied-facet-absent", + "why": "The policy reads interfaceSegregationViolations, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads domainImportsInfrastructure this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "GIT-08": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.git this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "HXA-01": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-02": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-04": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-05": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "INH-02": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "MM-R01": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R02": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R04": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R05": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R06": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R07": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R08": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R09": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R10": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R11": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "MM-R12": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "OCB-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.openCore this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "QT-05": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "QT-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads behaviorChangedWithoutDocUpdate this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SEC-RL-01": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SEC-RL-02": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SLSA-AUTH-L2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SLSA-BUILD-L1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SLSA-HOSTED-L2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SLSA-PROV-L1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SPACE-04": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.scorecards, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SPACE-05": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.scorecards, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SSDF-PO.3.1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PS.3.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.4.1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.4.4": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.6.1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.7.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-RV.1.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-RV.1.3": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SVC-01": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SVC-03": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SVC-04": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-02": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-04": { + "class": "supplied-facet-absent", + "why": "The policy reads repository, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-07": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-08": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-09": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-10": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-11": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + } + }, + "opaOnly": { + "MCP-01": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-02": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-03": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-05": { + "class": "handler-declined", + "why": "The declaration gives `handler-declined`; declared facts: core.cli, repository.", + "followUp": "The native handler found nothing to judge here; a fixture with the subject would decide it." + }, + "OBS-EVD-01": { + "class": "needs-runtime", + "why": "The report states `needs-runtime`; declared facts: satellite.packageJson, traces.", + "followUp": "An adapter that observes the running system, through the enforcer seam." + }, + "OBS-EVD-02": { + "class": "needs-runtime", + "why": "The report states `needs-runtime`; declared facts: satellite.packageJson, traces.", + "followUp": "An adapter that observes the running system, through the enforcer seam." + }, + "OBS-EVD-03": { + "class": "needs-external-system", + "why": "The report states `needs-external-system`; declared facts: satellite.packageJson, telemetryBackend.", + "followUp": "An adapter over the external system, through the enforcer seam." + } + } + }, + "init-satellite": { + "nativeOnly": { + "ACL-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "ACL-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "ACL-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads adapter this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-04": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-07": { + "class": "supplied-facet-absent", + "why": "The policy reads context, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-09": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "DOD-10": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads context this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-D-01": { + "class": "supplied-facet-absent", + "why": "The policy reads duplicateCodeRatio, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-D-02": { + "class": "supplied-facet-absent", + "why": "The policy reads duplicateConfigCount, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-K-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads maxCyclomaticComplexity this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-K-02": { + "class": "supplied-facet-absent", + "why": "The policy reads prematureAbstractionSignals, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-01": { + "class": "supplied-facet-absent", + "why": "The policy reads classLineCount, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-02": { + "class": "supplied-facet-absent", + "why": "The policy reads openClosedViolations, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads liskovViolations this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-04": { + "class": "supplied-facet-absent", + "why": "The policy reads interfaceSegregationViolations, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "EM-S-05": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads domainImportsInfrastructure this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "GIT-08": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads satellite.git this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "HXA-01": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-02": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-04": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "HXA-05": { + "class": "opa-gave-no-reason", + "why": "The OPA path skipped without stating why — the reporter fell back to the declaration's `unimplemented-native`; the row says: Enforcer 'dependency-cruiser' failed to run: dependency-cruiser cannot certify these rules: no compiled config was materialized for 6 routed rule(s) (HXA-01, HX", + "followUp": "Make the OPA path state why it declined (an enforcer route that failed, a strategy that returned skipped without a class)." + }, + "INH-02": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "QT-05": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "QT-06": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads behaviorChangedWithoutDocUpdate this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SEC-RL-01": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SEC-RL-02": { + "class": "no-policy-in-bundle", + "why": "The report states `no-policy-in-bundle`.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SLSA-HOSTED-L2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SPACE-04": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.scorecards, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SPACE-05": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.scorecards, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SSDF-PO.3.1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PS.3.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.4.1": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.4.4": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-PW.7.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-RV.1.2": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SSDF-RV.1.3": { + "class": "no-policy-in-bundle", + "why": "No reachable policy in the compiled bundle emits this id.", + "followUp": "Author the Rego twin, or record that the rule is native-only." + }, + "SVC-01": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SVC-03": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "SVC-04": { + "class": "supplied-facet-absent", + "why": "The policy reads satellite.contracts, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-01": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-02": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-03": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-04": { + "class": "supplied-facet-absent", + "why": "The policy reads repository, which the input builder does not emit on a bare run.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-07": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + }, + "TAX-08": { + "class": "supplied-facet-absent", + "why": "The report states the policy reads repository this run did not supply.", + "followUp": "Supply the facet through `facts.satellite` (GT-694), or stop reading it in the policy." + } + }, + "opaOnly": { + "MCP-01": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-02": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-03": { + "class": "unimplemented-native", + "why": "The report states `unimplemented-native`; declared facts: core.evidence, repository.", + "followUp": "Write the native handler — the rule declares an observed fact." + }, + "MCP-05": { + "class": "handler-declined", + "why": "The declaration gives `handler-declined`; declared facts: core.cli, repository.", + "followUp": "The native handler found nothing to judge here; a fixture with the subject would decide it." + }, + "OBS-EVD-01": { + "class": "needs-runtime", + "why": "The report states `needs-runtime`; declared facts: satellite.packageJson, traces.", + "followUp": "An adapter that observes the running system, through the enforcer seam." + }, + "OBS-EVD-02": { + "class": "needs-runtime", + "why": "The report states `needs-runtime`; declared facts: satellite.packageJson, traces.", + "followUp": "An adapter that observes the running system, through the enforcer seam." + }, + "OBS-EVD-03": { + "class": "needs-external-system", + "why": "The report states `needs-external-system`; declared facts: satellite.packageJson, telemetryBackend.", + "followUp": "An adapter over the external system, through the enforcer seam." + } + } + } + } +} diff --git a/docs/known-limitations.es.md b/docs/known-limitations.es.md index 3eda1f42..4c3680e5 100644 --- a/docs/known-limitations.es.md +++ b/docs/known-limitations.es.md @@ -17,7 +17,7 @@ Auditoría completa de nuestras propias afirmaciones, con qué bloquea cada pend | `--engine opa` | 133 de 159 | 26 | | nativo (por defecto) | 41 de 159 | 118 | -CI exige que coincidan sobre **hechos**, no sobre cobertura; eso es por diseño. Que el comando por defecto no lo diga, no lo es ([#628](https://github.com/beyondnetcode/evolith_arch32/issues/628)). Por eso la portada usa `--engine opa` en todas partes. Medido de nuevo el 2026-09-20 con la CLI construida desde este árbol: el motor por defecto decide 56 de las mismas 159, y de las 76 reglas que solo `--engine opa` decide, 73 son veredictos sobre facetas que una ejecución a secas nunca suministra — así que la mayor parte de esa cobertura extra no es cobertura. Se sigue como GT-716 en el [Tablero de Gaps](../reference/core/control-center/gaps/gap-tracking.es.md). Desde `b2840947` (en el árbol, aún no en una CLI publicada) el motor OPA reporta esas reglas como `skipped` con la faceta que le falta: sobre el mismo satélite decide 10 de 159 a partir de lo que una ejecución a secas observa, y el resto solo cuando el llamador suministra los hechos por `facts.satellite`. +CI exige que coincidan sobre **hechos**, no sobre cobertura; eso es por diseño — y desde el AC3 de GT-716 cada regla que solo un motor decide queda registrada por regla, en ambas direcciones, sobre este repositorio y sobre un satélite recién salido de `init` (`73-validate-engine-coverage-parity.mjs`), de modo que una diferencia de cobertura es un diff que alguien lee y no un número que nadie lee. Que el comando por defecto no lo diga, no lo es ([#628](https://github.com/beyondnetcode/evolith_arch32/issues/628)). Por eso la portada usa `--engine opa` en todas partes. Medido de nuevo el 2026-09-20 con la CLI construida desde este árbol: el motor por defecto decide 56 de las mismas 159, y de las 76 reglas que solo `--engine opa` decide, 73 son veredictos sobre facetas que una ejecución a secas nunca suministra — así que la mayor parte de esa cobertura extra no es cobertura. Se sigue como GT-716 en el [Tablero de Gaps](../reference/core/control-center/gaps/gap-tracking.es.md). Desde `b2840947` (en el árbol, aún no en una CLI publicada) el motor OPA reporta esas reglas como `skipped` con la faceta que le falta: sobre el mismo satélite decide 10 de 159 a partir de lo que una ejecución a secas observa, y el resto solo cuando el llamador suministra los hechos por `facts.satellite`. ## Dos reglas de infraestructura no están en ningún denominador diff --git a/docs/known-limitations.md b/docs/known-limitations.md index be586b56..eb9fb506 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -17,7 +17,7 @@ Full audit of our own claims, with what blocks each pending item and who can unb | `--engine opa` | 133 of 159 | 26 | | native (default) | 41 of 159 | 118 | -CI holds them to agreement over **facts**, not over coverage; that part is by design. That the default command never says so is not ([#628](https://github.com/beyondnetcode/evolith_arch32/issues/628)). That is why the front page uses `--engine opa` everywhere. Measured again on 2026-09-20 with the CLI built from this tree: the default decides 56 of the same 159, and of the 76 rules only `--engine opa` decides, 73 are verdicts on facets a bare run never supplies — so most of that extra coverage is not coverage. Tracked as GT-716 in the [Gap Tracking Board](../reference/core/control-center/gaps/gap-tracking.md). Since `b2840947` (in the tree, not yet in a published CLI) the OPA engine reports those rules as `skipped` with the facet it lacks: on the same satellite it decides 10 of 159 from what a bare run observes, and the rest only when the caller supplies the facts through `facts.satellite`. +CI holds them to agreement over **facts**, not over coverage; that part is by design — and since GT-716 AC3 every rule only one engine decides is registered per rule, in both directions, on this repository and on a satellite fresh from `init` (`73-validate-engine-coverage-parity.mjs`), so a coverage difference is a diff somebody reads rather than a number nobody does. That the default command never says so is not ([#628](https://github.com/beyondnetcode/evolith_arch32/issues/628)). That is why the front page uses `--engine opa` everywhere. Measured again on 2026-09-20 with the CLI built from this tree: the default decides 56 of the same 159, and of the 76 rules only `--engine opa` decides, 73 are verdicts on facets a bare run never supplies — so most of that extra coverage is not coverage. Tracked as GT-716 in the [Gap Tracking Board](../reference/core/control-center/gaps/gap-tracking.md). Since `b2840947` (in the tree, not yet in a published CLI) the OPA engine reports those rules as `skipped` with the facet it lacks: on the same satellite it decides 10 of 159 from what a bare run observes, and the rest only when the caller supplies the facts through `facts.satellite`. ## Two infrastructure rules are in no denominator diff --git a/reference/core/control-center/gaps/gap-reference-catalog.es.md b/reference/core/control-center/gaps/gap-reference-catalog.es.md index 620d15e3..d5d8c974 100644 --- a/reference/core/control-center/gaps/gap-reference-catalog.es.md +++ b/reference/core/control-center/gaps/gap-reference-catalog.es.md @@ -10354,7 +10354,7 @@ Los dos se arreglaron de forma estructural y no como correcciones: el rethrow no - **Criterios de aceptación:** - [x] **Una faceta suministrada ausente es `skipped` en OPA, nunca un veredicto.** El manifiesto del bundle lleva, por id de regla, las rutas de input que lee su política (`compile-opa-wasm.mjs` ya parsea el AST de cada política), y `OpaEvaluator` devuelve `skipped` con una clase que nombra la faceta ausente cuando el input no la lleva. **FALSABILIDAD:** sobre el satélite nuevo `--engine opa` pasa de 133 decididas a 60 como máximo, las siete entradas `supplied-facet-absent` de `engine-verdict-parity.baseline.json` se eliminan porque el guard 68 las reporta obsoletas, y suministrar la faceta por el canal de GT-694 sigue devolviendo `MTN-01` — una evaluación real, no una respuesta fija. Ambas salidas registradas. **CUMPLIDO en `b2840947`, y el falsador se movió más lejos de lo que pedía el criterio.** `compile-opa-wasm.mjs` compila un segundo entrypoint de manifiesto, `evolith/manifest/rule_input_paths` (200 de los 238 ids declarados enuncian qué leen; los otros 38 son ids de gate), extraído del AST del compilador por `.harness/scripts/lib/rego-rule-inputs.mjs` — lecturas directas, cabeceras, reglas auxiliares seguidas transitivamente — y `OpaEvaluator` devuelve `skipped` / `supplied-facet-absent` nombrando la faceta cuando el input no lleva nada de lo que la regla lee; `27-opa-parity-gate` hace fallar un bundle que deje de exponerlo. Medido sobre el mismo satélite nuevo: `--engine opa` pasó de **133 decididas a 10**, no a 60 — el «73» que registró esta fila contaba solo las reglas que el nativo saltaba, y 47 de las 57 reglas que ambos motores «decidían» eran también veredictos de OPA sobre input ausente (`DOD-*` sobre `context`, `TAX-*` sobre `repository`, `KI-R*` sobre `knowledge_id`/`review`, `INH-02` sobre `contracts`…); las 10 que quedan leen lo que el builder observa (`INH-06` falla por el `DECISIONS.md` ausente, `OBS-EVD-01/02/03` por los paquetes ausentes — `OBS-EVD-03` lee `packageJson`, no `scorecards` como dice la tabla de arriba; gana el AST). El guard 68 reportó **ocho** entradas obsoletas en la línea base, no siete: las siete `supplied-facet-absent` y `TAX-01`, cuya razón registrada (listas de exención distintas) era errónea — `repository-taxonomy.rego` lee `input.repository.files`, una faceta que ningún canal transporta, así que su `passed` era una lista ausente; las ocho eliminadas, quedan 3 conflictos (`CLI-EXIT-01/03`, `GOV-RULE-NON-EXECUTABLE`), decididas por ambos 28 → 14. Suministrar la faceta por el canal de GT-694 devuelve `MTN-01` `failed` con `applicationFiltering: false` y `passed` con `true` (`opa-supplied-facts.spec.ts`, bundle real). Sobrevive una excepción mantenida a mano, `ABSENCE_IS_A_FACT` (`qualityEvidence`, `evaluationDate`, `qualityAdmissibilityPolicy`, `evidence`, `waiver`, `tenantId`): facetas cuya ausencia ambos motores ya tratan como hecho (`PEA-01..04` pasan sobre una ejecución a secas en nativo y en Rego, ADR-0111) — el AC2 la lleva a los ficheros de regla. `GOV-ENGINE-COVERAGE` ya no le dice al lector que OPA «decide más». - [x] **Una sola declaración de evaluabilidad por regla, y ambos motores derivan de ella.** Cada regla de `*.rules.json` declara los hechos que necesita y su procedencia (`observed` / `supplied` / `runtime` / `external` / `none`); `RULE_TRIAGE` pasa a ser una proyección de ella (o una comprobación contra ella), y el manifiesto del bundle también. Un guard falla cuando un `.rego` lee una faceta que su regla no declara, cuando un handler nativo reclama una regla declarada no ejecutable, y cuando los dos motores clasifican distinto la misma regla. **FALSABILIDAD:** las 12 reglas que hoy el nativo clasifica como no ejecutables y OPA decide (`KI-R01..07`, `INH-03..05`, `PROT-03`, `PROT-06`) lo ponen en rojo hasta que un lado cambie; `PEA-01..04` siguen en verde sin tocarlas. **CUMPLIDO en `3d76f4a6`.** Cada regla de `*.rules.json` (415 de 415) declara `facts` — las facetas que lee su comprobación, por id desde el nuevo vocabulario `src/rulesets/schema/facets.json` (85 facetas, cada una con su procedencia: `observed` / `supplied` / `external` / `runtime`) — y ambos motores derivan de ella: `classifyRule` toma la clase de la procedencia más exigente (`runtime` > `external` > `supplied` > `observed`; `facts: []` es documentación tras un juicio o un placeholder del generador, sin especificar tras nada), `RULE_TRIAGE` ya no existe, y `npm run build:policy` rechaza un bundle cuando una política lee una faceta que su regla no declaró o cuando una entrada del vocabulario no la declara ninguna regla ni la lee ninguna política. Se añadió una clase nativa, `needs-supplied-facts` (una postura que solo los dueños pueden declarar; OPA la decide cuando se suministra, el nativo nunca). **FALSADOR, ambas salidas registradas:** declarar `facts: []` para las doce pone el chequeo del build en rojo con doce hallazgos, uno por regla nombrando las facetas que lee su política (`INH-03 … reads corePath, satellite.contracts, satellitePath which its facts (none) do not declare`); declarar lo que leen las políticas lo devuelve a verde y mueve las doce al denominador ejecutable; `PEA-01..04` siguen `native-handler` con `qualityEvidence` declarado. **Lo que se movió cuando la declaración sustituyó a la tabla — 44 de 415 reglas, sin implementar nada:** `unimplemented-native` 52 → 21 (25 de esas filas las decidían políticas sobre una postura declarada, el sistema de CI o una ejecución de tests, nunca un handler sobre el árbol), `needs-external-system` 20 → 27, `needs-runtime` 17 → 23, `needs-supplied-facts` 0 → 31, `documentation-only` 141 → 138, `underspecified` 14 → 4; las 73 reglas bloqueantes que no corren son las mismas 73, recosteadas (11 handlers, 16 adaptadores, 15 observadores de runtime, 27 posturas declaradas, 4 decisiones de autoría). El guard 68 declaró entonces obsoleto `GOV-RULE-NON-EXECUTABLE` — con las doce ejecutables, nada no ejecutable queda en alcance en este repositorio —, así que la línea base queda en `CLI-EXIT-01/03`. Sobre el satélite nuevo no cambió lo que decide ningún motor (nativo 56, OPA 10); las siete filas `KI-R` hacen fallar ahora la ejecución como `needs-supplied-facts` en vez de esconderse en el conteo de no ejecutables. Snapshot, mapeo ISO 5055 (backlog 21: 9 adoptables, 2 parciales, 10 por escribir) y el README de estándares rederivados. Un juicio queda en código: `ABSENCE_IS_A_FACT`, reducido a las cuatro facetas cuya ausencia ambos motores ya tratan como hecho. - - [ ] **La diferencia de cobertura es un ratchet por regla, en ambas direcciones y en ambos escenarios.** `68-validate-engine-verdict-parity.mjs` (o un hermano que reutilice su `deriveOutcomes`) deja en línea base cada id de `coverageOnly` con su propia razón medida y su seguimiento, sobre la raíz del repositorio Y sobre un satélite producido por `init`; una regla de un solo motor sin registrar falla, y una registrada que ahora deciden ambos falla hasta que se retire su entrada. **FALSABILIDAD:** quitar un `import` de `main.rego` lo pone en rojo nombrando los ids que el bundle dejó de decidir; restaurarlo lo devuelve a verde. + - [x] **La diferencia de cobertura es un ratchet por regla, en ambas direcciones y en ambos escenarios.** `68-validate-engine-verdict-parity.mjs` (o un hermano que reutilice su `deriveOutcomes`) deja en línea base cada id de `coverageOnly` con su propia razón medida y su seguimiento, sobre la raíz del repositorio Y sobre un satélite producido por `init`; una regla de un solo motor sin registrar falla, y una registrada que ahora deciden ambos falla hasta que se retire su entrada. **FALSABILIDAD:** quitar un `import` de `main.rego` lo pone en rojo nombrando los ids que el bundle dejó de decidir; restaurarlo lo devuelve a verde. **CUMPLIDO en `29c8a4ba`.** `73-validate-engine-coverage-parity.mjs` — hermano de 68 que reutiliza su `deriveOutcomes` — corre ambos motores sobre la raíz del repositorio y sobre un satélite que crea con `evolith init` en un directorio temporal, y sujeta cada regla que solo un motor decide a `engine-coverage-parity.baseline.json`: por escenario, por dirección, por regla, con la razón que dio el otro motor — la clase que enuncia su informe y las facetas que nombra su fila de salto cuando la fila existe; el manifiesto del bundle y el conjunto que emite el constructor de input cuando no — y un seguimiento por clase. Una regla sin registrar, una entrada obsoleta o una clase cambiada hacen fallar; `--write` regenera el fichero para revisión. Ambos escenarios leen el Core desde una exportación del árbol versionado más el bundle compilado — la primera corrida en CI discrepó del portátil que escribió la línea base (un directorio `coverage/` local decidía `EM-Y-01`/`QT-01`, el historial git decidía `DRIFT-01`, y la corrida sobre el árbol de trabajo resolvía las referencias de las reglas ADR-conformance contra la copia empaquetada de la CLI y fallaba 138 de ellas falsamente), así que los artefactos no versionados no pueden voltear una regla entre máquinas. Medido el 2026-09-20 sobre esa exportación: repositorio 82 solo-nativo (52 `supplied-facet-absent`, 26 `no-policy-in-bundle`, 4 en que la vía OPA saltó sin una razón propia — las `HXA-01/02/04/05` enrutadas al enforcer, archivadas como `opa-gave-no-reason` y no como deuda de handler) y 7 solo-OPA (`MCP-01..03` `unimplemented-native`, `OBS-EVD-01/02` `needs-runtime`, `OBS-EVD-03` `needs-external-system`, `MCP-05` handler que declinó); satélite `init` 49 solo-nativo (34 / 11 / 4) y las mismas 7 solo-OPA. **FALSADOR, ambas salidas registradas:** la prueba que nombraba este criterio — quitar un `import` de `main.rego` — no puede correr desde GT-675, porque el build del bundle rechaza una política alcanzable que nadie importa; la prueba equivalente es renombrar `OBS-EVD-03` en `telemetry-evidence.rego` para que el bundle deje de decidirla: el guard se puso en rojo en ambos escenarios nombrando el id (`opaOnly OBS-EVD-03 … no longer coverage-only`), y volvió a verde al restaurarla. Corre en el job `Test` tras 68 (≈17 s + ≈3 s); clasificado INSTRUMENTED por el guard 42 y en rojo en el sandbox vacío del guard 43; 11 tests unitarios. `known-limitations` dice ahora que CI registra las diferencias de cobertura por regla en vez de solo permitirlas. - [ ] **El backlog nombrado queda implementado o declarado, nada de él dejado en `coverageOnly` por omisión:** handlers nativos para `MCP-05`, `OBS-EVD-01`, `OBS-EVD-02` (y, sobre el corpus, `MCP-01..04`, `DEP-08`, `TAX-07/08`); un `.rego` para las 11 reglas `no-policy-in-bundle` nombradas arriba; ambos para las 7 que no decide ningún motor; y una decisión registrada para las 138 reglas ADR-conformance y `MM-R*` — un gemelo `.rego` generado, o `documentation-only` en los dos motores. - [ ] **El informe y la página dicen lo mismo.** `GOV-ENGINE-COVERAGE` y `docs/known-limitations.es.md` enuncian la cobertura por motor con el desglose de facetas suministradas, y la portada o deja de necesitar `--engine opa` por razones de cobertura o dice qué razón queda. - **Dependencias:** GT-694 (el canal de facetas suministradas, COMPLETADO), GT-675 (el manifiesto del bundle, COMPLETADO), GT-704 (la línea base de veredictos que esta extiende, COMPLETADO); la costura del enforcer (GT-514) para `satellite.layers`. diff --git a/reference/core/control-center/gaps/gap-reference-catalog.md b/reference/core/control-center/gaps/gap-reference-catalog.md index f85d2af2..b6cc5864 100644 --- a/reference/core/control-center/gaps/gap-reference-catalog.md +++ b/reference/core/control-center/gaps/gap-reference-catalog.md @@ -10447,7 +10447,7 @@ Both were fixed structurally rather than corrected: the rethrow now names BOTH f - **Acceptance criteria:** - [x] **An absent supplied facet is `skipped` on OPA, never a verdict.** The bundle manifest carries, per rule id, the input paths its policy reads (`compile-opa-wasm.mjs` already parses each policy's AST), and `OpaEvaluator` reports `skipped` with a class that names the missing facet when the input does not carry it. **FALSIFIABILITY:** on the fresh satellite `--engine opa` moves from 133 decided to at most 60, the seven `supplied-facet-absent` entries of `engine-verdict-parity.baseline.json` are removed because guard 68 reports them stale, and supplying the facet through GT-694's channel still returns `MTN-01` — a real evaluation, not a fixed answer. Both outputs recorded. **MET in `b2840947`, and the falsifier moved further than the criterion asked.** `compile-opa-wasm.mjs` compiles a second manifest entrypoint, `evolith/manifest/rule_input_paths` (200 of the 238 declared ids state what they read; the 38 others are gate ids), extracted from the compiler's AST by `.harness/scripts/lib/rego-rule-inputs.mjs` — direct reads, heads, helper rules followed transitively — and `OpaEvaluator` reports `skipped` / `supplied-facet-absent` naming the facet when the input carries none of what the rule reads; `27-opa-parity-gate` fails a bundle that stops exposing it. Measured on the same fresh satellite: `--engine opa` went from **133 decided to 10**, not to 60 — the "73" this row registered counted only the rules the native engine skipped, and 47 of the 57 rules both engines "decided" were OPA verdicts on absent input as well (`DOD-*` on `context`, `TAX-*` on `repository`, `KI-R*` on `knowledge_id`/`review`, `INH-02` on `contracts`…); the 10 that remain read what the builder observes (`INH-06` fails on the missing `DECISIONS.md`, `OBS-EVD-01/02/03` on the missing packages — `OBS-EVD-03` reads `packageJson`, not `scorecards` as the table above says; the AST wins). Guard 68 reported **eight** baseline entries stale, not seven: the `supplied-facet-absent` seven and `TAX-01`, whose registered reason (differing exemption lists) was wrong — `repository-taxonomy.rego` reads `input.repository.files`, a facet no channel carries, so its `passed` was an absent list; all eight removed, 3 conflicts remain (`CLI-EXIT-01/03`, `GOV-RULE-NON-EXECUTABLE`), jointly decided 28 → 14. Supplying the facet through GT-694's channel returns `MTN-01` `failed` on `applicationFiltering: false` and `passed` on `true` (`opa-supplied-facts.spec.ts`, real bundle). One hand-kept exemption survives, `ABSENCE_IS_A_FACT` (`qualityEvidence`, `evaluationDate`, `qualityAdmissibilityPolicy`, `evidence`, `waiver`, `tenantId`): facets whose absence both engines already treat as a fact (`PEA-01..04` pass on a bare run natively and in Rego, ADR-0111) — AC2 moves it into the rule files. `GOV-ENGINE-COVERAGE` no longer tells the reader OPA "decides more". - [x] **One evaluability declaration per rule, and both engines derive from it.** Each rule in `*.rules.json` declares the facts it needs and their provenance (`observed` / `supplied` / `runtime` / `external` / `none`); `RULE_TRIAGE` becomes a projection of it (or a check against it), and so does the bundle manifest. A guard fails when a `.rego` reads a facet its rule does not declare, when a native handler claims a rule declared non-executable, and when the two engines class the same rule differently. **FALSIFIABILITY:** the 12 rules native classes non-executable and OPA decides today (`KI-R01..07`, `INH-03..05`, `PROT-03`, `PROT-06`) turn it red until one side changes; `PEA-01..04` stay green untouched. **MET in `3d76f4a6`.** Every rule in `*.rules.json` (415 of 415) declares `facts` — the facets its check reads, by id from the new vocabulary `src/rulesets/schema/facets.json` (85 facets, each with its provenance: `observed` / `supplied` / `external` / `runtime`) — and both engines derive from it: `classifyRule` takes the class of the most demanding provenance (`runtime` > `external` > `supplied` > `observed`; `facts: []` is documentation behind a judgement or a generator placeholder, underspecified behind nothing), `RULE_TRIAGE` is gone, and `npm run build:policy` refuses a bundle when a policy reads a facet its rule did not declare or when a vocabulary entry is declared by no rule and read by no policy. One native class was added, `needs-supplied-facts` (a posture only the owners can declare; OPA decides it when supplied, native never). **FALSIFIER, both outputs recorded:** declaring `facts: []` for the twelve turns the build check red with twelve findings, one per rule naming the facets its policy reads (`INH-03 … reads corePath, satellite.contracts, satellitePath which its facts (none) do not declare`); declaring what the policies read turns it green and moves all twelve into the executable denominator; `PEA-01..04` stay `native-handler` with `qualityEvidence` declared. **What moved when the declaration replaced the table — 44 of 415 rules, nothing implemented:** `unimplemented-native` 52 → 21 (25 of those rows were decided by policies over a declared posture, the CI system or a test run, never by a handler over the tree), `needs-external-system` 20 → 27, `needs-runtime` 17 → 23, `needs-supplied-facts` 0 → 31, `documentation-only` 141 → 138, `underspecified` 14 → 4; the 73 blocking rules that do not run are the same 73, re-costed (11 handlers, 16 adapters, 15 runtime observers, 27 declared postures, 4 authoring decisions). Guard 68 then declared `GOV-RULE-NON-EXECUTABLE` stale — with the twelve executable, nothing non-executable remains in scope on this repository — so the baseline is down to `CLI-EXIT-01/03`. On the fresh satellite nothing changed in what either engine decides (native 56, OPA 10); the seven `KI-R` rows now fail the run as `needs-supplied-facts` instead of hiding inside the non-executable count. Snapshot, ISO 5055 mapping (backlog 21: 9 adoptable, 2 partial, 10 to author) and the standards README re-derived. One judgement stays in code: `ABSENCE_IS_A_FACT`, down to the four facets whose absence both engines already treat as a fact. - - [ ] **The coverage difference is a per-rule ratchet, both directions, both scenarios.** `68-validate-engine-verdict-parity.mjs` (or a sibling reusing its `deriveOutcomes`) baselines every id in `coverageOnly` with its own measured reason and follow-up, over the repository root AND a satellite produced by `init`; an unregistered one-engine rule fails, and a registered one that both engines now decide fails until its entry is removed. **FALSIFIABILITY:** removing one `import` from `main.rego` turns it red naming the ids the bundle stopped deciding; restoring it turns it green. + - [x] **The coverage difference is a per-rule ratchet, both directions, both scenarios.** `68-validate-engine-verdict-parity.mjs` (or a sibling reusing its `deriveOutcomes`) baselines every id in `coverageOnly` with its own measured reason and follow-up, over the repository root AND a satellite produced by `init`; an unregistered one-engine rule fails, and a registered one that both engines now decide fails until its entry is removed. **FALSIFIABILITY:** removing one `import` from `main.rego` turns it red naming the ids the bundle stopped deciding; restoring it turns it green. **MET in `29c8a4ba`.** `73-validate-engine-coverage-parity.mjs` — a sibling of 68 reusing its `deriveOutcomes` — runs both engines on the repository root and on a satellite it creates with `evolith init` in a temporary directory, and holds every rule only one engine decides to `engine-coverage-parity.baseline.json`: per scenario, per direction, per rule, with the reason the other engine gave — the class its report states and the facets its skip row names when the row exists; the bundle manifest and the input builder's emitted set when it does not — and a follow-up per class. An unregistered rule, a stale entry or a changed class fails; `--write` regenerates the file for review. Both scenarios read the Core from an export of the tracked tree plus the compiled bundle — the first CI run disagreed with the laptop that wrote the baseline (a local `coverage/` directory decided `EM-Y-01`/`QT-01`, git history decided `DRIFT-01`, and the working-tree run resolved the ADR-conformance rules' references against the CLI's bundled copy and failed 138 of them falsely), so untracked artifacts cannot flip a rule between machines. Measured 2026-09-20 on that export: repository 82 native-only (52 `supplied-facet-absent`, 26 `no-policy-in-bundle`, 4 where the OPA path skipped without a reason of its own — the enforcer-routed `HXA-01/02/04/05`, filed as `opa-gave-no-reason` rather than as handler debt) and 7 opa-only (`MCP-01..03` `unimplemented-native`, `OBS-EVD-01/02` `needs-runtime`, `OBS-EVD-03` `needs-external-system`, `MCP-05` handler declined); init satellite 49 native-only (34 / 11 / 4) and the same 7 opa-only. **FALSIFIER, both outputs recorded:** the probe this criterion named — removing an import from `main.rego` — cannot run since GT-675, because the bundle build refuses a reachable policy nobody imports; the equivalent probe is renaming `OBS-EVD-03` in `telemetry-evidence.rego` so the bundle stops deciding it: the guard went red on both scenarios naming the id (`opaOnly OBS-EVD-03 … no longer coverage-only`), and green again once restored. Runs in the `Test` job after 68 (≈17 s + ≈3 s); classified INSTRUMENTED by guard 42 and red in guard 43's empty sandbox; 11 unit tests. `known-limitations` now says CI registers coverage differences per rule rather than merely allowing them. - [ ] **The named backlog is implemented or declared, none of it left in `coverageOnly` by omission:** native handlers for `MCP-05`, `OBS-EVD-01`, `OBS-EVD-02` (and, over the corpus, `MCP-01..04`, `DEP-08`, `TAX-07/08`); a `.rego` for the 11 `no-policy-in-bundle` rules named above; both for the 7 neither engine decides; and one recorded decision for the 138 ADR-conformance rules and `MM-R*` — a generated `.rego` twin, or `documentation-only` on both engines. - [ ] **The report and the page say the same thing.** `GOV-ENGINE-COVERAGE` and `docs/known-limitations.md` state coverage per engine with the supplied-facet split, and the front page either stops needing `--engine opa` for coverage reasons or says which reason remains. - **Dependencies:** GT-694 (the supplied-facet channel, DONE), GT-675 (the bundle manifest, DONE), GT-704 (the verdict baseline this extends, DONE); the enforcer seam (GT-514) for `satellite.layers`. diff --git a/reference/core/control-center/gaps/gap-tracking.es.md b/reference/core/control-center/gaps/gap-tracking.es.md index d211f084..fb943427 100644 --- a/reference/core/control-center/gaps/gap-tracking.es.md +++ b/reference/core/control-center/gaps/gap-tracking.es.md @@ -4,6 +4,7 @@ **Estado:** Seguimiento Activo **Responsable:** Evolith Architecture Board +**Última Actualización:** 2026-09-20 (**Aterrizó el AC3 de GT-716: lo que solo un motor decide es una diferencia registrada, no libre.** `29c8a4ba` — `73-validate-engine-coverage-parity.mjs` corre ambos motores sobre este repositorio y sobre un satélite que crea con `evolith init`, y sujeta cada regla de un solo motor a una línea base por escenario, por dirección, por regla, con la razón que dio el otro motor (la clase que enuncia su informe; las facetas que la política lee y una ejecución a secas no suministra). Una regla sin registrar, una entrada obsoleta o una clase cambiada hacen fallar. Ambos escenarios leen el Core desde una exportación del árbol versionado, porque la primera corrida en CI discrepó del portátil que escribió la línea base (un directorio `coverage/` local, el historial git y la copia empaquetada del corpus en la CLI habían decidido seis reglas y fallado 138 reglas ADR-conformance falsamente). Medido sobre esa exportación: repositorio 82 solo-nativo / 7 solo-OPA, satélite `init` 49 / 7 — 52 de las 82 son políticas que leen facetas que nadie suministró, 26 no tienen política alguna, y cuatro son las `HXA-01/02/04/05` enrutadas al enforcer, donde la vía OPA salta sin una razón propia. La prueba del criterio (quitar un `import` de `main.rego`) no puede correr desde GT-675 — el build la rechaza —, así que se registró la equivalente: renombrar `OBS-EVD-03` en su política y el guard se pone en rojo en ambos escenarios nombrando el id. AC4–AC5 abiertos; contadores sin cambio: **688 / 715 completados · 3 en progreso · 3 pendientes · 21 diferidos**.) **Última Actualización:** 2026-09-20 (**Aterrizó el AC2 de GT-716: una declaración de evaluabilidad por regla, y ambos motores derivan de ella.** `3d76f4a6` — cada regla de `*.rules.json` declara `facts`, las facetas que lee su comprobación, desde un vocabulario que dice dónde vive la verdad de cada una (`observed` / `supplied` / `external` / `runtime`); `classifyRule` deriva la clase nativa de la procedencia más exigente y la tabla de triaje por id de regla ya no existe; `npm run build:policy` rechaza un bundle cuya política lea una faceta que su regla no declaró. Declarar `facts: []` para las doce reglas que la tabla llamaba no ejecutables mientras Rego las decidía puso ese chequeo en rojo con doce hallazgos; declarar lo que leen las políticas lo devolvió a verde. **Lo que se movió, sin implementar nada: 44 de 415 reglas cambiaron de clase.** El «backlog de handlers» es 21, no 52 — 25 de esas filas las decidían políticas sobre una postura declarada, el sistema de CI o una ejecución de tests, nunca un handler sobre el árbol — y una nueva clase nativa, `needs-supplied-facts`, recoge las 31 reglas que necesitan una postura que solo los dueños del satélite pueden declarar. Las 73 reglas bloqueantes que no corren son las mismas 73, recosteadas. Snapshot, mapeo ISO 5055 y README de estándares rederivados; la línea base del guard 68 queda en `CLI-EXIT-01/03`. AC3–AC5 abiertos; contadores sin cambio: **688 / 715 completados · 3 en progreso · 3 pendientes · 21 diferidos**.) **Última Actualización:** 2026-09-20 (**Aterrizó el AC1 de GT-716: el motor OPA ya no llama veredicto a un hecho ausente.** `b2840947` — el bundle compila un segundo manifiesto, `rule_input_paths` (por id de regla, las rutas `input.…` que lee su política, desde el AST del compilador), y `OpaEvaluator` devuelve `skipped` / `supplied-facet-absent` nombrando la faceta cuando una ejecución no lleva nada de lo que una regla lee. Medido sobre un satélite recién salido de `init`: `--engine opa` pasó de **133 decididas a 10**. La fila había previsto ≤ 60 a partir de un conteo de 73; el conteo era corto, no el arreglo excesivo — 47 de las 57 reglas que ambos motores «decidían» eran también veredictos de OPA sobre input ausente (`DOD-*` sobre `context`, `TAX-*` sobre `repository`, `INH-02` sobre `contracts`). El guard 68 declaró obsoletos ocho conflictos de la línea base — los siete `supplied-facet-absent` y `TAX-01`, cuyas «listas de exención distintas» eran en realidad ninguna lista (`repository-taxonomy.rego` lee `input.repository.files`, que ningún canal transporta) —, quedando 3 desacuerdos reales sobre 14 reglas decididas por ambos. Suministrar la faceta sigue decidiendo: `MTN-01` falla con `applicationFiltering: false` y pasa con `true`, contra el bundle real. `GOV-ENGINE-COVERAGE` deja de decirle al lector que OPA «decide más». AC2–AC5 abiertos; contadores sin cambio: **688 / 715 completados · 3 en progreso · 3 pendientes · 21 diferidos**.) **Última Actualización:** 2026-09-20 (**Un gap registrado a partir de la pregunta del propietario sobre la tabla de la página de estado real —«cómo logramos la paridad entre los dos motores»—, medido antes de responderla.** `GT-716` → PENDIENTE. La página dice que `--engine opa` decide 133 de 159 donde el motor por defecto decide 41; medido hoy con la CLI construida desde este árbol el motor por defecto decide 56, y las 76 reglas que solo OPA decide se cruzaron una a una contra el cuerpo de la política que las decide: **73 leen una faceta que una ejecución a secas nunca suministra** (`satellite.git`, `.testing`, `.multiTenancy`, `.runtime`, `.protocol`, `.scorecards`, `.contracts`, `.ci`, `.layers`, `adapter`, `context.dod`, `user`, las métricas `QT-*`) y son veredictos sobre input ausente, la misma familia `supplied-facet-absent` que GT-704 dejó en su línea base; tres (`MCP-05`, `OBS-EVD-01/02`) son el backlog real de handlers. Sobre el corpus completo el signo se invierte (nativo 247 / OPA 187) porque 138 reglas ADR-conformance tienen handler y ningún `.rego`, doce reglas son no ejecutables para un motor y decididas por el otro, y el guard 68 imprime todo eso como `coverageOnly` sin bloquear por nada. La fila pide tres cosas, en orden: que una faceta suministrada ausente sea `skipped` también en OPA; una sola declaración de evaluabilidad por regla que genere la tabla de triaje y el manifiesto del bundle; y la diferencia de cobertura como ratchet por regla en ambas direcciones, sobre el repositorio y sobre un satélite de `init`. No se pide cobertura igual (ADR-0041 nunca la prometió) ni `--engine opa` por defecto: su alcance extra sobre un repositorio nuevo en su mayoría no es alcance. Contadores recalculados desde las filas: **688 / 715 completados · 3 en progreso · 3 pendientes · 21 diferidos**.) @@ -31,7 +32,7 @@ Este tablero es la única fuente de verdad para deuda técnica, gaps, oportunida | ID | Gap | En simple | Qué resuelve | Componente | Fase | Criticidad | Complejidad | Estado | |---|---|---|---|:---:|:---:|:---:|:---:|:---:| | [`GT-717`](./gap-reference-catalog.es.md#gt-717) | **Las dos vistas de GitHub Pages se publicaban a mano y llevaban tres meses envejeciendo; nada derivaba sus números del árbol.** Medido el 2026-09-19/20 contra `origin/gh-pages` (`2c8f487a`, tocado por última vez el 2026-07-07) y develop `6491c3bc`: el Atlas desplegado era anterior a la mudanza de taxonomía del 2026-07-04 — 18 de sus 21 enlaces a fuentes respondían 404 desde github.io porque `docHref` construía rutas `../` contra la raíz de Pages —, era monolingüe y tenía 4 escenarios sin explicaciones; el propio `architecture-map.json` del árbol nombraba 20 rutas de docs de las que 10 ya no existían. La vista maestra imprimía `8 controllers`, `26 tools · 9 resources`, `20 commands`, `5 KindEvaluators`, `PWA · Web · Mobile` + `BFF (NestJS · ADR-0075)` para un Tracker que es .NET 10 + React 19, y la cabecera `ADR-0101 · 0074 · 0075 · 0102`; su generador imprimía `12 hexagonal ports · 30 adapters` (línea 149) mientras el SVG versionado decía `11 of 20 ports · 53 adapters` bajo una anotación `` antepuesta a mano que el generador nunca emitió — frente a un árbol que mide 21 interfaces de puerto declaradas (20 ficheros) · 11 en el camino caliente (7 + 4) · 47 clases adaptador · 55 tools · 12 resources · 8 prompts · 38 ficheros de comando · 12 controladores · 33 endpoints · 12 kinds / 7 evaluadores. Ningún workflow referenciaba `gh-pages` (`git grep -l gh-pages 6491c3bc -- .github/workflows` = nada). **CERRADO 2026-09-20** en `a09a1fc7` + `7eac8445`: `.harness/scripts/pages/build-pages.mjs` deriva cada número impreso (`derive-page-metrics.mjs`; puertos y adaptadores por las funciones propias del guard 45), guarda los hechos de fuera del árbol en un `observed-facts.json` fechado, valida el modelo autorado, resuelve 129 `{{placeholders}}` distintos por idioma, rechaza cualquier `../` en el sitio construido, y `--check` más un self-test de 8 pruebas corren en el job requerido `Validate documentation`; el Atlas son 45 nodos · 62 aristas · 11 capítulos · 3 niveles de lectura · EN/ES (197 cadenas de UI), el tour del póster recorre los mismos `chapters[]` sobre regiones emitidas por el generador, y `pages.yml` proyecta `main` sobre `gh-pages` (acciones fijadas por SHA, permiso de escritura solo en el job de publicación). Observado en rojo antes de confiar: mergear develop (release 1.4.0) por debajo hizo fallar `--check` sobre el SVG versionado («generated files are stale»), y `--write-svg` movió las tarjetas de las puertas a 1.4.0 sin teclear un número. **La prosa se puso al día el mismo día en [#787](https://github.com/beyondnetcode/evolith_arch32/pull/787) → [#789](https://github.com/beyondnetcode/evolith_arch32/pull/789):** los conteos que las vistas tecleaban vivían también en `SECURITY.md`, `known-limitations`, la visión maestra, los READMEs de MCP / CLI / OPA / interfaces, el catálogo de tools y la línea base del scorecard (una integración de salida → dos; 47 tools → 55; 26 categorías → 21; 17 puertos → 21; 6 mutativas → 20; 8 contextos requeridos → 10), corregidos contra el árbol con un puntero al inventario generado allí donde un puntero basta; gitleaks se puso en rojo por el camino porque sus huellas llevan número de línea y las filas nuevas las movieron, y `gh-pages` no necesitó reconstruirse — su `metrics.json` ya derivaba los mismos números (una clave distinta: `commit`). | Las dos imágenes públicas del producto tenían tres meses, la mayoría de sus enlaces estaban muertos y cada número se había tecleado a mano. | Las dos páginas se construyen desde el repositorio en cada promoción a `main`, en los dos idiomas; un número que envejece pone en rojo el check requerido, no la imagen. | `Documentation` | Cross | P2 | L | `COMPLETADO` | -| [`GT-716`](./gap-reference-catalog.es.md#gt-716) | **Sobre un `evolith validate` a secas, la brecha de cobertura entre los dos motores es en su mayoría veredictos sobre facetas que nadie suministró, y nada en CI fija esa brecha en ninguna de las dos direcciones.** Medido el 2026-09-20 con la CLI construida desde este árbol (1.4.0, `b3df7e96`) sobre un satélite recién salido de `init`, 159 reglas en alcance: el nativo decide 56 y salta 103; `--engine opa` decide 133 y salta 26. Cruzado regla por regla, 76 reglas ejecutables las decide solo OPA, y el cuerpo de la política de **73** de ellas lee una faceta que una ejecución a secas nunca envía (`input.satellite.{git,runtime,testing,multiTenancy,ci,findings,protocol,scorecards,layers,contracts}`, `input.adapter`, `input.context.dod`, `input.user`, las métricas `QT-*`) — un hecho ausente leído como veredicto, la familia a la que pertenecen seis de los ocho conflictos de veredicto de la ejecución y que la línea base de GT-704 ya registra como `supplied-facet-absent`. Solo `MCP-05`, `OBS-EVD-01` y `OBS-EVD-02` se deciden desde el árbol: ese es el backlog real de handlers. En sentido contrario, el nativo decide 11 reglas para las que el bundle no declara política (`SSDF-*` ×7, `SEC-RL-01/02`, `QT-05`, `SLSA-HOSTED-L2`), 7 no las decide ninguno (`SEC-INJ/PATH/TIMING-*`, `SEC-RL-03`), y 12 que el nativo clasifica como no ejecutables (`KI-R01..07`, `INH-03..05`, `PROT-03/06`) OPA las decide igualmente — dos clasificaciones de la misma regla desde dos fuentes sin relación (`RULE_TRIAGE` y `declared_rule_ids`), que coinciden por construcción solo para `PEA-01..04`. Sobre el corpus completo el signo se invierte — nativo 247 / OPA 187 de 358, porque 138 reglas ADR-conformance y `MM-R*` tienen handler nativo y ningún `.rego` — y `68-validate-engine-verdict-parity.mjs` imprime ambas cifras como `coverageOnly` y no bloquea por ninguna. **AC1 cerrado el 2026-09-20 en `b2840947`:** el bundle enuncia ahora qué lee cada regla (`rule_input_paths`) y OPA salta una regla cuyo hecho la ejecución no suministró; medido sobre el mismo satélite, `--engine opa` pasó de 133 decididas a **10** — las 73 contadas aquí eran solo las reglas que el nativo saltaba; 47 de las decididas por ambos también eran veredictos sobre input ausente — y ocho conflictos de la línea base (los siete `supplied-facet-absent` más `TAX-01`, mal diagnosticado) quedaron obsoletos y se retiraron. **AC2 cerrado el 2026-09-20 en `3d76f4a6`:** cada regla declara `facts` y ambos motores derivan de ello (`classifyRule` desde la procedencia, el build del bundle rechazando lecturas no declaradas); la tabla de triaje ya no existe, 44 reglas cambiaron de clase sin implementar nada — el backlog de handlers es 21, no 52, y 31 reglas necesitan una postura declarada (`needs-supplied-facts`) — y las doce contradicciones están declaradas y son ejecutables. | `--engine opa` parece comprobar más del doble que el motor por defecto, y sobre un repositorio nuevo casi todo ese extra son veredictos sobre hechos que nadie le dio; nada avisaría si la brecha creciera. | Un hecho ausente significa «no evaluado» en los dos motores; una sola declaración por regla alimenta la tabla de handlers, el manifiesto del bundle y el informe; y lo que un motor decide y el otro no es una línea base por regla que CI hace fallar, en ambas direcciones y en ambos escenarios. | `Core Domain` | Cross | P1 | L | `PENDIENTE` | +| [`GT-716`](./gap-reference-catalog.es.md#gt-716) | **Sobre un `evolith validate` a secas, la brecha de cobertura entre los dos motores es en su mayoría veredictos sobre facetas que nadie suministró, y nada en CI fija esa brecha en ninguna de las dos direcciones.** Medido el 2026-09-20 con la CLI construida desde este árbol (1.4.0, `b3df7e96`) sobre un satélite recién salido de `init`, 159 reglas en alcance: el nativo decide 56 y salta 103; `--engine opa` decide 133 y salta 26. Cruzado regla por regla, 76 reglas ejecutables las decide solo OPA, y el cuerpo de la política de **73** de ellas lee una faceta que una ejecución a secas nunca envía (`input.satellite.{git,runtime,testing,multiTenancy,ci,findings,protocol,scorecards,layers,contracts}`, `input.adapter`, `input.context.dod`, `input.user`, las métricas `QT-*`) — un hecho ausente leído como veredicto, la familia a la que pertenecen seis de los ocho conflictos de veredicto de la ejecución y que la línea base de GT-704 ya registra como `supplied-facet-absent`. Solo `MCP-05`, `OBS-EVD-01` y `OBS-EVD-02` se deciden desde el árbol: ese es el backlog real de handlers. En sentido contrario, el nativo decide 11 reglas para las que el bundle no declara política (`SSDF-*` ×7, `SEC-RL-01/02`, `QT-05`, `SLSA-HOSTED-L2`), 7 no las decide ninguno (`SEC-INJ/PATH/TIMING-*`, `SEC-RL-03`), y 12 que el nativo clasifica como no ejecutables (`KI-R01..07`, `INH-03..05`, `PROT-03/06`) OPA las decide igualmente — dos clasificaciones de la misma regla desde dos fuentes sin relación (`RULE_TRIAGE` y `declared_rule_ids`), que coinciden por construcción solo para `PEA-01..04`. Sobre el corpus completo el signo se invierte — nativo 247 / OPA 187 de 358, porque 138 reglas ADR-conformance y `MM-R*` tienen handler nativo y ningún `.rego` — y `68-validate-engine-verdict-parity.mjs` imprime ambas cifras como `coverageOnly` y no bloquea por ninguna. **AC1 cerrado el 2026-09-20 en `b2840947`:** el bundle enuncia ahora qué lee cada regla (`rule_input_paths`) y OPA salta una regla cuyo hecho la ejecución no suministró; medido sobre el mismo satélite, `--engine opa` pasó de 133 decididas a **10** — las 73 contadas aquí eran solo las reglas que el nativo saltaba; 47 de las decididas por ambos también eran veredictos sobre input ausente — y ocho conflictos de la línea base (los siete `supplied-facet-absent` más `TAX-01`, mal diagnosticado) quedaron obsoletos y se retiraron. **AC2 cerrado el 2026-09-20 en `3d76f4a6`:** cada regla declara `facts` y ambos motores derivan de ello (`classifyRule` desde la procedencia, el build del bundle rechazando lecturas no declaradas); la tabla de triaje ya no existe, 44 reglas cambiaron de clase sin implementar nada — el backlog de handlers es 21, no 52, y 31 reglas necesitan una postura declarada (`needs-supplied-facts`) — y las doce contradicciones están declaradas y son ejecutables. **AC3 cerrado el 2026-09-20 en `29c8a4ba`:** `73-validate-engine-coverage-parity.mjs` registra cada regla que solo un motor decide — por escenario (este repositorio, un satélite `init`), por dirección, con la razón del otro motor — en `engine-coverage-parity.baseline.json`; una regla sin registrar, una entrada obsoleta o una clase cambiada hacen fallar. | `--engine opa` parece comprobar más del doble que el motor por defecto, y sobre un repositorio nuevo casi todo ese extra son veredictos sobre hechos que nadie le dio; nada avisaría si la brecha creciera. | Un hecho ausente significa «no evaluado» en los dos motores; una sola declaración por regla alimenta la tabla de handlers, el manifiesto del bundle y el informe; y lo que un motor decide y el otro no es una línea base por regla que CI hace fallar, en ambas direcciones y en ambos escenarios. | `Core Domain` | Cross | P1 | L | `PENDIENTE` | | [`GT-715`](./gap-reference-catalog.es.md#gt-715) | **La Core API rechazaba cualquier contexto de evaluación inline de más de 100 KB con un 500 enmascarado y sin línea de log, así que la llamada de conformidad del repositorio del Tracker nunca funcionó contra un repositorio real.** Medido el 2026-09-20 desde el entorno UAT del Tracker: `POST /products/{id}/evaluate-architecture` sobre un producto que apunta a este repositorio leyó 150 ficheros (~1 MB) de GitHub y se los envió al Core, que respondió `500 INTERNAL_ERROR "An unexpected error occurred"`; el Tracker registró un `synthetic BLOCKED`. Reproducido sobre la misma imagen (`main@142b8324`): 14 ficheros / cuerpo de 99.797 bytes → `200`, 15 ficheros / 101.578 bytes → `500`; un cuerpo sintético de 92.956 bytes → `200`, de 126.556 → `500`. El límite de 100 KB por omisión de Express para json, lanzado como un `PayloadTooLargeError` que no es `HttpException`, clasificado por mensaje, enmascarado para el cable y escrito en ninguna parte. **CERRADO 2026-09-20** en `3b276c9a`: `EVOLITH_MAX_BODY_BYTES` (2 MiB por defecto) registra los parsers explícitamente, los errores del body parser conservan el estado del parser y el 413 nombra los dos tamaños y la variable, y todo 5xx enmascarado se registra con su traza. Rojo primero: 4 de las 5 specs nuevas fallan sobre el filtro viejo; después: el mismo cuerpo de 1,1 MB → `200` con el veredicto de `gate-f1`, 3 MB → `413`. Promovido en #778 (`9c5deedf`) y redesplegado por el job `Deploy UAT (Coolify)` de la corrida 35490911533 el 2026-09-20; medido justo después: la misma llamada `evaluate-architecture` responde `200`, `provenance: core`, `status: COMPLETED`, `resultDecision: FAILED` — un veredicto real sobre 150 ficheros (gates f1–f5 fallidos por artefactos de fase ausentes), 174 ms en el Core. La portada conserva la captura de la compuerta de fase. | El Tracker no podía obtener un veredicto de arquitectura sobre ningún repositorio real: el Core rechazaba la petición por su tamaño y no le decía nada útil a nadie. | El Core evalúa un repositorio real enviado inline, rechaza con una razón que nombra cuando debe, y deja una traza que el operador puede leer. | `Core API` | Cross | P1 | S | `COMPLETADO` | | [`GT-714`](./gap-reference-catalog.es.md#gt-714) | **`gate evaluate` y `phase advance` en la CLI publicada necesitan un checkout de este repositorio en disco, porque el tarball trae las reglas pero no las definiciones de gate.** Medido el 2026-09-20 con `@beyondnet/evolith-cli@1.3.2` en un contenedor `node:20` limpio sobre un satélite recién salido de `init`: sin `--core` las dos órdenes salen con `1` y `ENOENT … reference/governance/sdlc/gates` (el tarball trae `rulesets/sdlc/phase-gates.rules.json` y el registro de artefactos en las rutas propias del paquete, pero el validador compone `/reference/governance/sdlc/gates` y `/src/rulesets/sdlc/artifact-registry.json`, y `findCorePath` cae en el propio satélite); con `--core ../evolith` las dos salen con `2` y el veredicto real del gate. GT-705 arregló el mismo defecto para el paquete MCP empaquetando los dos árboles e instalando un único resolutor; el paquete de la CLI no entró en ese cambio, y `sdlc gate-status` (sub-hallazgo de GT-461) no tiene `--core` en absoluto. Hasta que aterrice, la portada muestra las dos órdenes con `--core ../evolith` y lo explica en el pie. | Las dos órdenes que hacen de Evolith algo más que un linter no corren desde el paquete publicado sin un clon de este repositorio junto al proyecto. | `npx -y @beyondnet/evolith-cli gate evaluate --phase discovery` sobre un satélite nuevo da el veredicto, sin opción, y la portada pierde su salvedad. | `Evolith CLI` | Cross | P1 | S | `PENDIENTE` | | [`GT-713`](./gap-reference-catalog.es.md#gt-713) | **El análisis al que están ligadas las alertas de la pestaña Security solo lo produce la corrida `push` de `sdk-cli-ci.yml`, y su filtro de rutas saltaba casi todo el código.** El job `CodeQL SAST` es el único que sube el análisis `/language:javascript-typescript` para `refs/heads/main`; las corridas de pull request son diff-informed (se recortan al diff y nunca mueven las alertas de la rama) y la configuración por defecto "Code Quality" es otra suite. El trigger `push` estaba filtrado a `src/sdk/cli/**`, `.harness/**` y los lockfiles. Medido el 2026-09-19: la promoción `19d736da` (cambios solo bajo `src/packages` y `src/apps`) llegó a `main` sin análisis alguno, así que la pestaña siguió mostrando 10 alertas sobre código que ya no existía; `c5547114` hizo lo mismo 40 minutos después. Las dos necesitaron `gh workflow run sdk-cli-ci.yml --ref main` a mano. **CERRADO 2026-09-19** en `72aceb70`: el filtro cubre ahora `src/packages/**` y `src/apps/**` — todo lo que CodeQL escanea — y un push solo de documentación sigue sin gastar la corrida. | El escáner que decide qué muestra la pestaña Security no volvía a correr cuando cambiaba la mayor parte del código, así que la pestaña describía el commit anterior. | Una promoción de código a `main` re-analiza `main`; la pestaña está al día sin que nadie tenga que acordarse de lanzarla. | `Infra` | Cross | P2 | XS | `COMPLETADO` | diff --git a/reference/core/control-center/gaps/gap-tracking.md b/reference/core/control-center/gaps/gap-tracking.md index 1bc4b078..8a07b034 100644 --- a/reference/core/control-center/gaps/gap-tracking.md +++ b/reference/core/control-center/gaps/gap-tracking.md @@ -4,6 +4,7 @@ **Status:** Active Tracking **Owner:** Evolith Architecture Board +**Last Updated:** 2026-09-20 (**GT-716 AC3 landed: what only one engine decides is a registered difference, not a free one.** `29c8a4ba` — `73-validate-engine-coverage-parity.mjs` runs both engines on this repository and on a satellite it creates with `evolith init`, and holds every coverage-only rule to a baseline per scenario, per direction, per rule, with the reason the other engine gave (the class its report states; the facets the policy reads that a bare run does not supply). An unregistered rule, a stale entry or a changed class fails. Both scenarios read the Core from an export of the tracked tree, because the first CI run disagreed with the laptop that wrote the baseline (a local `coverage/` directory, git history and the CLI's bundled corpus copy had decided six rules and failed 138 ADR-conformance rules falsely). Measured on that export: repository 82 native-only / 7 opa-only, init satellite 49 / 7 — 52 of the 82 are policies reading facets nobody supplied, 26 have no policy at all, and four are the enforcer-routed `HXA-01/02/04/05`, where the OPA path skips without a reason of its own. The criterion's probe (drop an import from `main.rego`) cannot run since GT-675 — the build refuses it — so the equivalent one was recorded: rename `OBS-EVD-03` in its policy and the guard goes red on both scenarios naming the id. AC4–AC5 open; counters unchanged: **688 / 715 done · 3 in progress · 3 pending · 21 deferred**.) **Last Updated:** 2026-09-20 (**GT-716 AC2 landed: one evaluability declaration per rule, and both engines derive from it.** `3d76f4a6` — every rule in `*.rules.json` declares `facts`, the facets its check reads, from a vocabulary that says where the truth of each one lives (`observed` / `supplied` / `external` / `runtime`); `classifyRule` derives the native class from the most demanding provenance and the triage table keyed by rule id is gone; `npm run build:policy` refuses a bundle whose policy reads a facet its rule did not declare. Declaring `facts: []` for the twelve rules the table called non-executable while Rego decided them turned that check red with twelve findings; declaring what the policies read turned it green. **What moved, with nothing implemented: 44 of 415 rules changed class.** The "handler backlog" is 21, not 52 — 25 of those rows were decided by policies over a declared posture, the CI system or a test run, never by a handler over the tree — and a new native class, `needs-supplied-facts`, holds the 31 rules that need a posture only the satellite's owners can declare. The 73 blocking rules that do not run are the same 73, re-costed. Snapshot, ISO 5055 mapping and the standards README re-derived; guard 68's baseline down to `CLI-EXIT-01/03`. AC3–AC5 open; counters unchanged: **688 / 715 done · 3 in progress · 3 pending · 21 deferred**.) **Last Updated:** 2026-09-20 (**GT-716 AC1 landed: the OPA engine no longer calls an absent fact a verdict.** `b2840947` — the bundle compiles a second manifest, `rule_input_paths` (per rule id, the `input.…` paths its policy reads, from the compiler's AST), and `OpaEvaluator` reports `skipped` / `supplied-facet-absent` naming the facet when a run carries none of what a rule reads. Measured on a satellite fresh from `init`: `--engine opa` went from **133 decided to 10**. The row had predicted ≤ 60 from a count of 73; the count was too small, not the fix too large — 47 of the 57 rules both engines "decided" were OPA verdicts on absent input as well (`DOD-*` on `context`, `TAX-*` on `repository`, `INH-02` on `contracts`). Guard 68 declared eight baseline conflicts stale — the seven `supplied-facet-absent` and `TAX-01`, whose "differing exemption lists" were in fact no list at all (`repository-taxonomy.rego` reads `input.repository.files`, which no channel carries) — leaving 3 real disagreements over 14 jointly-decided rules. Supplying the facet still decides: `MTN-01` fails on `applicationFiltering: false` and passes on `true`, against the real bundle. `GOV-ENGINE-COVERAGE` stops telling the reader OPA "decides more". AC2–AC5 open; counters unchanged: **688 / 715 done · 3 in progress · 3 pending · 21 deferred**.) **Last Updated:** 2026-09-20 (**One gap registered from the owner's question over the known-limitations table — "how do we reach parity between the two engines" — measured before it was answered.** `GT-716` → PENDING. The page says `--engine opa` decides 133 of 159 where the default decides 41; measured today with the CLI built from this tree the default decides 56, and the 76 rules only OPA decides were crossed one by one against the policy body that decides them: **73 read a facet a bare run never supplies** (`satellite.git`, `.testing`, `.multiTenancy`, `.runtime`, `.protocol`, `.scorecards`, `.contracts`, `.ci`, `.layers`, `adapter`, `context.dod`, `user`, the `QT-*` metrics) and are verdicts on absent input, the same `supplied-facet-absent` family GT-704 baselined; three (`MCP-05`, `OBS-EVD-01/02`) are the real handler backlog. Over the whole corpus the sign flips (native 247 / OPA 187) because 138 ADR-conformance rules have a handler and no `.rego`, twelve rules are non-executable to one engine and decided by the other, and guard 68 prints all of it as `coverageOnly` without gating any of it. The row asks for three things in order: an absent supplied facet is `skipped` on OPA too; one evaluability declaration per rule that generates both the triage table and the bundle manifest; and the coverage difference as a per-rule ratchet in both directions, on the repository and on an `init` satellite. Not asked: equal coverage (ADR-0041 never promised it) or `--engine opa` as the default — its extra reach on a fresh repository is mostly not reach. Counters recomputed from the rows: **688 / 715 done · 3 in progress · 3 pending · 21 deferred**.) @@ -31,7 +32,7 @@ This board is the single source of truth for technical debt, gaps, opportunities | ID | Gap | In plain terms | What it fixes | Component | Phase | Criticality | Complexity | Status | |---|---|---|---|:---:|:---:|:---:|:---:|:---:| | [`GT-717`](./gap-reference-catalog.md#gt-717) | **The two GitHub Pages views were published by hand and had aged three months; nothing derived their numbers from the tree.** Measured 2026-09-19/20 against `origin/gh-pages` (`2c8f487a`, last touched 2026-07-07) and develop `6491c3bc`: the deployed Atlas predated the 2026-07-04 taxonomy move — 18 of its 21 source links answered 404 from github.io because `docHref` built `../` paths against the Pages root — was monolingual and had 4 scenarios with no explanations; the tree's own `architecture-map.json` named 20 docs paths of which 10 no longer existed. The master view printed `8 controllers`, `26 tools · 9 resources`, `20 commands`, `5 KindEvaluators`, `PWA · Web · Mobile` + `BFF (NestJS · ADR-0075)` for a Tracker that is .NET 10 + React 19, and the header `ADR-0101 · 0074 · 0075 · 0102`; its generator printed `12 hexagonal ports · 30 adapters` (line 149) while the tracked SVG said `11 of 20 ports · 53 adapters` under a hand-prepended `` annotation the generator never emitted — against a tree measuring 21 declared port interfaces (20 files) · 11 on the hot path (7 + 4) · 47 adapter classes · 55 tools · 12 resources · 8 prompts · 38 command files · 12 controllers · 33 endpoints · 12 kinds / 7 evaluators. No workflow referenced `gh-pages` (`git grep -l gh-pages 6491c3bc -- .github/workflows` = nothing). **CLOSED 2026-09-20** in `a09a1fc7` + `7eac8445`: `.harness/scripts/pages/build-pages.mjs` derives every printed number (`derive-page-metrics.mjs`; ports and adapters through guard 45's own functions), keeps out-of-tree facts in a dated `observed-facts.json`, validates the authored model, resolves 129 distinct `{{placeholders}}` per language, refuses any `../` in the built site, and `--check` plus an 8-test self-test run in the required `Validate documentation` job; the Atlas is 45 nodes · 62 edges · 11 chapters · 3 reading levels · EN/ES (197 UI strings), the poster's tour rides the same `chapters[]` on generator-emitted regions, and `pages.yml` projects `main` onto `gh-pages` (SHA-pinned, write permission only on the publish job). Observed red before trusted: merging develop (release 1.4.0) underneath made `--check` fail on the tracked SVG ("generated files are stale"), and `--write-svg` moved the door cards to 1.4.0 with no number typed. **The prose caught up the same day in [#787](https://github.com/beyondnetcode/evolith_arch32/pull/787) → [#789](https://github.com/beyondnetcode/evolith_arch32/pull/789):** the counts the views had typed also lived in `SECURITY.md`, `known-limitations`, the vision master, the MCP / CLI / OPA / interfaces READMEs, the tools catalog and the scorecard baseline (one outbound integration → two; 47 tools → 55; 26 categories → 21; 17 ports → 21; 6 mutative → 20; 8 required contexts → 10), corrected against the tree with a pointer to the generated inventory wherever a pointer suffices; gitleaks went red on the way because its fingerprints carry line numbers and the new rows moved them, and `gh-pages` needed no rebuild — its `metrics.json` already derived the same numbers (one differing key: `commit`). | The two public pictures of the product were three months old, most of their links were dead, and every number on them had been typed by hand. | Both pages are built from the repository on every promotion to `main`, in both languages; a number that ages turns the required check red instead of the picture. | `Documentation` | Cross | P2 | L | `DONE` | -| [`GT-716`](./gap-reference-catalog.md#gt-716) | **On a bare `evolith validate` the coverage gap between the two engines is mostly verdicts on facets nobody supplied, and nothing in CI pins the gap in either direction.** Measured 2026-09-20 with the CLI built from this tree (1.4.0, `b3df7e96`) on a satellite fresh from `init`, 159 rules in scope: native decides 56 and skips 103; `--engine opa` decides 133 and skips 26. Crossed rule by rule, 76 executable rules are decided by OPA alone, and the policy body of **73** of them reads a facet a bare run never sends (`input.satellite.{git,runtime,testing,multiTenancy,ci,findings,protocol,scorecards,layers,contracts}`, `input.adapter`, `input.context.dod`, `input.user`, the `QT-*` metrics) — an absent fact read as a verdict, the family six of the run's eight verdict conflicts belong to and GT-704's baseline already carries as `supplied-facet-absent`. Only `MCP-05`, `OBS-EVD-01` and `OBS-EVD-02` are decided from the tree: that is the real handler backlog. The other way, native decides 11 rules the bundle declares no policy for (`SSDF-*` ×7, `SEC-RL-01/02`, `QT-05`, `SLSA-HOSTED-L2`), 7 are decided by neither (`SEC-INJ/PATH/TIMING-*`, `SEC-RL-03`), and 12 that native classes non-executable (`KI-R01..07`, `INH-03..05`, `PROT-03/06`) OPA decides anyway — two classifications of the same rule from two unrelated sources (`RULE_TRIAGE` and `declared_rule_ids`), agreeing by construction only for `PEA-01..04`. Over the whole corpus the sign flips — native 247 / OPA 187 of 358, because 138 ADR-conformance rules and `MM-R*` have a native handler and no `.rego` — and `68-validate-engine-verdict-parity.mjs` prints both figures as `coverageOnly` and gates neither. **AC1 closed 2026-09-20 in `b2840947`:** the bundle now states what each rule reads (`rule_input_paths`) and OPA skips a rule whose fact the run did not supply; measured on the same satellite, `--engine opa` went from 133 decided to **10** — the 73 counted here were only the rules native skipped; 47 of the jointly-decided ones were absent-input verdicts too — and eight baseline conflicts (the seven `supplied-facet-absent` plus `TAX-01`, misdiagnosed) went stale and were removed. **AC2 closed 2026-09-20 in `3d76f4a6`:** every rule declares `facts` and both engines derive from it (`classifyRule` from the provenance, the bundle build refusing undeclared reads); the triage table is gone, 44 rules changed class with nothing implemented — the handler backlog is 21, not 52, and 31 rules need a declared posture (`needs-supplied-facts`) — and the twelve contradictions are declared and executable. | `--engine opa` looks like it checks more than twice what the default does, and on a fresh repository almost all of the extra is verdicts on facts nobody gave it; nothing would notice the gap growing. | An absent fact means "not evaluated" on both engines; one declaration per rule feeds the handler table, the bundle manifest and the report; and what one engine decides and the other does not is a per-rule baseline CI fails on, in both directions, on both scenarios. | `Core Domain` | Cross | P1 | L | `PENDING` | +| [`GT-716`](./gap-reference-catalog.md#gt-716) | **On a bare `evolith validate` the coverage gap between the two engines is mostly verdicts on facets nobody supplied, and nothing in CI pins the gap in either direction.** Measured 2026-09-20 with the CLI built from this tree (1.4.0, `b3df7e96`) on a satellite fresh from `init`, 159 rules in scope: native decides 56 and skips 103; `--engine opa` decides 133 and skips 26. Crossed rule by rule, 76 executable rules are decided by OPA alone, and the policy body of **73** of them reads a facet a bare run never sends (`input.satellite.{git,runtime,testing,multiTenancy,ci,findings,protocol,scorecards,layers,contracts}`, `input.adapter`, `input.context.dod`, `input.user`, the `QT-*` metrics) — an absent fact read as a verdict, the family six of the run's eight verdict conflicts belong to and GT-704's baseline already carries as `supplied-facet-absent`. Only `MCP-05`, `OBS-EVD-01` and `OBS-EVD-02` are decided from the tree: that is the real handler backlog. The other way, native decides 11 rules the bundle declares no policy for (`SSDF-*` ×7, `SEC-RL-01/02`, `QT-05`, `SLSA-HOSTED-L2`), 7 are decided by neither (`SEC-INJ/PATH/TIMING-*`, `SEC-RL-03`), and 12 that native classes non-executable (`KI-R01..07`, `INH-03..05`, `PROT-03/06`) OPA decides anyway — two classifications of the same rule from two unrelated sources (`RULE_TRIAGE` and `declared_rule_ids`), agreeing by construction only for `PEA-01..04`. Over the whole corpus the sign flips — native 247 / OPA 187 of 358, because 138 ADR-conformance rules and `MM-R*` have a native handler and no `.rego` — and `68-validate-engine-verdict-parity.mjs` prints both figures as `coverageOnly` and gates neither. **AC1 closed 2026-09-20 in `b2840947`:** the bundle now states what each rule reads (`rule_input_paths`) and OPA skips a rule whose fact the run did not supply; measured on the same satellite, `--engine opa` went from 133 decided to **10** — the 73 counted here were only the rules native skipped; 47 of the jointly-decided ones were absent-input verdicts too — and eight baseline conflicts (the seven `supplied-facet-absent` plus `TAX-01`, misdiagnosed) went stale and were removed. **AC2 closed 2026-09-20 in `3d76f4a6`:** every rule declares `facts` and both engines derive from it (`classifyRule` from the provenance, the bundle build refusing undeclared reads); the triage table is gone, 44 rules changed class with nothing implemented — the handler backlog is 21, not 52, and 31 rules need a declared posture (`needs-supplied-facts`) — and the twelve contradictions are declared and executable. **AC3 closed 2026-09-20 in `29c8a4ba`:** `73-validate-engine-coverage-parity.mjs` registers every rule only one engine decides — per scenario (this repository, an `init` satellite), per direction, with the other engine's reason — in `engine-coverage-parity.baseline.json`; an unregistered rule, a stale entry or a changed class fails. | `--engine opa` looks like it checks more than twice what the default does, and on a fresh repository almost all of the extra is verdicts on facts nobody gave it; nothing would notice the gap growing. | An absent fact means "not evaluated" on both engines; one declaration per rule feeds the handler table, the bundle manifest and the report; and what one engine decides and the other does not is a per-rule baseline CI fails on, in both directions, on both scenarios. | `Core Domain` | Cross | P1 | L | `PENDING` | | [`GT-715`](./gap-reference-catalog.md#gt-715) | **The Core API rejected any inline evaluation context over 100 KB with a masked 500 and no log line, so the Tracker's repository-conformance call never worked against a real repository.** Measured 2026-09-20 from the Tracker's UAT environment: `POST /products/{id}/evaluate-architecture` on a product pointing at this repository read 150 files (~1 MB) from GitHub and posted them to the Core, which answered `500 INTERNAL_ERROR "An unexpected error occurred"`; the Tracker recorded a `synthetic BLOCKED`. Reproduced on the same image (`main@142b8324`): 14 files / 99,797-byte body → `200`, 15 files / 101,578 bytes → `500`; a synthetic 92,956-byte body → `200`, 126,556 → `500`. Express's 100 KB json default, thrown as a `PayloadTooLargeError` that is not an `HttpException`, classified by message, masked for the wire and written nowhere. **CLOSED 2026-09-20** in `3b276c9a`: `EVOLITH_MAX_BODY_BYTES` (2 MiB by default) registers the parsers explicitly, body-parser errors keep the parser's status and the 413 names both sizes and the variable, and every masked 5xx is logged with its stack. Red first: 4 of 5 new specs fail on the old filter; after: the same 1.1 MB payload → `200` with the `gate-f1` verdict, 3 MB → `413`. Promoted in #778 (`9c5deedf`) and redeployed by the `Deploy UAT (Coolify)` job of run 35490911533 on 2026-09-20; measured right after: the same `evaluate-architecture` call answers `200`, `provenance: core`, `status: COMPLETED`, `resultDecision: FAILED` — a real verdict on 150 files (gates f1–f5 failed for missing phase artifacts), 174 ms in the Core. The front page keeps the phase-gate capture. | The Tracker could not get an architecture verdict on any real repository: the Core refused the request for its size and said nothing useful to anyone. | The Core evaluates a real repository sent inline, refuses with a reason it names when it must, and leaves a trace the operator can read. | `Core API` | Cross | P1 | S | `DONE` | | [`GT-714`](./gap-reference-catalog.md#gt-714) | **`gate evaluate` and `phase advance` in the published CLI need a checkout of this repository on disk, because the tarball carries the rules but not the gate definitions.** Measured 2026-09-20 with `@beyondnet/evolith-cli@1.3.2` in a clean `node:20` container on a satellite fresh from `init`: without `--core` both commands exit `1` with `ENOENT … reference/governance/sdlc/gates` (the tarball ships `rulesets/sdlc/phase-gates.rules.json` and the artifact registry at the package's own paths, but the validator composes `/reference/governance/sdlc/gates` and `/src/rulesets/sdlc/artifact-registry.json`, and `findCorePath` falls back to the satellite itself); with `--core ../evolith` both exit `2` with the real gate verdict. GT-705 fixed the same defect for the MCP package by bundling both trees and installing one resolver; the CLI package was not part of that change, and `sdlc gate-status` (GT-461 sub-finding) has no `--core` at all. Until it lands, the front page shows the two commands with `--core ../evolith` and says why in the caption. | The two commands that make Evolith more than a linter do not run from the published package without a clone of this repository next to the project. | `npx -y @beyondnet/evolith-cli gate evaluate --phase discovery` on a fresh satellite gives the verdict, no flag, and the front page loses its caveat. | `Evolith CLI` | Cross | P1 | S | `PENDING` | | [`GT-713`](./gap-reference-catalog.md#gt-713) | **The analysis that the Security tab's alerts are keyed to is produced only by the `push` run of `sdk-cli-ci.yml`, and its path filter skipped most of the code.** The `CodeQL SAST` job is the sole uploader of the `/language:javascript-typescript` analysis for `refs/heads/main`; pull-request runs are diff-informed (they prune to the diff and never move the branch's alerts) and the default "Code Quality" setup is a different suite. The `push` trigger was filtered to `src/sdk/cli/**`, `.harness/**` and the lockfiles. Measured 2026-09-19: promotion `19d736da` (changes under `src/packages` and `src/apps` only) reached `main` with no analysis at all, so the tab kept reporting 10 alerts on code that no longer existed; `c5547114` did the same 40 minutes later. Both needed `gh workflow run sdk-cli-ci.yml --ref main` by hand. **CLOSED 2026-09-19** in `72aceb70`: the filter now spans `src/packages/**` and `src/apps/**` — every tree CodeQL scans — while a docs-only push still skips the run. | The scanner that decides what the Security tab shows was not re-run when most of the code changed, so the tab described the previous commit. | A code promotion to `main` re-analyses `main`; the tab is current without anyone remembering to dispatch it. | `Infra` | Cross | P2 | XS | `DONE` | diff --git a/reference/core/sdlc/assets/master-view.svg b/reference/core/sdlc/assets/master-view.svg index b25a55ac..3100ee84 100644 --- a/reference/core/sdlc/assets/master-view.svg +++ b/reference/core/sdlc/assets/master-view.svg @@ -435,7 +435,7 @@ generated 2026-09-20 10 required checks · 18 workflows -73 guards + 31 self-tests +74 guards + 32 self-tests enforce_admins Security tab 0 · 0 · 0 · 0 diff --git a/src/rulesets/opa/README.es.md b/src/rulesets/opa/README.es.md index 3544dcfb..edc9ce15 100644 --- a/src/rulesets/opa/README.es.md +++ b/src/rulesets/opa/README.es.md @@ -48,6 +48,8 @@ Ambos motores derivan de esa única declaración, y la derivación se comprueba Añadir una regla significa, por tanto, declarar qué lee; añadir una lectura en una política exige que los `facts` de la regla la nombren. Lo que se movió cuando la declaración sustituyó a la tabla (2026-09-20): el «backlog de handlers» (`unimplemented-native`) pasó de 52 a 21 — 25 de esas filas las decidían políticas sobre una postura declarada, el sistema de CI o una ejecución de tests, nunca un handler sobre el árbol. +Lo que un motor decide y el otro no es entonces una **diferencia registrada, no libre** (AC3 de GT-716): [`73-validate-engine-coverage-parity.mjs`](../../../.harness/scripts/ci/73-validate-engine-coverage-parity.mjs) corre ambos motores sobre este repositorio y sobre un satélite recién salido de `evolith init`, y sujeta cada regla de un solo motor a [`engine-coverage-parity.baseline.json`](../../../.harness/scripts/ci/engine-coverage-parity.baseline.json) — por regla, por escenario, por dirección, con la razón que dio el otro motor (la clase que enuncia su informe, las facetas que la política lee y una ejecución a secas no suministra). Una regla sin registrar, una entrada obsoleta o una clase cambiada hacen fallar; `--write` regenera el fichero para revisión. Nada en él es una tolerancia: es la lista de lo que cada motor aún no puede decidir, y por qué. + ## Políticas de enforcement agregadas Estas 35 políticas son importadas y unidas por [`main.rego`](./main.rego) en el entrypoint Wasm `evolith/main/violations`. Cada una tiene un `*.test.rego` co-ubicado y (salvo indicación) un schema de entrada en `schemas/`. La lista autoritativa es el bloque `import data.evolith.*` de `main.rego`; el build rechaza una política que emite rule ids sin estar importada allí. diff --git a/src/rulesets/opa/README.md b/src/rulesets/opa/README.md index 41c5ba6f..8876b916 100644 --- a/src/rulesets/opa/README.md +++ b/src/rulesets/opa/README.md @@ -48,6 +48,8 @@ Both engines derive from that one declaration, and the derivation is checked in Adding a rule therefore means declaring what it reads; adding a policy read means the rule's `facts` must name it. What moved when the declaration replaced the table (2026-09-20): the "handler backlog" (`unimplemented-native`) went from 52 to 21 — 25 of those rows were decided by policies over a declared posture, the CI system or a test run, never by a handler over the tree. +What one engine decides and the other does not is then a **registered difference, not a free one** (GT-716 AC3): [`73-validate-engine-coverage-parity.mjs`](../../../.harness/scripts/ci/73-validate-engine-coverage-parity.mjs) runs both engines on this repository and on a satellite fresh from `evolith init`, and holds every coverage-only rule to [`engine-coverage-parity.baseline.json`](../../../.harness/scripts/ci/engine-coverage-parity.baseline.json) — per rule, per scenario, per direction, with the reason the other engine gave (the class its report states, the facets the policy reads that a bare run does not supply). An unregistered rule, a stale entry or a changed class fails; `--write` regenerates the file for review. Nothing in it is a tolerance: it is the list of what each engine cannot yet decide, and why. + ## Aggregated enforcement policies These 35 policies are imported and unioned by [`main.rego`](./main.rego) into the `evolith/main/violations` Wasm entrypoint. Each has a co-located `*.test.rego` and (unless noted) an input schema under `schemas/`. The authoritative list is the `import data.evolith.*` block of `main.rego`; the build refuses a policy that emits rule ids without being imported there.