fix(governance): stage ConceptWeave Product ruleset enforcement - #2350
seonghobae wants to merge 45 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
left a comment
There was a problem hiding this comment.
P1 runtime finding: the privileged verify-live and mutate-owner-plane jobs execute the new Python 3.10+ reconciler but do not install/pin Python at all. The validation job does, and canonical #1644 apply also does. On a GitHub-hosted runner this makes the live control path depend on the runner image's incidental Python, so the source can validate while the privileged path later fails before governance verification/mutation. Add the same pinned actions/setup-python@5fda3b95... / Python 3.12 boundary to both live jobs and contract-test that requirement. Do not weaken the Python requirement or fall back to whatever python happens to be on the image.
seonghobae
left a comment
There was a problem hiding this comment.
P1 activation-path correctness: _decode_workflow() uses base64.b64decode(content, validate=True) directly on the repository-contents API content field. GitHub's current REST documentation shows that field as line-wrapped base64 containing \n separators. Python strict base64 validation rejects those separators, so a legitimate protected-base .github/workflows/product.yml can fail before canary validation and make evaluate→active promotion impossible. Please preserve strict alphabet validation but normalize only GitHub's documented CR/LF wrapping before decoding, and add a regression that uses a line-wrapped contents payload. RED is a valid GitHub contents response that fails current _decode_workflow; GREEN is the same payload decoding exactly while malformed non-base64 remains fail-closed. Source: https://docs.github.com/en/rest/repos/contents#get-repository-content (current response example contains \n in content).
seonghobae
left a comment
There was a problem hiding this comment.
P1 hosted entrypoint failure confirmed by current run 35805657970: validate fails at Validate reviewed Product target manifest before any lifecycle tests. The workflow executes python scripts/ci/reconcile_conceptweave_product_ruleset.py, but that file imports scripts.ci.reconcile_ruleset_governance; direct script execution sets the import root to scripts/ci, so the package-qualified scripts.ci import is not a reliable CLI entrypoint. The same failure already existed at predecessor run 35805011918, so it is not caused by the wrapped-base64 repair. Repair the workflow to use the package entrypoint python -m scripts.ci.reconcile_conceptweave_product_ruleset for validate/verify/bootstrap/activate, and contract-test that direct-script invocation is absent. Do not paper over it with PYTHONPATH.
seonghobae
left a comment
There was a problem hiding this comment.
P1 — bootstrap create has a protected-main TOCTOU window. bootstrap_product_ruleset() checks expected_main_sha only at entry, then performs repository ruleset discovery before _create_evaluate_ruleset() issues the POST. If protected .github/main advances during that discovery/read window, the live repository ruleset can be created from a stale trusted source revision and only be rejected after the mutation already exists. Activation already rechecks immediately before its PUT. Bootstrap should do the same immediately before POST, with a regression that proves a second current-main check aborts before _create_evaluate_ruleset() is called. Do not paper over this in the workflow or with PYTHONPATH; keep the guard in the owner mutation boundary.
seonghobae
left a comment
There was a problem hiding this comment.
P1 — activation/bootstrap do not bind the protected Product workflow to an immutable reviewed content coordinate. The ruleset correctly points at ContextualWisdomLab/ConceptWeave:.github/workflows/product.yml@refs/heads/main, but _assert_base_product_workflow() currently accepts any protected-main file containing four marker strings. Between #35 producer landing and this Product ruleset becoming active, a later Product change can retain those markers while weakening/removing substantive gates; Product itself is not yet a required merge gate in that bootstrap window. A successful canary plus evaluate rule-suite PASS would then validate the drifted workflow, not the #35-reviewed producer. RED: manifest/lifecycle tests must reject bootstrap/activation when the protected Product Contents API blob SHA differs from a reviewed immutable coordinate, and must reject mutation when no such coordinate has been adopted. GREEN: keep ConceptWeave as Product source owner, but carry only its immutable workflow blob coordinate in the owner-plane manifest; allow the initial source PR to keep that coordinate unadopted/null, require a normal reviewed manifest update after #35 lands, and re-check the exact blob before evaluate creation, canary acceptance, and active PUT. Do not copy Product YAML semantics into .github beyond the existing sanity checks.
seonghobae
left a comment
There was a problem hiding this comment.
P1 — the activation canary contract currently requires evidence that GitHub's ruleset workflow cannot naturally produce for the intended Foundation canary. _latest_base_retarget() requires a base_ref_changed event and rejects any later committed event; _canary_evidence() then requires a successful Product run created after that retarget. GitHub's current required-workflow semantics ignore pull_request activity filters and trigger only the default opened, synchronize, and reopened activities, so a base retarget (edited) does not create the required Product run. Foundation #1 already targets main and its current timeline has no base_ref_changed; after #35 lands, the intended ordinary/non-force reconciliation creates a real source commit and therefore a synchronize event, which the current code explicitly rejects if it follows a retarget. This makes the documented first substantive canary either impossible or dependent on artificial source-neutral PR-state manipulation/manual rerun, both outside the accepted contract.
Keep the current immutable-workflow-coordinate RED head unchanged until its hosted RED executes. On the subsequent repair head, replace the mandatory base-retarget proof with an event model that matches GitHub required workflows: accept a first-attempt terminal Product run from the exact current PR head/base produced by a supported ruleset event, and for the intended Foundation path require a substantive ordinary/non-force reconciliation commit after protected main advances. Bind the run to the exact PR/head/base and evaluate-mode rule-suite PASS; continue rejecting manual reruns/no-op wake, stale base, later source movement, and predecessor evidence. The existing target-main/blob/history TOCTOU guards remain required.
seonghobae
left a comment
There was a problem hiding this comment.
P1 — the immutable Product blob coordinate is parsed and _assert_base_product_workflow() can compare it, but neither live mutation path actually consumes it yet.
Current bootstrap_product_ruleset() calls _assert_base_product_workflow(target_main_sha) without manifest["product_workflow_blob_sha"]; therefore product_workflow_blob_sha: null does not fail closed and an evaluate ruleset can still be created from marker-compatible Product content. _canary_evidence() does the same unpinned call, and activate_product_ruleset() never requires a non-null reviewed blob coordinate before evaluate evidence or active PUT. That violates #2348's staged bootstrap contract and leaves the exact mutable-main gap this repair is intended to close.
Required repair after the currently queued exact-head owner run settles: add a contract RED proving bootstrap and activation reject null coordinates and marker-compatible blob drift before any POST/PUT, then thread the reviewed coordinate through bootstrap, canary admission and the final pre-PUT revalidation. Keep Product semantics ConceptWeave-owned; .github should compare only the immutable Contents blob coordinate plus existing marker/TOCTOU/ruleset-history guards. Do not pre-adopt #35's unlanded candidate blob.
seonghobae
left a comment
There was a problem hiding this comment.
P1 — the supported Foundation canary is still impossible without a governance-state manipulation because _canary_evidence() rejects draft=true. Canonical Foundation #1 is intentionally OPEN / Draft on main, and its ordinary/non-force parent reconciliation is supposed to produce the substantive pull_request:synchronize canary after #35 lands. GitHub's current ruleset-workflow contract does not require a PR to be Ready for synchronize: required workflows run on the default pull_request activities opened, synchronize, and reopened, and GitHub explicitly lists pushing/updating the branch as the way to trigger a newly required workflow on an already-open PR. A Ready transition is a different ready_for_review activity and is not a supported ruleset-workflow trigger.
Therefore, after the current 5289945740 mutation-boundary RED → repair → GREEN is complete, fold this into the separate supported-canary repair 5288830215: accept an open Draft Foundation PR as canary evidence while still requiring the exact PR number, substantive current head, current protected main base, reviewed Product blob, first-attempt terminal Product success from the supported pull_request:synchronize run, exact evaluate-mode workflows rule PASS, and no later source movement. Keep Ready/Draft toggling, reopen, no-op commits, manual reruns, predecessor evidence and administrator bypass explicitly invalid as canary evidence. Do not weaken the PR's own Draft/acceptance lifecycle merely to satisfy owner-plane activation.
Primary GitHub authority: https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/troubleshooting-rules and https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows.
seonghobae
left a comment
There was a problem hiding this comment.
P1 follow-up on the mutation-boundary repair: the immutable Product blob is now revalidated before POST/PUT, but the network read itself re-opens the protected-ref TOCTOU window. Bootstrap currently checks .github/main and ConceptWeave main, then performs _assert_base_product_workflow(... expected_blob_sha=...), then POSTs without re-reading either protected ref. Activation does the same before active PUT. If either protected ref advances while the Contents request is in flight, the mutation can still be authorized from stale owner source/base evidence even though the immutable blob check itself passed. Preserve the current exact head until its owner run settles; then add a reality RED where the first ref checks and blob check succeed but a final ref read observes drift, and require both _assert_current_main(expected_main_sha) and _assert_target_main(base_sha/target_main_sha) after the blob revalidation and immediately before POST/PUT. Do not weaken or remove the blob coordinate guard; this is the final ordering guard around it.
|
2026-09-24 KST fresh coordination: #2350 source remains exact |
|
Fresh ConceptWeave owner-path correction: the Producer prerequisite paragraph in this PR body is now stale only for downstream CodeQL execution state. |
|
Authority correction — producer prerequisite only; no source/head change. The
The two scan failures remain #2350 itself remains exact |
|
Authority update — Ordinary-forward test-only head is now Fresh owner run Separate central CodeQL state remains owner-external: #35 downstream Python/Actions shards completed scan + Medium+ SARIF and fail only at GHAS configuration-identity verification; only settlement remains queued. Keep that under #1929/#2275/#2276 rather than moving ConceptWeave source. |
|
Exact-current correction (2026-09-24 KST): PR head is
Owner run Separately, ConceptWeave #35 downstream settlement |
seonghobae
left a comment
There was a problem hiding this comment.
Review 5294002721 has now produced attributable hosted RED on 1a6ddce2.../35944402217. Current 97acdfeef0830d56c7b9b78226fec53de791ee8f is the minimal causal repair: exactly two protected-ref reads after the final Product blob validation in bootstrap and two in activation, immediately before POST/PUT. No policy/workflow/manifest/test/live-state delta is included. Keep the finding open until owner run 35961585829 is terminal GREEN; do not transfer predecessor GREEN.
|
Current-head RCA / repair update (2026-09-24 KST)
The failure is attributable to a retained success-path unit expectation, not the four-line production repair. Ordinary-forward test-only repair: New exact owner run |
|
A concurrent ordinary-forward commit, |
Purpose
Close #2348 without weakening ConceptWeave Product governance or deadlocking the one-time producer bootstrap. This remains a dependent Draft owner-plane PR on top of #1644.
Current authority — 2026-09-24 KST
fix/ruleset-owner-plane-reconciler/ fix(governance): automate ruleset owner-plane reconciliation #1644 exact722fec9de67aece7500993ee2999b21df0fab83b1e8da05d5b0851da212b3e340e7085fb1de550ce.githubowns protected-policy lifecycle and immutable workflow coordinates only.Accepted policy remains one dedicated organization
branch/workflowsruleset scoped only to ConceptWeave repository id1353201939and protected/defaultmain, binding source repository id1353201939,.github/workflows/product.yml,refs/heads/main; no repository status fallback and no Product addition to global ruleset18156473.Completed owner evidence
Immutable-source review
5287728521is GREEN through7a627f3e9c564bcf7ce2f826b57c1b6ef8d1ebae/ run35845521134.Mutation-boundary review
5289945740has attributable hosted REDfdbe09abb9079638098de92f5a40575d0696b675/ run35863230526. Production repaira1628e75dc056098f4ea2d03ec799f2026025e84requires a reviewed Product blob before mutation, binds bootstrap/canary admission to it, and revalidates the exact blob before active PUT. Test-only successora1af52d7bf2fcb5dfd27790961faf62651b16510closed the retained coverage defect with terminal owner GREEN35918073452.Review
5294002721now has a complete RED -> repair -> exact-head GREEN chain. Test-only exact1a6ddce2d02df50dddeb81450ff05cb9778c94a9/ run35944402217produced hosted RED for protected-ref drift during the final reviewed Product blob read. Minimal production repair97acdfeef0830d56c7b9b78226fec53de791ee8frevalidates protected.github/mainand ConceptWeavemainimmediately after the final blob validation and before evaluate POST / active PUT. Its hosted run exposed only stale retained call-count expectations. Test-only successor04414c5e30ba6aed2bf3d82268b356c6d6b1ad84corrected those two expectations; owner run35983462019is terminal SUCCESS, with validate107580552666GREEN on hosted runner1002122732.mutate-owner-planeandverify-livewere skipped, so this evidence changed no live Product-specific ruleset state.Current P1 supported-canary contract — reviews
5288830215+5291871021Current exact
1e8da05d5b0851da212b3e340e7085fb1de550ceis an ordinary-forward test-only RED candidate on top of the completed04414c5...owner GREEN. It changes onlytests/test_conceptweave_product_ruleset_reconciliation.pyand adds retained expectations that:pull_request:synchronizepath and therefore must not require_latest_base_retarget()/base_ref_changedevidence.The current production still rejects
draft=trueand still calls_latest_base_retarget(), so these tests intentionally target the known unsupported canary model rather than pre-applying the repair. Exact owner run36010121000is currently queued. Preserve1e8da05...until that run reaches terminal hosted evidence; do not blind-rerun, no-op wake, or add the production fix before attributable RED.The eventual minimum repair must keep all existing exact PR/head/current-main-base, reviewed immutable Product blob, first-attempt Product success, evaluate-mode
workflowsrule-suite PASS, no-later-source-movement and protected-ref/blob/history TOCTOU guards. Reopen, Ready/Draft toggle, manual rerun, no-op commit, predecessor run or administrator bypass remain invalid evidence.Producer prerequisite and central evidence
ConceptWeave #35 exact
d7b7e30b278ec2f27096b4d313c7d5eaf5387ddcremains the producer prerequisite. SAST35825043007and Security35825042856are terminal GREEN. Required CodeQL35825042996is terminal FAILURE from central ordering/publication, not ConceptWeave source.Downstream
.githubrun35870670165validated the exact #35/head/base/required-run tuple. Python107322443235and Actions107322443261both passed exact-head materialization, CodeQL analysis and the Medium+ SARIF gate, then failed only at GHAS base/head configuration identity; settlement107396154783later failed atSettle exact CodeQL required run. Keep GHAS credential/permission work in #2275/#2276 and lifecycle/publication/recovery in #1929/#2040. Queue continuation remains #2356. Do not move #35 to manufacture central evidence.Required order
1e8da05...until run36010121000is terminal and classify the supported-canary RED from exact hosted evidence.5288830215+5291871021as satisfied..githubsource.evaluate; adopt its positive identity through ordinary review.pull_request:synchronizerun is the evaluate-mode Product canary..github/main, protected ConceptWeavemain/canary base, immutable Product blob, repository/ref targeting, ruleset state/history and concurrent drift; promoteevaluate -> activeonly after exact canary and evaluate-rule PASS.No force push, destructive rebase, self-approval, synthetic status/reviewer, direct protected-branch write, routine admin bypass,
PYTHONPATHworkaround, manual/blind rerun, no-op wake, Ready/Draft trigger manipulation, or live ruleset mutation is authorized.Refs #2348, #1644, #772, #1351, ContextualWisdomLab/ConceptWeave#35, ContextualWisdomLab/ConceptWeave#1, ContextualWisdomLab/ConceptWeave#4.