Progressive disclosure M3: the mover — fold LEARNINGS + DECISIONS#146
Merged
Conversation
Decompose specs/progressive-disclosure.md milestone 3 into an execution plan at specs/plans/pd-m3.md: the mover that first WRITES canonical knowledge files. 18 tasks across three epics — mover core (SplitStaging lossless byte-cut, WriteThemeBullet gist write-back, Apply IO mover with validate->append->verify->single-root-rewrite), the ctx disclosure apply CLI, and the ctx-digest apply-path skill + human-gated real rollout. Entry kinds only; CONVENTIONS digestion stays M4. No blocking TBDs. Rolling-wave gate passes (pd-m2 DoD complete). TASKS.md gains the PD-M3 phase with epic anchors E1-E3 partitioning T01-T18. Spec: specs/progressive-disclosure.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
Milestone 3's disclosure-package core: the guarded pass that first WRITES canonical knowledge files (the clobber risk class the M1 guards exist for). - SplitStaging: lossless header-to-next-header byte-cut of moved entries from raw staging (NOT via ParseEntryBlocks, which trims trailing blanks and would drop inter-entry whitespace). - WriteThemeBullet: gist write-back over raw ## Themes bullet lines — replace the touched theme's line, byte-preserve the rest, create the section on first run. - Apply: validate precondition -> append bodies to theme files (additive first) -> verify byte-presence -> single root rewrite last. Fail-loud, no auto-repair; any failure leaves the root byte-identical. - Mover error sentinels + ThemeDir / ThemeArrow / link constants. Tests: split byte-exactness, abort/root-untouched ordering, first-run creation, idempotency, post-apply invariants, and conservation. Unexported helpers split into movehelpers.go per the mixed-visibility convention. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
Wire the mover behind a command: ctx disclosure apply <file> --plan <path|-> reads a digest plan (JSON, from a file or stdin), resolves the file's kind, and hands both to disclosure.Apply, then reports what moved (--json for the ApplyResult). It refuses a non-knowledge file and a malformed root before any write, leaving the file byte-identical on refusal. - apply subcommand (cmd/run/doc) registered under the disclosure group. - Use/DescKey command constants, --plan/--json flag constants, and their commands/flags/write YAML entries (linkage bijection holds). - ApplyHuman / ApplyJSON result renderers in write/disclosure. Tests: apply moves + rewrites + reports, --json decodes to ApplyResult, non-knowledge-file rejection, malformed-root write-safety. Verified the built binary end-to-end: an un-migrated LEARNINGS.md folded into a bounded root with its entry bodies moved verbatim into the theme file. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
Graduate the ctx-digest skill from dry-run-only to the full guarded
pass: inspect -> propose themes -> author gists -> present for human
approval -> ctx disclosure apply -> confirm. The apply is human-gated and
every write goes through the CLI; the skill never hand-edits a file.
Copilot copy synced.
Two supporting changes landed with it:
- Plan format: Assignment.Entries is now [{timestamp,title}] (StagedEntry),
not a NUL-joined id string. It is exactly inspect --json's staging shape,
so the skill lifts entries verbatim with no NUL to hand-author in JSON.
FlattenPlan builds the ids internally (new entryID helper). No behaviour
or acceptance criterion changed; SplitStaging still takes []string. See
pd-m3.md Amendments. Verified end-to-end against the built binary.
- File organization: split the movehelpers.go bag file into
semantically-named files — move.go (the mover's append/verify/rewrite
mechanics), regions.go (renderThemeBullet + line/offset scanners, beside
parseThemeBullet/lineAt), collect.go (entryID, beside entryIDs). No
helpers/utils bag files.
Plan: specs/plans/pd-m3.md
Signed-off-by: Jose Alekhinne <jose@ctx.ist>
Flip T16 [ ] -> [x] in the pd-m3 plan: the measurement gate that drives ctx-digest apply on a realistic LEARNINGS-staging fixture (>=4 staged entries, >=2 themes) and checks entries moved with their bodies, ## Themes gists written, conservation, all four disclosure invariants, and a bounded root. T17 (rollout on the real roots) stays human-gated; T18 (milestone gate) verified green this session: make lint 0, go test ./... green, make audit pass. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
…a root The staging-boundary scan (firstLinePrefixOffset) and the themes-section scan (headingLineOffsets) matched any line starting with "## [" or equal to "## Themes" — including lines inside an HTML comment. A knowledge file's format guide can carry such an example: DECISIONS.md's template has a "## [YYYY-MM-DD] Decision Title" line inside its <!-- ... --> guide. With real entries present the example was harmless — staging still parsed into discrete entries. But once a digest folds every real entry out, the commented example is the only "## [" left above ## Themes: Parse marks the staging zone as starting there, Validate then finds a non-empty staging that parses to zero entries, and returns ErrStagingUnparsable — a valid, fully-folded root wrongly refused. Both scans now skip matches inside <!-- ... --> (htmlCommentSpans; an unterminated open runs to EOF). The commented example stays in the preamble, staging goes cleanly empty, and Validate passes. No knowledge file changes — the fix is in the parser. Surfaced by the first real LEARNINGS/DECISIONS rollout (pd-m3 T17); mirrors how pd-m1's measurement gate flushed out the insert.AfterHeader tail-truncation bug. Regression tests: a commented "## [" example is not staging; a commented "## Themes" is not counted as a duplicate section. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
…ONS (T17) Drive ctx disclosure apply on the real canonical roots — progressive disclosure's first write to live knowledge files (the clobber-risk class the M1 guards exist for). - LEARNINGS.md: 88 staged entries -> 12 themes under .context/learnings/; root 1831 -> 33 lines (bounded to the ## Themes gists + links). - DECISIONS.md: 63 staged entries -> 10 themes under .context/decisions/; root 1638 -> 59 lines (template header preserved byte-identical). Every entry body moved verbatim into exactly one theme file. Conservation verified independently (each original title present once, 0 missing, 0 duplicated) and the guarded move held: bodies appended and byte-verified before each root was rewritten once, last. Themes and gists were human-approved before apply. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
- T17 (real rollout): LEARNINGS + DECISIONS folded on the live roots, human-approved themes, conservation + all four invariants verified. - T18 (milestone gate): make lint 0, go test ./... green, make audit pass on the changed tree (incl. the fix(disclosure) comment-parse fix). Also records the comment-parse amendment: the T17 gate surfaced a real disclosure-parser bug (a "## [" example inside DECISIONS.md's HTML-comment format guide was mistaken for staging once every real entry folded out), fixed in its own fix(disclosure) commit. Plan: specs/plans/pd-m3.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Milestone 3 of progressive disclosure: the guarded mover that folds a
knowledge file's staging zone into per-theme files, leaving a bounded root
of gists + links. This is the first milestone that writes canonical
knowledge files, so every write goes through an append→verify→remove guard
— entry bodies are appended to their theme file and byte-verified present
before the root is rewritten once, and any failure leaves the root
byte-identical. Applied for real on this project's roots, and the rollout's
gate surfaced (and fixed) a genuine disclosure-parser bug.
Changes
SplitStaging(lossless header-to-next-header byte-cut),WriteThemeBullet(gist write-back over raw## Themesbullets), andApply(validate precondition → append bodies → byte-verify → single rootrewrite, last); abort / first-run / idempotency / invariant / conservation
tests — serves
specs/plans/pd-m3.md.ctx disclosure applyCLI — reads a digest plan (JSON via--planorstdin), refuses a non-knowledge file and the
conventionkind, iswrite-safe on any error (root left byte-identical on refusal),
--jsonresult renderer, doc.go + command-wiring guards — serves
specs/plans/pd-m3.md.ctx-digestskill (apply path) + copilot sync — inspect → proposethemes → author gists → human approve →
ctx disclosure apply→ confirm;plan schema refined to
[]StagedEntryso the skill lifts entries verbatimfrom
inspect --json— servesspecs/plans/pd-m3.md.fix(disclosure): ignore headings inside HTML comments — thestaging-boundary and themes-section scans now skip
## [/## Themeslines inside
<!-- … -->, so a knowledge file's commented format-guideexample is no longer misparsed as a staging entry after a fold empties the
real staging (regression tests included) — serves
specs/progressive-disclosure.md..context/LEARNINGS.md(88 entries → 12themes, 1831 → 33 lines) and
.context/DECISIONS.md(63 entries → 10themes, 1638 → 59 lines) into bounded roots under
.context/learnings/**and
.context/decisions/**, with human-approved themes and gists —serves
specs/plans/pd-m3.md.Specs
specs/progressive-disclosure.md— the design: canonical knowledge filesbecome bounded roots with a themed rollout.
specs/plans/pd-m3.md— milestone 3 decomposition (the mover): datamodel, apply CLI, skill, invariant-test matrix, tasks T01–T18.
Closed tasks
[E1]Mover core —Plan/Assignmenttypes,SplitStaginglosslessbyte-cut,
WriteThemeBulletgist write-back,ApplyIO mover, and theabort/first-run/idempotency/invariant/conservation tests (T01–T09).
[E2]ctx disclosure applyCLI — plan-JSON reader, non-knowledge-fileand convention-kind refusals, on-error write-safety, doc.go + wiring
guards (T10–T13).
(The plan
specs/plans/pd-m3.mdtracks the full milestone T01–T18 ascomplete, including the skill apply path (T14–T15), the measurement gate
(T16), the real rollout (T17), and the milestone gate (T18).)
Verification
make lint0,go test ./...green,make auditpass on the final tree.theme file — 0 missing, 0 duplicated (88 learnings, 63 decisions); roots
byte-preserved apart from the removed entry bodies + the new
## Themessection.
any failure leaves the root byte-identical.
applyverified against the built binary (un-migratedroot folded into a bounded root with entry bodies moved verbatim).