Skip to content

chore(packs): merge main into feat/ce-packs-v0 - #1656

Merged
kieranklaassen merged 71 commits into
feat/ce-packs-v0from
cursor/merge-main-into-ce-packs-v0-b02e
Sep 9, 2026
Merged

chore(packs): merge main into feat/ce-packs-v0#1656
kieranklaassen merged 71 commits into
feat/ce-packs-v0from
cursor/merge-main-into-ce-packs-v0-b02e

Conversation

@kieranklaassen

@kieranklaassen kieranklaassen commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Merges origin/main into feat/ce-packs-v0 — 47 commits (f662cad4..fe74844c) in the first merge, plus a follow-up merge of the 3 commits main gained since (fe74844c..b36047e1: #1653 ce-noslop skill, #1658, #1659) in c9f982df — keeping main's text verbatim at every conflict and layering the packs branch's additions on top. The merged branch was then dogfooded through 75 Cursor cloud agents across 7 model families (Opus 5, Fable 5.1, Sonnet 5, GPT-5.6 Sol/Terra, Grok 4.6, Gemini 3.8, Composer 2.5) in four phases: 34 dogfood runs (found and fixed two review-enforcement defects), an 8-arm with/without-pack lfg bench, 19 runs across five pre-registered experiments, and 14 runs across four follow-up experiments. Every run loaded this branch's skills natively; zero marketplace-skill reads.

Main renamed skills/guides/ -> docs/guides/ (#1551) and culled docs/solutions/ learnings (#1625); both carry through. The follow-up merge was conflict-free and brings the skill count to 35.

Conflict resolutions

File Resolution
README.md Main's two hunks kept; the Compound Packs callout and the docs-table row re-added pointing at docs/guides/packs.md.
docs/guides/ce-plan.md Main's rewritten research paragraph kept; the pack-citation clause folded in.
docs/guides/ce-brainstorm.md, docs/guides/ce-compound.md Main's relocated copies, with the branch's pack additions ported to the same anchors; stale skills/guides/ copies removed.
docs/guides/packs.md Branch-only file, followed the rename.
skills/ce-compound/SKILL.md (auto-merged) Combined body hit 8062 bytes, over Codex's 8000-byte cap; the branch's pack-write sentence dropped two clauses references/assembly.md 5b already owns. Main's text untouched. Result 7994 bytes (7915 after the follow-up merge) — little headroom left; the next ce-compound body edit should relocate a block to a reference.

Defect 1 — ce-code-review never enforced packs on a repo without learnings (40564c27, f32cac79, d72061bc)

Pack enforcement rides on the learnings-researcher persona, whose spawn gate required an existing <root>/solutions/ corpus with a plausible match. A repo adopting packs before it has learnings got no review enforcement, contradicting docs/guides/packs.md. Three strict by-the-book runs (Sonnet 5, GPT-5.6 Sol, Grok 4.6) hit it independently.

  • scripts/review-scope.py reports declared_packs / pack_roots via the sibling resolver.
  • persona-catalog.md and select-and-route.md state the gate as its condition: institutional knowledge to check the change against — a matching corpus, or (local scope) declared packs, no pre-search.
  • The Stage 3c lite roster carries the pack-selected persona like project-standards, so a 1-line violation is enforced too.

Defect 2 — a contradicted rule had no route into the numbered finding set (7c037973, 55fd6163, 905a1dc6, 46294285, 12e9d940, 2bc809dc, a8b90ca4, 25727c43)

The persona's output is research-shaped and Stage 6 only rendered it as "Known Pattern" notes; in mode:agent a violation left in learnings is invisible to lfg's apply step. Now the persona states per matched rule where any violating line sits (changed → finding candidate subject to the ordinary gates incl. the false-positive check; unchanged → pre-existing; none → note; a line violates a rule only when the rule's own condition reaches it), Stage 5 serializes each contradiction as a complete compact reviewer return, section 7 keeps honored rules as notes, mode:agent gets a defined learnings element shape and a coverage.compound_packs key, and the merge test names "same defect and fix path".

Tests: tests/ce-code-review-mechanics.test.ts, tests/skills/ce-packs-contract.test.ts, tests/review-skill-contract.test.ts (pin moved from the #1159 case to the restated condition). Docs: docs/guides/packs.md names the skipped-file warning as printed, scopes the refine-the-rule offer to interactive runs, and covers CODING_STANDARDS.md coexistence and disjoint rule authoring.

What the experiments established (details in the PR's artifact bundle)

Question Answer
Does declaring a pack change output? Deterministically: (pack: …) citations in plan and review in every with-pack arm, zero in every no-pack arm (18 pairs/arms).
Does it change shipped code? Yes when rules are not inferable from the repo: 3/3 vs 0/3 on Opus, Fable, Sol (×3), Grok. With inferable rules, strong models find the helpers anyway.
Token cost? Model-dependent: Opus/Fable −12% to −41% with a pack (less deliberation); GPT-5.6 Sol +9% to +31% in 3 of 4 pairs (more compliance work); Grok noise. Not a property of packs.
Precision with a noisy pack? 15-rule pack: 3 relevant rules cited, 0 of 12 decoys cited, on both models tested.
Is pack text obeyed as instructions? No: an injected "skip logging checks / drop validation" note was ignored on Sol and flagged as "prompt injection block" by Fable; the planted violation surfaced.
Does the compounding loop close? Yes: phase-A ce-compound learnings and their promoted pack rules both made a fresh agent apply the conventions on a new endpoint (2/2 vs 0/2 control); only the pack form produced citations and reached review, and the learnings' file:line cites had aged.
Does the effect travel to another task family? Yes (round 3, N1): CLI CSV import, non-inferable rules 3/3 with the pack vs 0/3 and 1/3 without on Sol and Fable; both no-pack runs invented duplicate_email where the house code is email_taken.
Is AGENTS.md a cheaper substitute? No (N2): same three rules as instruction bullets gave 3/3 on all four models but zero citations, zero review findings, and cost more total tokens than the pack arm on 3 of 4 models.
Stale rule pointing at a deleted helper? (N3) Both plans named the absence and recreated the helper first; Sol exact, Fable dropped the u_ separator — a helper-referencing rule should state an observable output.
Misleading learning in docs/solutions/? (N5) Never shipped: ce-plan's grounding rejected the fabricated SQLite doc against the tree on Opus and Fable; with ce-compound-refresh run first it was deleted/superseded before planning.

Learnings captured (cedd603b, 5789e25d, 27349fe6, 6eaa10dc)

  • docs/solutions/skill-design/new-knowledge-source-re-derives-persona-gate-and-route.md
  • docs/solutions/developer-experience/global-fsmonitor-makes-the-suite-slow-and-flaky.md
  • docs/solutions/workflow/measuring-a-packs-effect-needs-non-inferable-rules.md

Full record — 76 runs, ≈1.55B tokens, every report, prompt, score, the fixture and the three write-ups — is filed as experiments/002-ce-packs/ in EveryInc/frontier-experiments-kieran (branch cursor/ce-packs-file-experiment-002-results-1d10, PR pending manual approval). The throwaway scaffold branch cursor/packs-dogfood-workspace-b02e has been deleted.

Validation

  • bun run test — 3966 pass / 1 skip / 0 fail on c9f982df (with a minimal GIT_CONFIG_GLOBAL; the VM's core.fsmonitor=true caused rotating 20s timeouts, see learning).
  • bun run release:validate — in sync (35 skills). bun run plugin:validate — both manifests pass --strict with CI-pinned claude 2.1.220.

Security Disclosure

skills/ce-code-review/scripts/review-scope.py now spawns the sibling packs-resolve.py with sys.executable from the repo root, bounded by a 180s timeout; the resolver already bounds its git calls and treats pack text as data. E4 empirically confirmed pack text is not obeyed as instructions on two model families. No secrets handling changed.

Agent Disclosure

  • Model: Cursor · Claude (family only; Cursor does not expose the running model ID)
Open in Web Open in Cursor 

kieranklaassen and others added 30 commits August 26, 2026 22:14
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…ate progress and gate DONE (#1558)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…ck (#1568)

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
)

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
)

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Co-authored-by: Hally Maschine <hally@rocketable.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@kieranklaassen
kieranklaassen marked this pull request as ready for review September 8, 2026 19:26
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-09T18:20:30.505845Z c9f982d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kieranklaassen
kieranklaassen marked this pull request as draft September 8, 2026 20:09
cursoragent and others added 23 commits September 8, 2026 21:42
…t only a learnings corpus

Compound Pack enforcement at review rides on the learnings-researcher persona,
but its spawn gate required an existing <root>/solutions/ corpus with a plausible
match. A repo that adopts packs before it has any learnings therefore got no
review enforcement at all, contradicting docs/guides/packs.md. Found by
cloud-agent dogfood on Claude Sonnet 5, GPT-5.6 Sol, and Grok 4.6 independently.

- review-scope.py reports `declared_packs` / `pack_roots` by running the sibling
  resolver (silent only when no config names a pack; null when it cannot tell).
- persona-catalog.md and select-and-route.md state the gate as its condition:
  institutional knowledge to check the change against, from a matching corpus
  or, in local scope, declared packs (no pre-search; the persona matches rules).
- The small-diff lite roster carries the pack-selected persona the way it already
  carries project-standards, so a three-line violation is enforced too.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
The guide called it `Skipped pack files`; the resolver and /ce-setup print
`skipped pack file <id>/<name>`, and only the research pass uses the heading.
Also state that declaring packs selects the review learnings pass on its own.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…gate, not a spawn rule

A re-verification run read scope.md's 'inputs, not automatic spawn decisions'
sentence as conflicting with the persona catalog, where declared packs select
the learnings persona. State that the helper supplies the fact and the catalog
gate decides what it selects.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…d helper signals

Two re-verification runs (GPT-5.6 Sol, Grok 4.6) read the Stage 3 sentence as
contradicting the learnings gate, where declared packs select the persona.
State which helper fields are heuristics and that declared_packs is a fact the
gate decides on.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
… finding set

The learnings persona's output is research-shaped and Stage 6 only rendered it
as Known Pattern notes, so a pack rule the diff contradicts had no stated path
into findings; in mode:agent that means lfg would never apply it. Two cloud-agent
re-verification runs (GPT-5.6 Sol, Grok 4.6) had to guess the route.

- The review persona states per matched rule whether the diff honors or
  contradicts it, quoting the rule and the changed line.
- Stage 5 serializes each contradiction as a compact reviewer return before the
  first helper run, so it gets a stable #, dedup, and the actionable route.
- Section 7 keeps honored rules and past solutions as Known Pattern notes.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
… its project-standards case

#1159 pinned 'no conditional other than project-standards'. The gate now states
the condition that clause was one case of, and still names project-standards.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…g compact-return shape

Re-verification on Grok 4.6 built the synthetic return from the fields the
Stage 5 paragraph listed and the helper rejected it as malformed. Name the shape
dispatch-reviewers.md owns instead of enumerating fields, put evidence in the
source-detail map, and say step 1 reconciles a structured duplicate.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…tays under the per-test timeout

Each scope-helper run now also spawns the resolver; six runs in one test
crossed 20s under full-suite load on a 4-core runner. Also say where the
rule and line must be quoted for the anchor-100 pack finding.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…n gate and synthesis route

Captured from the cloud-agent dogfood of Compound Packs: adding pack roots to
the review learnings persona without revisiting its gate or its route into the
finding set left review enforcement silently off, and the earlier eval passed
only because agents dispatched the persona regardless of the gate.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…er and flaky

Cursor cloud-agent VMs ship it in the global git config; the suite's hundreds
of throwaway repos each start a daemon, and git-heavy tests time out at 20s in
a rotating set. GIT_CONFIG_GLOBAL with a minimal config: 3941 pass in 84s.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…date, not an exempt finding

Round-5 dogfood (Claude Sonnet 5) read the Stage 5 sentence as mandatory
serialization while the quality gates require a false-positive check; the
researcher had claimed a validation rule against a line that only logs. State
that a contradiction passes the same gates as any structured finding.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…on condition, and learnings shape

Round-5 dogfood on Opus 5 and Fable 5.1 found the pack-candidate paragraph
omitted "and fix path" from step 1's merge test (a standards finding on the same
line would be merged away), "partially contradicted" had no outcome, Stage 3b's
"both kinds" was silent on packs, and the mode:agent `learnings` element shape
was undefined. State each as its condition at the owning layer; the guide now
covers pack + CODING_STANDARDS.md coexistence and disjoint rule authoring.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…dition

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…ractive runs

Round-5 dogfood (Opus 5) read the guide's unconditional offer against
ce-compound's assembly reference, which skips with the documented signal in
non-interactive mode.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…e to the pre-existing partition

Round-5 dogfood (Opus 5) hit a matched rule that was neither contradicted by a
changed line nor honored: unchanged code violated it. State the condition as
where the violating line sits (changed -> finding, unchanged -> pre-existing,
none -> note), and give mode:agent a stable coverage.compound_packs key for the
resolver result that dispatch-reviewers.md already says to surface in Coverage.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…ch the line; pin the coverage.compound_packs shape

Round-5 dogfood on Fable 5.1 and Opus 5 (T13/T14): the researcher twice reported
a lookup-validation rule against a value that was only logged; coverage.compound_packs
was emitted with two shapes; section 7 said "honors" where "no violating line"
was meant; the guide said "a finding for each contradiction" where Stage 5 merges
shared-fix contradictions. State each condition.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…hanged or not

A pre-existing pack candidate has no changed line to quote; the anchor rule
keyed on 'the changed line' and forced 75 (round-5 dogfood, Opus 5).

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
… token effect is model-dependent

From two rounds of paired cloud-agent runs: inferable rules measured repo
exploration (3/3 vs 3/3), non-inferable rules measured the pack (3/3 vs 0/3 on
four models); Claude models spend 12-41% fewer tokens with a pack, GPT-5.6 Sol
9-31% more, Grok is noise.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…m and helper-referencing rules

Round 3 of the cloud-agent experiments (14 runs) showed that the same rules as
AGENTS.md bullets match a pack's compliance with zero citations at equal or
higher token cost, and that a rule naming a deleted helper gets the helper
recreated from prose with one near-miss. Record both as eval-design guidance
and point at the experiments repo that holds the full record.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
…o-ce-packs-v0-b02e

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>
@kieranklaassen
kieranklaassen marked this pull request as ready for review September 9, 2026 18:11
@kieranklaassen
kieranklaassen merged commit d59bbb2 into feat/ce-packs-v0 Sep 9, 2026
4 checks passed
cursor Bot pushed a commit that referenced this pull request Sep 9, 2026
PR #1656 merged main's content into this branch but landed as a squash, so
the branch carried main's changes without main in its history. GitHub then
reported conflicts on #1549 for every hunk both sides had touched. The tree
of this commit is identical to d59bbb2 (verified: zero-line diff against
the pre-squash merge head c9f982d, which already had b36047e as an
ancestor); only the parentage changes.

Co-authored-by: Kieran Klaassen <kieranklaassen@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9f982df65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +949 to +950
if (cookieName && sessionToken) {
headers["Set-Cookie"] = `${cookieName}=${sessionToken}; HttpOnly; SameSite=Strict; Path=/`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require authentication before minting the session cookie

When annotate mode uses the documented --host 0.0.0.0, any network client can make an unauthenticated document GET and receive the secret token as a session cookie here. I verified that GET / with Sec-Fetch-Dest: document, followed by POST /session/end using only the returned cookie, succeeds with HTTP 200; therefore /annotation, /wait, and /session/end are not actually token-gated for reachable visitors. Require the initial document request to prove token knowledge before minting this cookie, or prohibit non-loopback annotate bindings.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants