Skip to content

perf(projects): read a large document once, not once per matching lane - #1676

Merged
leemour merged 1 commit into
mainfrom
perf/evidence-skip-known-large
Oct 20, 2026
Merged

leemour merged 1 commit into
mainfrom
perf/evidence-skip-known-large

Conversation

@leemour

@leemour leemour commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Why

The rerun of the large-document reproduction on Memo 0.79.0 (cli-testing#178) found the document, but charged 476,300 bytes for one 95,200-byte document. Each of its five matching lanes read the whole text again and only then dropped it as already found.

What

  • When a page reports a record too large to share it, and another lane in this run (or an earlier invocation) already returned that reference, gathering records the lane on it and moves on. It does not read the record again. The page's cursor already points past it.
  • take() and this path share one again(ref, lane) helper, so a lane is recorded the same way.
  • Docs: PROJECT-GATHERING.md and the changelog.

Testing

  • New test: a 95 KB document matched by two lanes charges less than twice its size. It fails on main (190,710 bytes) and passes here.
  • Services and context reader tests: 1,128 passed. Typecheck, lint, docs:check, docs:lint and the Bun smoke pass.

🤖 Generated with Claude Code

…e (PERF-51)

A record too large to share a page is read alone. When another lane had
already returned it, gathering still read its whole text again and only
then dropped it as found: on the released Memo 0.79.0, one 95,200-byte
document matched by five lanes charged 476,300 bytes (cli-testing#178).
Its page already moves past it, so the lane is now recorded on the known
reference without the second read. take() and this path share again().

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@leemour
leemour force-pushed the perf/evidence-skip-known-large branch from bb1664d to a30d703 Compare October 20, 2026 10:00
@leemour
leemour merged commit d09d394 into main Oct 20, 2026
12 checks passed
@leemour leemour mentioned this pull request Oct 20, 2026
leemour added a commit that referenced this pull request Oct 20, 2026
Release 0.369.0 from main d09d394: #1676 (project gathering reads a
large document once, not once per matching lane). Only package.json and
the CHANGELOG heading change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
leemour added a commit to WireCatLabs/cli-memo that referenced this pull request Oct 20, 2026
Bumps `@wirecat/cli-core` 0.367.0 → 0.369.0 and releases 0.80.0.

- 0.369.0
([cli-core#1676](WireCatLabs/cli-core#1676)):
`memo projects evidence` reads a large document once, not once per
matching search (measured 476,300 charged bytes for one 95,200-byte
document on 0.79.0,
[cli-testing#178](WireCatLabs/cli-testing#178)).
- 0.368.0
([cli-core#1670](WireCatLabs/cli-core#1670))
comes along: left-chat reads. No Memo code change.

Full suite 592 passed. Typecheck, lint, `lint:commands`,
`contracts:check`, `docs:check`, `docs:lint`, `test:matrix:check`,
`generate:check` and `release:check` pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
leemour added a commit to WireCatLabs/cli-testing that referenced this pull request Oct 20, 2026
## Why

[#178](#178) showed the
fixed release finding the large document, but charging 476,300 bytes for
one 95,200-byte document: each of its five matching lanes read it again.
[cli-core#1676](WireCatLabs/cli-core#1676) (Core
0.369.0, Memo 0.80.0) skips that read. Did it work on the released
packages?

## Results

Same script and invented document, exact registry Memo 0.80.0 / Core
0.369.0 / Base 0.47.1:

| | Memo 0.79.0 | Memo 0.80.0 |
| --- | --- | --- |
| Charged bytes | 476,300 | 95,260 |
| Reads | 10 | 6 |
| Found / complete | 1 / true | 1 / true |

## What

- `result-read-once.json` with the new run.
- `after-fix.mjs` now pins Memo 0.80.0 / Core 0.369.0. The README says
which commit produced the 0.79.0 result.
- README table, index line and changelog.

## Testing

Ran the commands in the README. lint, `docs:check`, `docs:lint`,
`docs:spell` and `standards:check` pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant