gh search/code silently undercounts by up to 31% while reporting incomplete_results:false — adopt enumerate-then-filter as doctrine

[hyperpolymath]

gh api search/code silently undercounts and reports itself complete — it has produced wrong numbers in at least 4 open issues

Surfaced while re-verifying the backlog. Seventeen issues were found carrying wrong counts today; the errors fall into exactly two mechanisms, and both are avoidable.

---

Mechanism A — code search undercounts, and lies about it

Measured today against enumerate-then-filter censuses of the same thing:

query	search/code says	census (ground truth)	short by
filename:AGENTIC.a2ml user:hyperpolymath	401	582	31%
filename:CLAUDE.md user:hyperpolymath	578	628	8%
81dbf2dd… (a 40-char SHA)	535	566+	≥5%

In every case the API returned "incomplete_results": false. It asserts the result is complete while being up to a third short.

Separately — and this is the documented limit everyone knows about — there is a hard ceiling:

GET search/code?...&page=11  →  HTTP 422
"Cannot access beyond the first 1000 results"

The 1000-item ceiling is not the problem. The problem is the silent undercount well below it, combined with a completeness flag that says everything is fine.

Issues this has already corrupted

• [campaign] Estate CodeQL weekly→monthly sweep (cut 3, standards#233 Option B — ~206 repos) #288 — derived "~206 repos" from code search. Enumeration: 1,046 files across 355 repos.
• [campaign] Long-tail non-canonical CodeQL cron sweep (~86 files / 30 repos) #324 — derived its cron table from code search. Every figure was wrong, and it missed the largest cluster entirely (0 0 * * 0, 41 files).
• Estate campaign: roll SonarCloud across all repos (parked, resume later) #443 — SonarCloud coverage counts.
• AGENTIC.a2ml instructs agents to 'Never use AGPL' — contradicts Rules 3, 4 and 5 #646 — the original "899 files" figure.

---

Mechanism B — counting vendored and local copies overcounts

The opposite error, from a different cause:

issue	claimed	measured	why
#252	~3,996 .res	281	6,909 of developer-ecosystem's were vendored upstream ReScript
#404	~11,000 PMPL headers	1,933	—
#276	~15 repos in band	1	vendored compiler counted as port targets
#307	~104 repos	6 real	vendored subdirs + a Joshua-owned repo
#324	86 non-canonical files	63, of which 697 estate-wide are in vendored subdirs GitHub never executes	—

Local working trees make this worse: they contain .claude/worktrees/, node_modules/, and recovery checkouts. My own first AGENTIC.a2ml count said 646 from local trees; the live default-branch census is 582.

---

The rule this issue asks to adopt

Never derive an estate count from gh api search/code or gh search.
Enumerate with gh repo list <owner> --limit 500 --no-archived, then filter per repo against git/trees/{default_branch}?recursive=1, checking the truncated flag on every call.
State the denominator in the issue. A count that lands on a round number (100, 1000) is a page size, not a measurement.
Exclude vendored paths explicitly, and say which patterns were excluded.

Three supporting facts worth recording alongside it:

• gh api prints 404 bodies to stdout. So n=$(gh api ... 2>/dev/null) yields a non-empty string on failure, and a naive [ -n "$n" ] scores every repo as present. This produced a wrong estate-wide result earlier today.
• /commits/{sha}/check-runs caps at 100 per page. Without --paginate it manufactures false "missing context" hits.
• Tree API truncation is real but rare and reported — 1 of 420 repos (julia-ecosystem, 69,115 entries). Unlike code search, it tells you.

Suggested deliverables

1. Add the rule above to standards as measurement doctrine (a short MEASUREMENT.adoc, or a section in the audit runbook).
2. A hypatia rule or PR-template line that flags an issue body quoting a count without a stated denominator.
3. Optionally a scripts/estate-census.sh implementing enumerate-then-filter once, so audits stop reinventing it — each reinvention has been wrong so far.

Related: #288, #324, #443, #646, #252, #404, #276, #307.