Skip to content

VALIDATION: independent review of the 2026-08-13 Track B migration work (waves B1' and B2') #84

Closed
Closed
VALIDATION: independent review of the 2026-08-13 Track B migration work (waves B1' and B2')#84

Description

@mmcky
mmcky
opened on Aug 13, 2026

Everything below was verified during the work, by the same agent that did the work. That is the bias this issue exists to break. Track B of the datasets migration completed today across two waves and two sessions — B1′ (the ols trio, fp.dta, NEWQDATA.csv) and B2′ (the hansen_singleton pair and their builders) — and both were checked with one set of tools, one shell, one mental model.

Bias to test for

The monoculture is curl from a single macOS shell plus bespoke Python extractor scripts written by the agent that made the claims. Nearly every reader-facing conclusion — "the published page is correct", "the repoint changed no figure", "the deleted file is still served" — rests on a content_numbers() regex extractor written during the session. That extractor produced a wrong answer twice and was patched twice: once it reported IDENTICAL ✅ against files it had never opened, and twice it reported hundreds of spurious differences that were two number streams knocked out of step by the page's own changelog widget. A third undetected flaw in it would silently confirm exactly what the session wanted to hear.

A second, quieter monoculture: every builder reproduction ran under the agent's local Python 3.13.9 / pandas 2.3.3, not under any pinned project environment. The builder_status: committed and integrity.upstream.status: verified claims both rest on that unpinned run.

Validator: confirm reader-facing outcomes without that toolchain. Open the published pages in a browser and read the rendered tables. Fetch from a different machine and network. Re-derive hashes with a different utility. Do not re-run the session's extractor except where a check says to.

What landed

Repo Merged today Open
QuantEcon/data-lectures #79 (99d2d6d, land B1′), #80 (f363d1b, flip B1′), #81 (2f7d3a3, AGENTS ordering trap), #82 (16ddbbf, land B2′), #83 (da01657, flip B2′) —
QuantEcon/lecture-python.myst #1034 (e168c7d, repoint B1′), #1035 (d2a5b21, delete B1′), #1036 (a501f03, repoint B2′) #1037 — delete B2′
QuantEcon/lecture-python.zh-cn #226 (3e1289b), #249 (940bd74), #250 (151c402) — translation syncs 30 unrelated sync PRs
QuantEcon/lecture-stats #61 (B1′ prose link) #62 (linkcheck, unowned)

Publishes: lecture-python.myst publish-2026aug13 and publish-2026aug13b; lecture-python.zh-cn the same two; lecture-stats publish-2026aug13.

Issues: QuantEcon/workspace-lectures#41 opened (build-cache prune); QuantEcon/workspace-lectures#40 body corrected and commented; QuantEcon/workspace-lectures#39 resume block and title rewritten.

1. Reader-facing outcomes

The clearest known incident to confirm: wave B1′'s five files were deleted at 06:31:37Z and a successful publish finished at 09:34:28Z, and the session claims they are STILL SERVED. That is the load-bearing observation behind QuantEcon/workspace-lectures#41.

  • Fetch each of https://python.quantecon.org/_static/lecture_specific/ols/maketable{1,2,4}.dta, .../mle/fp.dta, .../phillips_drifts_volatilities/NEWQDATA.csv from a machine that is not the one that did this work. Record status, content-type and actual downloaded byte count. Expected today: 200, with 14466 / 11210 / 12024 / 1024085 / 13738 bytes. Confirm a never-existed sibling path 404s in the same run.
  • After the next cache.yml clean rebuild (Mon 2026-08-17 03:00 UTC) and a publish following it, re-fetch the same five. Expected: 404. Done 2026-08-14, three days early — the rebuild was dispatched manually (run 31757133369) and publish-2026aug14 followed; all five, plus B2′'s six, now 404 with a never-existed control 404ing and ols.html 200ing in the same pass. The model held, so no new issue. Original wording follows: If any still 200s, the session's model of the trap is incomplete — open a new issue.
  • Open https://python.quantecon.org/hansen_singleton_1982.html and .../hansen_singleton_1983.html in a browser and confirm the estimate tables and figures render with numbers, not errors or blanks. The data cell is hide-cell, so a failed fetch surfaces only downstream.
  • Do the same for https://quantecon.github.io/lecture-python.zh-cn/hansen_singleton_1982.html and 1983.html.
  • Beyond what the session tested: the zh-cn site publishes downloadable notebooks into its own Pages output at /_notebooks/, and the session never checked that tree. Fetch https://quantecon.github.io/lecture-python.zh-cn/_notebooks/hansen_singleton_1982.ipynb and 1983.ipynb and confirm they carry data-lectures URLs and no _static/lecture_specific/hansen_singleton reference.
  • Beyond what the session tested: open one of the repointed notebooks in Colab from the published page's own launch button and run the data cell. CPython-in-CI passing does not prove the reader's runtime resolves the URL.
  • Fetch the published PDF for lecture-python.myst and search its URI annotations for lecture_specific/hansen_singleton and for lecture-python.myst/raw. Note strings is invalid on a compressed PDF — decompress the streams. Expected: no old-form references.

2. Artifact integrity

Two datasets moved. The claim is byte-identity between what the lectures read before and after.

  • Re-derive both hashes with a utility other than shasum (e.g. openssl dgst -sha256, or Python hashlib) after fetching https://github.com/QuantEcon/data-lectures/raw/main/lectures/hansen_singleton_1982_data.csv and ..._1983_data.csv. Expected 3e9d4f37c31dbcab26624418f3eeb630d57cc362bd162fa5501cfd992225609b (11,662 B) and dc5c1f8dac4b50abb3613dd297f6c22b4ebde80b9a7f02971e8d986b0c6eff18 (26,084 B), matching each manifest's integrity.sha256.
  • Triangulate against a third source the session did not use as the gate: extract the same two files from lecture-python.myst at commit a501f03 (before #1037 deletes them) and from lecture-python.zh-cn@main, and confirm all three copies hash identically. A divergence in the zh-cn copy would mean the translation's readers were served different bytes than the English readers.
  • Verify the claim that hansen_singleton_1982_data.csv is a bitwise strict subset of the 1983 file — identical date index, and gross_real_return / gross_cons_growth identical to the last digit. Do it textually (cut the columns and diff) rather than with pandas .equals(), which is how the session checked it.
  • Confirm both files are plain git, not LFS: git show main:lectures/hansen_singleton_1982_data.csv | head -1 must be a CSV header, not version https://git-lfs....

3. The highest-value claim to re-test

builder_status: committed and integrity.upstream.status: verified both rest on one unpinned local run. More future work depends on this than on anything else today: it is the difference between committed and committed-frozen, it licenses editing the builders, and it is the evidence recorded in both manifests.

  • Re-run builders/hansen_singleton_1982_data.py and builders/hansen_singleton_1983_data.py under the project's pinned environment, not an ambient one, and confirm each reproduces its committed CSV byte for byte. If it does not, the manifests' verified status is wrong and needs correcting — that is a finding, not a nuisance.
  • Re-run them again a day or more later. Both fetch live upstreams (FRED, the Ken French library). The session's claim is that the 1959–1978 window is stable; a divergence would mean verified has a shelf life the manifests do not state.
  • Prove the fetch is real, not cached: run with the network disabled and confirm both fail rather than silently producing output.
  • Confirm each validate() actually rejects. Feed each a deliberately corrupted frame — drop a row, multiply by 100, subtract 1, insert a NaN, rename a column, remove a mid-series month, and for 1983 copy gross_real_return over gross_real_tbill. All seven must raise.
  • Corroborate the provenance narrative from a source the session did not read: confirm the Ken French F-F_Research_Data_Factors_CSV.zip header still names a CRSP database vintage and Ibbotson for the pre-2024 T-bill leg, and check whether the data library publishes any terms or usage page linked from its main page. The manifests assert it states no licence at all, based on grepping one fetched HTML page — a linked terms page would refute that and change redistribution: permitted from a reasoned call into an unchecked one.

4. Records written

  • Validate lectures/hansen_singleton_1982_data.csv.yml and ..._1983_data.csv.yml field-by-field against manifest-schema.yml — every required field present, no invented fields, builder_status and integrity.upstream.status from the allowed sets.
  • Re-derive every schema claim from the bytes rather than trusting it: column names and order, float64 dtypes, row_count_floor: 239 as an exact count, date_range 1959-02-28 to 1978-12-31, known_nulls: {} genuinely empty, and the observed ranges quoted in the column descriptions (returns 0.865–1.159, growth 0.967–1.033, inflation 0.995–1.024, per-capita 2.14e-4–2.66e-4, T-bill 0.984–1.008).
  • Confirm consumers lists exactly the repos that read each file today, and that no listed file path is stale.
  • Confirm the two hansen_singleton entries were removed from scripts/audit_annotations.yml and that no dataset now carries both an annotation and a manifest.
  • Check that builders/README.md's counts are re-derivable from the manifests, not asserted: 21 constructed, 14 with a builder (9 committed + 5 committed-frozen), 12 distinct builder paths, 7 unrecovered, and 13 builder files on disk. These were corrected today after Copilot flagged them; one of them had been wrong since before this wave.

5. Tracker consistency

  • On main, run the strict audit and confirm exit 0 with no migration_inconsistencies warnings.
  • Flip each of the two hansen_singleton entries in migration.yml to landed in a scratch checkout and confirm the strict audit fails. A tracker that cannot go red is not evidence when it is green.
  • Regenerate CATALOG.md and confirm zero diff. Confirm it reports 33 datasets, 33 read by lectures today, and that both new rows show constructed, ✅ verified, ✅ committed, and a non-empty Used by column.
  • Confirm migration.yml records all seven Track B datasets at status: repointed with cutover: null, and that every repoints[].pr names a PR that is genuinely merged.

6. Known blind spots

The session documented three. Verify each blind spot is real, and re-derive the counts — and note that the sweep producing a count often shared the blind spot it documents.

  • lecture-python.zh-cn is excluded from build_audit.py's SCAN_REPOS, so no automated check ever sees its reads. Confirm the exclusion, then confirm by hand that both zh-cn lectures read data-lectures on main and that this is recorded in the manifests' consumers — because if it is not recorded there, it is recorded nowhere.
  • The consumer sweep argued that because the files were created 2026-06-26, only the 88 of 277 org repos pushed since could reference them. Re-derive both numbers. Then test the argument's limits: the sweep covered default branches only, and the QuantEcon org only. Check non-default branches on the plausible repos, and check outside the org — wave B1′ found three live third-party course sites referencing python.quantecon.org/_static/lecture_specific/mle/fp.dta, so the assumption that consumers are in-org is known to be false for this family.
  • gh search code cannot match a bare URL and cannot see archived repos. Confirm this independently, and confirm the session's mitigation (Trees-API enumeration) actually covers archived repos.
  • The session's sweep loop silently dropped one repo because a file's last line lacked a trailing newline — caught only by comparing wc -l against grep -c ''. Re-run the sweep with a method that cannot truncate, and confirm the total is 88, not 87.

7. Decisions settled today

  • D3 — committed, not committed-frozen. Re-test per section 3.
  • The flip goes last. The work plan said "flip, then merge the sync PR"; the session found neither wave did that, citing zh-cn#249 merged 02:30:40Z and flip Flip the five wave B1' datasets to repointed #80 merged 02:36:49Z. Confirm those timestamps and that Flip wave B2' to repointed — the hansen_singleton pair (Track B) #83 likewise postdates #250.
  • The cache trap affects 9 of 11 repos. Re-derive from each repo's ci.yml, publish.yml and cache.yml on origin/main — the session read local clones, which can lag. Confirm lecture-wasm is the only immune one, and confirm lecture-dp's build-cache key is built only from environment-file hashes so content changes never invalidate it.
  • jb clean --html removes only _build/html and preserves .jupyter_cache. Verify against the pinned jupyter-book version actually used by CI, and confirm that version is what CI installs.
  • The historical trap instance. Confirm us_adult_heights.csv and japan_population_by_age.xlsx are present in the publish-2026aug06 and publish-2026aug07 release tarballs for lecture-python-intro and absent from publish-2026aug11.
  • lecture-stats is not a B2′ consumer. Re-check against origin/main, with a positive control proving the search pattern matches when present.

8. Deliberately not done

Confirm each was a decision with a record, not an oversight.

  • lecture-python.zh-cn keeps byte-identical copies of all seven datasets plus both builders, unreferenced. Recorded as Track X (D5) — confirm the record exists and that nothing in that repo reads them.
  • Filenames were not changed during migration (D6), including the redundant 1982/1983 pair. Confirm the deferral is recorded and note that the rename-proposal issue has not been filed — that one is a genuine loose end, not a decision.
  • lecture-python.notebooks is not listed in any consumers block, on the grounds that it self-heals after a publish. Confirm it regenerated (last push 2026-08-13T09:34:21Z) and carries the new URLs.
  • linkcheck has been red for 13 months, hiding two other broken links lecture-stats#62 (linkcheck red for 13 months, plus two genuine dead links) is unowned and out of scope.
  • The B2′ deletion (#1037) is deliberately not merged-and-forgotten: it will not take effect until the Monday rebuild plus a publish.

Where the reasoning lives

PLAN.md (roadmap, repoint rules, per-track detail) · AGENTS.md (working rules, the four ordering traps, manifest and builder contracts) · migration.yml (per-dataset status and PR trail) · builders/README.md and sources/README.md (coverage and the fetch-vs-sources/ rule) · QuantEcon/workspace-lectures#39 (Track B work plan and the corrections this wave produced) · QuantEcon/workspace-lectures#40 (deletion verification) · QuantEcon/workspace-lectures#41 (the build-cache defect) · and the PR descriptions above, which carry the byte gates and acceptance tests in full.

For the validator: work in a session that did not do this work. Do not use the tool named under "Bias to test for" except where a check explicitly says to run it. Re-derive counts rather than confirming them. Where a check can be run against a surface the original session did not exercise, do that too — the margin beyond the checklist is where regressions hide. Deliver: one comment on this issue with a per-item verdict (confirmed / confirmed with caveat / refuted / not completable, with evidence), a new issue for any regression found (do not bury findings in the comment), and leave the checkboxes to the issue owner unless told otherwise.

Activity

  1. mmcky commented on Aug 13, 2026

    @mmcky
    mmcky
    on Aug 13, 2026
    ContributorAuthor

    Independent validation — Phase 1, 2026-08-13

    Independence achieved: this validation ran in a session that did none of the original work. The §1 fetch checks ran on a remote cloud machine on a different network. Hashes were re-derived with openssl dgst and python hashlib (never shasum); comparisons used cut/diff/cmp, jq, and GraphQL (never the session's content_numbers() extractor, which was not invoked anywhere); tracker and catalog checks ran in fresh scratch clones against fresh depth-1 clones of all SCAN_REPOS; builder reruns used fresh venvs, including a deliberately divergent one (Python 3.14.6 / pandas 3.0.5) to probe env-sensitivity. Residual monoculture, stated honestly: everything except §1 still ran on the same physical machine as the original work (different toolchains, fresh venvs, authenticated API fetches — but same host). §1, where host independence matters most, is the section that ran remotely.

    Headline: no reader-facing regression found. Of the checklist's ~40 sub-checks: 27 confirmed, 7 confirmed with caveat, 3 refuted (one wrong manifest bound, one failed cross-check inherited from earlier waves, one over-strong tool rule — none reader-facing), 4 not completable in this pass (the post-Monday 404s, the day-later builder rerun, the Colab run, and "pinned environment" as written — the last because no pinned environment exists, which is itself a finding).

    One correction to this issue's own text, load-bearing for the follow-up pass: the deleted B2′ paths are per-lecture directories — _static/lecture_specific/hansen_singleton_1982/… and …_1983/… — not a flat hansen_singleton/. A first fetch against the flat path 404'd and briefly looked like a refutation; it was a never-existed path (and now serves as two extra soft-404 controls). The post-Monday 404 list must use the per-lecture paths.

    Note on timing: lecture-python.myst#1037 merged at 11:40:20Z, after this issue was filed; verdicts below reflect the post-merge state.

    §1 Reader-facing outcomes

    Check Verdict Evidence
    Five B1′ files still served, from another machine Confirmed Remote machine, dual-tool (curl + urllib, agreeing): all five 200 with exactly 14466 / 11210 / 12024 / 1024085 / 13738 bytes; never-existed sibling 404s with the site's real 9379-byte 404 page
    B2′ pair still served post-#1037 Confirmed (beyond checklist) At the corrected per-lecture paths: 200 with 11662 / 26084 bytes on python.quantecon.org — the cache-trap model holds after a same-day deletion too
    Post-Monday 404 re-fetch Not completable until after Mon 2026-08-17 03:00 UTC + a publish Deferred to the follow-up pass; list below
    EN hansen pages render Confirmed with caveat Static-HTML parse, not a browser: estimate values present as text_latex array blocks (there are zero HTML <table> elements — the checklist's "tables" are LaTeX arrays), e.g. 1982 row 1 & 2.0806 & 0.1551 & 0.9948 & …; 2 and 1 output figures respectively; zero error indicators (Traceback/HTTPError/URLError/output_error/empty outputs all 0); all data reads point at data-lectures, zero old-path refs
    zh-cn hansen pages render Confirmed with caveat Same method: numbers match EN cell-for-cell (one last-digit rounding diff, 42.503 vs 42.504); one benign findfont stderr warning per page on figure cells that still produced their images
    zh-cn /_notebooks/ (surface the session never checked) Confirmed Both 200 (application/x-ipynb+json, valid JSON); DATA_URL = …data-lectures/raw/main/lectures/hansen_singleton_198x_data.csv present; _static/lecture_specific/hansen_singleton zero hits with passing positive controls
    Colab launch-button run Not completable by an agent Needs an interactive human browser session
    Published PDF Confirmed with caveat Whole-book /_pdf/quantecon-python.pdf (58,136,028 B): all 4894 FlateDecode streams decompressed (full coverage); lecture_specific/hansen_singleton and lecture-python.myst/raw zero in raw and decompressed bytes; positive control found 8 data-lectures URI annotations (incl. the migrated fp.dta read), so the zeros are meaningful. Caveat: glyph-encoded body text without a link annotation is invisible to this method; jupyter-book emits code-cell URLs as URI annotations, so residual risk is low

    §2 Artifact integrity

    Check Verdict Evidence
    Hashes via non-shasum tools Confirmed urllib fetches; openssl and hashlib agree: 1982 3e9d4f37…9b (11662 B), 1983 dc5c1f8d…18 (26084 B); both match integrity.sha256 in the sidecars
    Three-way triangulation Confirmed Stronger than byte-identity: lecture-python.myst@a501f03 and lecture-python.zh-cn@main carry the same git blob SHAs (9b28375…, 491212f…) as data-lectures@main; all copies downloaded and cmp-identical. No zh-cn divergence — Chinese readers get the same bytes
    1982 strict subset of 1983, textually Confirmed Headers 3 vs 6 columns, shared columns at positions 1–3 in both; cut-extracted date/return/growth columns diff-identical at full precision (e.g. 1.0113599312297732); positive control: different columns do differ; no pandas anywhere
    Plain git, not LFS Confirmed git show origin/main:…csv | head -1 is a CSV header; check-attr filter unspecified; positive control: sources/** correctly shows filter: lfs; blob sizes match on-disk exactly

    §3 Builders (the highest-value claims)

    Check Verdict Evidence
    Reproduce under the pinned environment Confirmed with caveat — and the check as written is not completable Finding: no environment spec in data-lectures covers the builders. requirements.txt pins wbgapi/PyYAML/scipy only — no pandas — though both builders import it; CI never runs the builders; the original verified was measured against an unrecorded ambient anaconda (3.13.9 / pandas 2.3.3). The reproduction itself holds emphatically: fresh venv at 3.13.9/2.3.3 → both outputs byte-identical to the committed CSVs; a deliberately divergent venv (Python 3.14.6 / pandas 3.0.5) → also byte-identical. The gap is process, not fragility
    Day-later stability rerun Not completable today This pass's evening rerun (fresh venvs, fresh network fetch, hours after the original) is a genuine second data point: upstreams still reproduce the committed bytes. The true day-later run belongs to the follow-up pass. The real risk window is the monthly Ken French refresh: today's zip is stamped 202606
    Offline runs fail loudly Confirmed Both builders under a dead proxy: exit 1, URLError: Connection refused, zero files written — no silent cache or fallback path
    validate() rejects corruption Confirmed 13/13 specified corruptions raise (positive control: the good frame passes). Adversarial extra: the checklist's tbill-overwrite is caught by the tbill band before the identical-legs assert; the in-band converse (market := tbill, inside every band) was added and is caught by the positive-excess-return check — the defence is layered, not one assert
    Ken French provenance from unread sources Confirmed with caveat Zip header verbatim: "created using the 202606 CRSP database", Ibbotson until 202405 — confirming the manifests, but the manifests' quotation truncates the sentence's ICE BofA continuation (from 202406) while presenting it in quote marks. Licence hunt over all 761 distinct hrefs on the library page: zero terms/licence links (footer copyright only), so redistribution: permitted stands as a reasoned call; however an HTML developer comment — "All images and code are property of Ken French. Use in part or whole is illegal — except by permission…" — scoped to site images/code, appears on the main and factors pages and is unrecorded; #35 should see the quote

    §4 Records written

    Check Verdict Evidence
    Manifest schema conformance Confirmed with caveat Programmatic checker: all required fields present, enums valid, in both manifests. Caveat: 4 fields per manifest (source.doi/version/note, license.note) are absent from manifest-schema.yml but used in 10–20 of all 33 manifests — the strawman schema lags established practice, not a hansen-specific deviation
    Schema claims from bytes Refuted in one digit, confirmed otherwise Columns/order/dtypes/239-exact/date-grid/zero-nulls and four of five quoted ranges re-derived exactly. The 1983 manifest's consumption_per_capita lower bound "2.14e-4" is wrong: actual min 2.1255e-4 (1960-12-31) → 2.13e-4 at 3 s.f. Bonus re-derivations confirm the 54%-of-months and positive-mean-excess claims (130/239 = 54.4%, +0.00245)
    Consumers accurate Confirmed All four listed consumer files on main carry live DATA_URL reads (line numbers verified); reverse search found no missing consumer — the notebooks mirror is deliberately excluded per its self-heal rule, and zh-cn's old _static copies are not consumers (they are Track X, see §8)
    Annotations retired Split: hansen confirmed; cross-check refuted hansen entries absent with passing positive controls. But the "no dataset has both an annotation and a manifest" invariant fails for 8 earlier-migrated datasets (assignat.xlsx, caron.npy, chapter_3.xlsx, dette.xlsx, fig_3.xlsx, longprices.xls, mpd2020.xlsx, nom_balances.npy) — violating audit_annotations.yml's own header rule; the entries are inert in build_audit.py and no check flags coexistence, so this rots silently. Inherited from earlier waves, surfaced by this one
    builders/README counts Confirmed All six numbers re-derived exactly from the manifests: 21 constructed, 14 with builder (9+5), 12 distinct paths (the two known doubles), 7 unrecovered, 13 files on disk — and the one unreferenced builder's three outputs are exactly the repo's only unmanifested files (36−33)

    §5 Tracker consistency

    Check Verdict Evidence
    Strict audit green Confirmed build_audit.py scan --strict against fresh depth-1 clones of all 8 SCAN_REPOS: exit 0, empty problems block including migration_inconsistencies
    Audit can go red Confirmed Each hansen entry flipped to landed (scratch clone): exit 1 with migration_inconsistencies naming exactly the flipped dataset, both directions; reverted; final rerun green
    CATALOG regenerates clean Confirmed Exit 0, git diff empty; header 33 datasets · 33 read by lectures today; both hansen rows constructed · ✅ verified · ✅ committed with non-empty Used-by (myst + zh-cn)
    migration.yml Track B state Confirmed All seven at status: repointed, cutover: null; every repoints[].pr verified MERGED via the API with timestamps

    §6 Known blind spots

    Check Verdict Evidence
    zh-cn outside automation Confirmed SCAN_REPOS = 8 repos, zh-cn absent (deliberate, commented); both zh-cn lectures read data-lectures on main (line numbers verified); both manifests record zh-cn in consumers — so it is recorded there, and only there
    88-of-277 re-derived Confirmed 277 exactly; 88 exactly (JSON-native jq — cannot drop a final line); premise verified: the files' creation commit is a3e8443 2026-06-26T04:09:22Z (#926). Limits are real: 7 stale myst branches + 31 zh-cn branches (30 translation-sync-* + gh-pages) still carry old files — expected for stale branches; out-of-org consumers re-confirmed as a phenomenon (the fp.dta URL verbatim in elliottserna/econometrics, quecheny/quecheny, mcherculano/ECON5129); hansen-specific out-of-org search: genuine best-effort zero
    gh search code blindness Split: archived-repo blindness confirmed decisively; bare-URL rule refuted as stated A token verifiably present in archived high_dim_data is invisible to gh search code (org and global) while the Trees API returns its full tree (truncated: false) — mitigation covers archived repos. But the unquoted URL query returned 19 real hits today (tokenized AND, verified against raw contents); it is the quoted exact-phrase form that returns the confident zero. The trap is real but narrower: quoting the URL — the careful instinct — is what produces the silent zero, and a tokenized hit-list is still not a sweep (archived repos remain invisible to it)
    Truncation-proof recount Confirmed — 88 Second method (GraphQL cursor pagination → python json): 277 / 88; symmetric differences with the jq method empty on both name-sets

    §7 Decisions settled

    Check Verdict Evidence
    D3 committed, not committed-frozen Confirmed Per §3: byte-identical reproduction in two environments incl. pandas 3; validate() layered
    The flip goes last Confirmed zh-cn#249 02:30:40Z → #80 02:36:49Z; zh-cn#250 08:34:10Z → #83 08:37:58Z (3m48s later). Both waves: sync before flip
    Cache trap 9-of-11 Confirmed with caveat All workflow files fetched from origin/main, not local clones. Exactly 9 affected; 8 restore the latest cache.yml artifact with no content key (dawidd6/action-download-artifact@v21, identical block); lecture-dp's key is build-${hash(environment.yml)}-${hash('')}- — content never keyed, confirmed verbatim from actions@main; lecture-wasm immune (clean checkout, no cache); data-lectures N/A. Caveat: "11" is not the workspace manifest family (manifest = 9 lecture + 2 infra); the 11 adds zh-cn and lecture-stats from outside it. Two observations beyond the checklist: the canary test-actions-lecture-intro has the trap in its CI-preview path (publish path clean), and ws#41's "lecture-dp persists until environment.yml changes" looks overstated — lecture-dp's weekly 0 2 * * 1 cache rebuild saves a fresh entry the prefix restore-keys will match, so it likely gets a Monday clean slate at 02:00
    jb clean --html semantics Confirmed with caveat Caveat: the pin is a floating range jupyter-book>=1.0.4post1,<2.0, resolving to 1.0.4.post1 today only because no newer 1.x exists on PyPI. Against that tag's source: --html → remove_option → rmtree(_build/html) and nothing else; .jupyter_cache touched only by the no-flag default (which explicitly spares it) and --all
    Historical trap instance Confirmed The publish artifacts (deployed _build/html tarballs, not source archives): both files present in publish-2026aug06 and publish-2026aug07, absent from publish-2026aug11; 806 entries in each; positive controls pass
    lecture-stats not a B2′ consumer Confirmed Trees API (87 paths, not truncated) zero for hansen_singleton, and stronger: full-content tarball grep zero; positive controls — same pattern hits the myst lectures, and a data-lectures content grep hits lectures/mle.md (the #61 prose link) in the same tarball

    §8 Deliberately not done

    Check Verdict Evidence
    Track X (D5) recorded; zh-cn reads nothing locally Confirmed Record in QuantEcon/workspace-lectures#39 (consumer table + D5 row, incl. "sync is .md-only… stranded permanently unless deleted deliberately"); all 7 data files + both make_data.py present in zh-cn@main and blob-SHA-identical to their sources; whole-tarball grep: nothing reads the local copies (every read is a data-lectures URL, line numbers verified; positive controls pass). Three validation agents independently converged on the zh-cn copies and all found them consistent with this record
    D6 recorded; rename issue unfiled Confirmed Deferral recorded in ws#39 (D6 row) and migration.yml:675–676; scan of all data-lectures issues finds no filename-rename proposal — the loose end stands exactly as described
    notebooks mirror self-healed Confirmed pushed_at 2026-08-13T09:34:21Z exactly; both hansen notebooks on main carry data-lectures URLs, zero old-path refs, positive controls pass
    lecture-stats#62 unowned Confirmed OPEN, assignees: []
    B2′ deletion not merged-and-forgotten Confirmed (state updated) #1037 merged 2026-08-13T11:40:20Z — before the Mon 03:00 UTC rebuild, so both waves ride the same clean rebuild. No publish since the merge: newest release publish-2026aug13b (08:28:25Z) is tagged on a501f03, the pre-deletion repoint. The post-Monday pass is the first surface where #1037 can take effect

    Findings routed to their own issues (per this issue's delivery contract)

    1. Manifest and provenance corrections (the 2.14e-4 bound; the truncated Ken French quote; the unrecorded images-and-code site comment for the Licensing: record-and-track policy, and the per-dataset inventory for migration #35 licence review; the missing builder environment spec) — Validation findings from #84: two manifest corrections and a missing builder environment spec #85.
    2. The 8 stale audit_annotations.yml entries and the unenforced manifest-or-annotation rule — 8 migrated datasets still hold audit_annotations.yml entries; nothing enforces the manifest-or-annotation rule #86, with a suggestion that --strict enforce the invariant so it can never rot silently again.

    Corrections to records living elsewhere: the lecture-dp wording and the canary's CI-preview trap are posted as a comment on QuantEcon/workspace-lectures#41; the gh-search bare-URL rule in the workspace docs should be narrowed to the quoted/unquoted distinction per §6 above.

    Deferred to the post-Monday pass (after Mon 2026-08-17 03:00 UTC rebuild and a subsequent publish)

    • Re-fetch all seven deleted paths on python.quantecon.org expecting 404: the five B1′ paths above plus _static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv and _static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv (per-lecture directories — not the flat path in this issue's §6 wording). The zh-cn site should still serve its copies (Track X) — a zh-cn 404 would be a different, new problem.
    • The day-later builder rerun (watching for a 202607+ Ken French vintage).
    • If any of the seven still 200s after rebuild + publish: per this issue's instruction, that is a new issue, not a checkbox.

    Checkboxes left to the issue owner, per the delivery contract.

  2. mmcky commented on Aug 14, 2026

    @mmcky
    mmcky
    on Aug 14, 2026
    ContributorAuthor

    Phase 1's deferred checkbox is now answerable: the five (and the other six) are 404

    This issue's §1 carries an unchecked item — "after the next cache.yml clean rebuild (Mon 2026-08-17 03:00 UTC) and a publish following it, re-fetch the same five. Expected: 404. If any still 200s, the session's model of the trap is incomplete."

    The rebuild was run manually today rather than waiting for Monday, so the item is resolved three days early. The model held. All five wave B1′ paths, plus all six from B2′, now return 404 (text/html, 9,379 B, byte-identical to a never-existed control fetched in the same pass), and none appears in the publish-2026aug14 release tarball. Nothing needs a new issue on this count.

    Sequence: cache.yml run 31757133369 00:22:41Z → 02:17:29Z, then publish-2026aug14 04:54:46Z → 05:10:52Z, verified 05:12Z. Full evidence in QuantEcon/workspace-lectures#39; the cause issue QuantEcon/workspace-lectures#41 has been reopened, since it was auto-closed by a closing keyword in an unrelated PR body and the prune step has landed in no repo.

    Two things this validation's own bias section asked for, which today's pass supplied:

    • The single-toolchain monoculture is partly broken now. The build-cache artifact was read two independent ways — a browser download extracted to local disk, and an HTTP range read of the zip central directory via Python zipfile — and they agreed exactly (0 of 11 present, 27 directories, 91 files).
    • The changelog widget struck again, and was caught this time. Baselining showed every lecture page gaining one data-lectures reference; the extra hit is the widget listing repoint PR titles, all of which contain the words "data-lectures". Counting URLs rather than the bare string makes live and freshly-built pages agree exactly. Worth adding to this issue's list of extractor hazards alongside the 645-value false diff.

    Still open from this validation and untouched by today's work: #85 (two manifest corrections, the pandas pin) and #86 (the 8 stale annotation entries). The §3 item asking for a re-run of both hansen builders a day or more later against live upstreams also remains open.

  3. mmcky commented on Aug 16, 2026

    @mmcky
    mmcky
    on Aug 16, 2026
    ContributorAuthor

    §3's day-later builder re-run: done, both reproduce byte-identically

    Run 2026-08-17, four days after the builders landed in #82 (merged 2026-08-13T07:18:08Z) and 3.4 days after the Phase-1 validation's own reproduction — so the "a day or more later" condition is met with margin.

    Run in a throwaway shallow clone rather than in a working tree, because both builders write straight into the published tree (PUBLISHED_DIR = <repo>/lectures, then frame.to_csv(...)). Re-running them in place silently dirties lectures/hansen_singleton_198*_data.csv, which would be easy to commit by accident.

    builder output sha256 vs manifest
    builders/hansen_singleton_1982_data.py 239 rows × 2 cols, 1959-02-28 .. 1978-12-31 3e9d4f37c31dbcab… — match
    builders/hansen_singleton_1983_data.py 239 rows × 5 cols, 1959-02-28 .. 1978-12-31 dc5c1f8dac4b50ab… — match

    Both exited 0, and git diff -- lectures/ in the clone is empty — the strongest form of the check, since it compares against the committed blobs rather than against a recorded hash. The hashes also match the integrity.sha256 in each manifest, so the two records agree.

    Environment: Python 3.13.9 / pandas 2.3.3 — which is now the pinned environment rather than an ambient one, per the requirements.txt pin in #89. That was the substance of the third finding in #85: the earlier verified status was measured against whatever python happened to be present. This run was made against the version the pin names.

    Both upstreams answered and neither had revised the 1959-1978 window. The Ken French zip still reports This file was created using the 202606 CRSP database. — the same vintage recorded in the manifests on 2026-08-13.

    One caveat on what this does and does not establish. Because the Ken French vintage has not rolled since the migration, this is a same-vintage reproduction: it tests FRED revision drift over the window, but it does not test the monthly Ken French refresh, which is the real shelf-life risk the validation named. A repeat once the header moves past 202606 would be the check that retires that question. Worth scheduling rather than treating as closed.

    What remains on this issue

    Only §1's in-Colab run — opening a repointed notebook from the published page's own launch button and executing the data cell. That needs a human with a browser; the Phase-1 validation already recorded it as not completable by an agent, and nothing has changed that.

  4. mmcky
    added a commit that references this issue on Aug 16, 2026

    Correct two manifest records and pin the builder environment (#89)

    9ea316c

  5. mmcky commented on Aug 17, 2026

    @mmcky
    mmcky
    on Aug 17, 2026
    ContributorAuthor

    Partial progress on the last open box, from the #96 validation: the data-loading cells of both published advanced.myst notebooks were extracted verbatim and executed in a fresh Python 3.14.7 venv — macro_q (260, 14), mich_m (507, 5), DATA_BUNDLE annual/monthly/quarterly at (95,4)/(334,3)/(270,4) all load from the new data-lectures URLs. Also verified: the pages' Colab launch buttons open the .notebooks mirror copies, which are byte-identical to _notebooks/ and carry the new URLs.

    Still not a true in-browser Colab run — that continues to need a human browser, so the box stays open unless the local-execution evidence is judged sufficient.

  6. mmcky commented on Aug 19, 2026

    @mmcky
    mmcky
    on Aug 19, 2026
    ContributorAuthor

    The last box is now unblocked: both Colab blockers are live-published as of publish-2026aug19 (2026-08-19). The published _notebooks/five_preferences.ipynb verified in-place: the dataBHS.csv read from data-lectures (the wave-C2 repoint), zero residual loadmat, and the new usetex=bool(shutil.which('latex')) guard from QuantEcon/lecture-python-advanced.myst#378 — so Colab's latex-less runtime falls back to mathtext instead of dying at the first rendered figure. One caution for the run: make sure Colab's runtime doesn't hit the jax 0.11.1 fori_loop hang — not applicable here (five_preferences uses no jax), just noting the family context from QuantEcon/workspace-lectures#49. A clean human "Run all" of the downloadable notebook closes this issue.

  7. mmcky commented on Aug 31, 2026

    @mmcky
    mmcky
    on Aug 31, 2026
    ContributorAuthor

    Closing — the last two boxes are done.

    Colab "Run all" from the published launch button — DONE 2026-09-01, by hand. Both notebooks the launch buttons resolve to were opened in Colab and executed to completion: lecture-python.notebooks/hansen_singleton_1982.ipynb (the Track B repoint this issue validates) and lecture-python-advanced.notebooks/five_preferences.ipynb (the wave-C2 dataBHS.csv read plus the usetex guard from QuantEcon/lecture-python-advanced.myst#378 named in the previous comment). So the reader's runtime resolves the data-lectures URLs, not only CPython-in-CI.

    Builder re-run "a day or more later" — DONE 2026-09-01, 19 days after the original verification. builders/hansen_singleton_1982_data.py and builders/hansen_singleton_1983_data.py were re-run in a fresh venv under the pinned pandas==2.3.3 (CPython 3.14.7), fetching FRED and the Ken French library live. Both reproduce their committed CSVs byte for byte — 3e9d4f37…9b (1982) and dc5c1f8d…18 (1983), matching each manifest's integrity.sha256. The 1959–1978 window has held across the gap, so verified needs no shelf-life caveat yet; the upstream-freshness dashboard proposed in #40 is where that would be watched routinely.

    40 of 40 boxes closed.

  8. mmcky
    closed this as completedon Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions