Skip to content

Backfill reform_validation.json for buildo (Modal reproduction) - #109

Merged
PavelMakarchuk merged 2 commits into
PolicyEngine:mainfrom
DTrim99:reform-validation-modal/populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z
Aug 3, 2026
Merged

Backfill reform_validation.json for buildo (Modal reproduction)#109
PavelMakarchuk merged 2 commits into
PolicyEngine:mainfrom
DTrim99:reform-validation-modal/populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z

Conversation

@DTrim99

@DTrim99 DTrim99 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Puts current-release data on the External-checks tab (/populace/reforms), which has shown the empty state for Build M/O since the scheduled backfill broke on 7/21.

  • Produced on a 64GB Modal container running the identical producer path (tools/reform_validation/backfill.py, unmodified, batch-checkpointed via the Chunk the baseline-levels batch so the reform-validation backfill survives the hosted runner #107 chunked-levels path) at the release-exact policyengine-us 1.764.6 / policyengine-core 3.26.11; provenance in the file's _backfill_note.
  • 236 rows — the complete post-populace#513 payload: the first Census state SPM poverty-rate suite (104 percent-unit rows; rendering landed in Render percent-unit rate backtests correctly on the reforms page #108), the six tracker-informed state credit/deduction bills, three federal benchmark rows (ARPA CTC, ARPA childless EITC, CBO all-rates-+1pt), and the $1,200/person UBI mechanical check (matches Census person mass to −0.01%). Producer at populace 51fbd3f.
  • gen-reform-overrides.mjs regenerated (5 overrides), bun run lint clean, reforms tests 9/9.

Workflow status caveat (updated 7/27): #107's chunking was necessary but not sufficient — the scheduled backfill still dies on ubuntu-latest (every 6-hour run through 7/27; even a single 16-spec levels chunk gets the runner shutdown signal ~5.5 min in). One baseline simulation plus the producer import stack no longer fits the 7GB hosted runner. So this override is currently the only path to Build O data, and future releases need either a manual Modal run like this one or the workflow re-pointed at Modal for the simulation step (GH Actions keeps the trigger/commit role; needs MODAL_TOKEN secrets on the repo) — proposal to follow separately.

The failing Vercel check is the fork-deploy authorization gate, not a build failure — needs a team member's authorize click (or ignore; the app deploys from main).

🤖 Generated with Claude Code

@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown

@DTrim99 is attempting to deploy a commit to the PolicyEngine Team on Vercel.

A member of the Team first needs to authorize it.

DTrim99 and others added 2 commits July 27, 2026 09:12
…loss100-22bd902-20260722T232627Z (Modal reproduction)

The scheduled workflow was OOM-broken until PolicyEngine#107 merged (its 18:26Z run
today still ran pre-fix code), so this override was produced on a 64GB
Modal container running the identical producer path
(tools/reform_validation/backfill.py, unmodified) at the release-exact
policyengine-us 1.764.6 / policyengine-core 3.26.11; provenance in the
file's _backfill_note. 226 rows - the first payload carrying the Census
state SPM poverty-rate suite (104 percent-unit rows, rendered by PolicyEngine#108).

Producer predates populace#513, so the State reform suite is the
original 8 rows. Since the scheduled workflow skips releases that
already have an override, the expanded-registry regeneration for this
release comes from a follow-up Modal run against post-#513 main
(superseding commit to this file); Build P onward picks up the expanded
registry automatically.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regenerated on Modal at the same release-exact engine versions with the
producer at populace 51fbd3f (post-#513 registry expansion): adds the six
tracker-informed state credit/deduction bills, three federal benchmark
rows (ARPA CTC, ARPA childless EITC, CBO all-rates-+1pt), and the
$1,200/person UBI mechanical check to the buildo override. Chunked-levels
path from PolicyEngine#107 exercised end-to-end on this run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@DTrim99
DTrim99 force-pushed the reform-validation-modal/populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z branch from 70c2d89 to 9a3bc47 Compare July 27, 2026 13:13
@DTrim99

DTrim99 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Superseded the payload in-place with the post-populace#513 regeneration (236 rows: +6 tracker-informed state bills, +3 federal benchmark rows, +the UBI mechanical check). Same release-exact engine versions; producer at populace 51fbd3f; the #107 chunked-levels path ran end-to-end on this Modal run. This is now the complete Build O artifact — ready to merge, and the External-checks tab has been empty for the current release since 7/17, so review would be appreciated. Note the scheduled workflow is still failing post-#107 (separate cause, investigating now — this PR unblocks the dashboard regardless).

@PavelMakarchuk PavelMakarchuk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified locally:

  • Registry entry matches gen-reform-overrides.mjs output exactly (re-ran the generator on the branch — zero diff) and keeps alphabetical order.
  • Schema matches the consumer: top-level keys identical to the buildj override; the new per-row unit field is the #108 addition, and the parser defaults missing unit to currency, so the 58 rows without it are fine.
  • Claimed counts are accurate: 236 rows, 104 percent-unit Census state SPM rows.
  • Independently verified the UBI mechanical check: 340,110,988 × $1,200 = $408.133B matches the stored anchor exactly; the populace total of $408.096B is −0.009% off, matching the claimed −0.01%.
  • Reforms tests 9/9 and tsc --noEmit clean on the branch.

The Vercel failure is just the fork-deploy authorization gate. LGTM — this unblocks the empty External-checks tab now, and remains fine to merge even if #112 lands first.

@PavelMakarchuk
PavelMakarchuk merged commit 6b75f6f into PolicyEngine:main Aug 3, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants