Repository navigation
Add competition schema and durable FRC import worker - #83
Conversation
Backfill missing team parents, constrain alliance stations, and apply the migration atomically. Restore the existing one-to-one registration relation and verify current-season report preservation and rollback against disposable PostgreSQL.
Store provider cache validators independently per endpoint and deduplicate queued work by kind and target. Verify existing-data preservation, retry defaults, and transactional rollback of validators.
Import teams, seasons, districts, rosters and canonical matches with atomic cache updates. Persist recurring jobs with renewable leases and derive tournament gaps while preserving scouting links.
Add a pinned container build, readiness checks and graceful shutdown for the separate worker service. Keep database migrations owned by the API release and verify worker imports in CI.
Use manual redirect handling so Bun accepts TBA HTTP 304 responses without forwarding authentication to redirects. Exercise real HTTP responses and create isolated team records required by the competition schema in Server integration fixtures.
Queue refresh jobs instead of writing participant assignments from API requests. Preserve worker leases and retries so scouting reconciliation remains enforced by one importer.
Use complete timing sequences or imported display order before format-aware bracket ordering. Cover alternating historical series and preserve double-elimination numbering.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
🚅 Deployed to the lovat-pr-83 environment in lovat
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: caabbc162c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await prismaClient.importJob.upsert({ | ||
| where: { kind_targetKey: { kind: "matches", targetKey: tournamentKey } }, | ||
| // Preserve the worker's active lease, refresh cadence and retry backoff. | ||
| update: {}, | ||
| create: { kind: "matches", targetKey: tournamentKey, runAt: new Date() }, |
There was a problem hiding this comment.
When a scouting client submits a report before its TeamMatchData row has been stored, both addScoutReport.ts and addScoutReportDashboard.ts await this function and immediately repeat the lookup, assuming the await means the import finished. This implementation only inserts a job and returns, so the second lookup normally still misses and rejects the report with 404 even when the match exists in TBA; an already-scheduled job may not even run immediately because update: {} preserves its future runAt. The async handoff needs a pending/retry flow in these write endpoints, or it must otherwise preserve the prior guarantee that the row exists before returning.
Useful? React with 👍 / 👎.
This branch was successfully deployed
1 participant
What changed
Lovat needs official competition data alongside scouting reports. This PR adds the shared season/district/team/event/match schema and a Bun/Hono worker that imports TBA metadata, rosters, results, alliance selections, awards, avatars, and event weeks. Existing report keys are preserved; imports reject participant changes that would orphan scouting reports.
PostgreSQL stores endpoint validators and recurring jobs with renewable leases and retry delays. Match refreshes also rebuild inferred lunch, overnight, transition, and delay gaps. The API now queues refreshes rather than running a competing importer; it returns stored data until the worker completes the job.
This is the infrastructure prerequisite for #82. Public website pages and public scouting aggregates are reviewed separately there. Other providers, practice matches, rankings, and district points remain future work.
Deployment and merge order
Merge this PR first. Verify production's recorded baseline, take a backup, and rehearse the five additive migrations on a production clone. The API owns migration deployment. Start the worker only after migrations finish, using repository root
/,/apps/worker/railway.json, the environment's private database reference,TBA_KEY, andWORKER_CONCURRENCY=2. Production does not currently have a configured worker instance.See
docs/frc-schema-worker-rollout.mdfor the exact deployment order, smoke tests, and recovery limits. A healthy worker is required after the API's refresh handoff. Historical backfill is optional; current-season imports start automatically.Verification
Production read-only preflight confirmed
0_baselineis recorded and the new match/import tables are absent. API root/config/Docker settings already match the rollout guide; production still needs a backup/rehearsal and a configured worker. No production data or settings were changed.Preview database preservation
Restored the current #82 snapshot into a separate
lovat_preserveddatabase in Railwaylovat-pr-83. Only #83 API/worker connections were changed. Verified matching 115,227 reports, 4,213,011 actions, and migration history. The copy also contains 1,602,886 team/match slots, 269,326 matches, 4,181 tournaments, 35 seasons, and persisted import jobs. Both original databases and the original September backup remain untouched. A checksum-verified recovery dump is saved outside the repository at/Users/jackshim/Documents/robotics/lovat-database-backups/split-pr-82-2026-10-12.dump.The new environment is still an ephemeral Railway PR preview: its volume survives deployments, but closing/merging the PR may remove it. #82 retains its independent populated database and the recovery dump survives preview cleanup. Imports continue independently in each copy.
Final preview validation: #83 API, worker, and Website deployments are SUCCESS; API
/statusreturns 200 and the worker passed Railway/health. Repository checks and all relevant Server/Worker/Monorepo CI jobs pass. Swift CodeQL jobs are still running; there are no failed checks at this point. The inherited Website preview build context was corrected only in #83 Railway settings; no unrelated Website code was added to this PR.Production rollout validation
With maintainer authorization, took a fresh private production backup and rehearsed all five migrations on a local restore. Deployment and repeat deployment succeed, migration status is up to date, and schema diff reports no differences. Comparing original columns against a separate untouched restore verifies every original row is identical: 116,536 reports, 4,281,592 actions, 258,847 team/match slots, and 1,035 tournaments. The backup remains outside the repository. All PR CI, CodeQL, and preview deployment checks are now green. The production worker is staged without a source and will start only after API migrations are verified.
Production rollout completed: merged as
a0aacdc3. The production API applied all five migrations and passed/status; the dedicated production worker followsmain, uses the private production database, and passed Railway/health. The API refresh helper queued2026casnf; the worker completed it with 91 canonical matches, no retry/error, and a future refresh time. Production still contains 116,536 reports and 4,281,592 actions. Current-season district/roster/detail/match imports are running. One tournament-details response (2026catt) failed provider validation and remains queued for automatic retry; other imports continue. #82 is retargeted to main and remains open with no infrastructure changes in its diff.