Skip to content

source-health oracle: reach check fails at random in CI #2287

Open

Description

@KyleAMathews

Problem

The generated histories refine the model property in packages/db/tests/source-health-oracle.test.ts sometimes fails its reach check in CI. No law fails. The error is:

Error: source-health.history: generated histories reached these states fewer than 1 times, so their laws were not enforced there: markReady on a ready run (0), reload's insert of a kept row throws (0)

Evidence

  • PR fix(db): keep a live query running when a lazy demand fails during a truncate replay #2231, run 40735418214, job Test (db-1), seed 727029764. All 8251 tests passed. Only this reach assertion failed.
  • Branch investigate/progressive-network-first, run 40719621050, has the same error.
  • Locally, three full runs of the file with TANSTACK_DB_ORACLE_SEED=727029764 pass (504/504). The miss does not come from the seed alone. It appears to depend on timing under CI load (2 workers, sharded).

Cause (not verified)

The property runs 60 histories. A history reaches these two states only if a random reach prefix (from 6 prefixes, fc.option(..., { freq: 6 })) or random steps produce them, and the async order agrees. With 60 runs, a run that reaches neither state is possible.

Possible fixes

  • Add a fixed history for each of the two states that always runs through the same driver and comparison, so reach does not depend on chance.
  • Or make the generator choose a reach prefix in a fixed order for the first runs, so each declared state gets at least one history.
  • Or record the states as probabilistic in the coverage map, with a maintainer decision.

The oracle came from #2168, and the reach requirement came from #2278. PR #2231 does not change this file.

This issue was written by Isaac.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions