Reduce dependency on TLA runs for CI - #8205
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR separates lightweight PR-gated TLA verification from expensive weekly verification.
Changes:
- Adds shallow simulation and trace-validation checks.
- Moves long and continuous verification to weekly schedules.
- Runs shallow checks on hosted runners.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Summary | Review notes |
|---|---|---|
.github/workflows/tla-shallow.yml |
Adds PR-gated shallow verification jobs. | Include the workflow itself in path filters and document the workflow and trigger behavior in README.md. |
.github/workflows/long-verification.yml |
Restricts deep verification to weekly schedules. | Update documentation describing PR, manual, and label-based triggers. |
.github/workflows/ci-verification.yml |
Restricts continuous verification to weekly schedules. | Update documentation so it no longer promises PR or manual checks. |
Suppressed comments (3)
.github/workflows/tla-shallow.yml:99
tests.shforwards these arguments directly toctest, whose--timeoutoption requires a numeric value. With--no-compress-outputas the next token, this command exits during option parsing, so the raft scenarios never run and no traces are produced; pass an explicit timeout such as--timeout 360.
./tests.sh -VV --timeout --no-compress-output -T Test -L raft_scenario
.github/workflows/tla-shallow.yml:107
- Because
traces/consensusalready exists,mv ../build/consensus traces/targets the existingtraces/consensusdirectory and fails rather than placing the generated files there. Create only the parenttracesdirectory before movingbuild/consensus, so the resulting path is the one consumed by the next command.
mkdir -p traces/consensus
mv ../build/consensus traces/
.github/workflows/tla-shallow.yml:39
- The job is capped at 10 minutes, but this simulation omits
--max-seconds, sotlc.pyuses its 1200-second default. A run lasting 10-20 minutes will be killed by the job before TLC reaches its configured timeout; set a limit compatible with the job (or increase the job timeout) so failures are reported and traces can be uploaded predictably.
./tlc.py --trace-name consistency-simulation sim --num 500 --depth 50 consistency/MultiNodeReads.tla || status=$?
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Amaury Chamayou (achamayou)
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It appears that there are either shallow or deep bugs in the TLA.
If the bug is shallow Simulation fails on it quickly.
If the bug is deep it is extremely expensive to wait for either Model Checking or Simulation to prove its absence.
Hence I think we should block CI only on the shallow bugs.
And rely on periodic costly deep checking for the proof of absence.
The shallow runs are cheaper and so should be fine to run on small github runners which also seem to be slightly more reliable and quick to spin-up.
They both only trigger on PRs which make changes to raft.h or the TLA directory.