Repository navigation
Conversation
The hints were medians of eight four-shard runs from 2026-10-28, from before the 10 s reconciler ticks, plus estimates for scenarios added since. Several had drifted far from what Kind now measures: the four inline-execution recovery scenarios ran in 49-173 s against estimates of 120-215 s, the orphaned-restore-pod GC in 127 s against 98, and three forced-drain scenarios had no hint and took the 30 s default for 53-63 s. Partitioned by them, twelve shards put up to 7 minutes on the slowest shard against 3.6 on the lightest. Each hint is now the median of the successful twelve-shard Kind runs of Pull CI and Main CI from 2026-11-02T22:15Z to 2026-11-03T10:00Z: 61 runs, 29-51 samples for each of the 133 scenarios in the inventory. Against those medians, the twelve-shard partition's slowest shard drops from about 7.0 to 5.6 minutes, every shard within 0.1 minute of the others.
Every Kind CI shard's baseline bring-up takes about 220 s from Tilt's start to its last pod, and 608 timelines from the last day show two waits on that path: - install-space-shim, a serial local resource, held every other resource for its 19 s: Tilt runs nothing beside one, and the Rust artifacts resource also waited for it. - space-seed-publish's image ran build-seed.sh, which builds the seed's rootfs from the Rust artifacts. Tilt holds a resource's image build until its resource_deps are ready, so that build started only once the internal API was up, at about 126 s, and the seed's Job, the bring-up's last step, waited until about 192 s. install-space-shim now runs in parallel and the Rust artifacts no longer wait for it; space-api's dependency on it still orders it before the first sandbox create. A new space-seed-build resource runs build-seed.sh as soon as the Rust artifacts exist, and space-seed-publish depends on it; its image build then finds the seed's inputs unchanged and skips the rootfs.
A pull request that changes a crate most of the workspace depends on spends 4-7 minutes in the Kind Rust producer, and every Kind shard waits for it. The producer restores main's build state, but builds the release profile without incremental compilation, so each crate it rebuilds because a dependency changed is optimised again from scratch. More cores do not help: on 32 instead of 16, the same 150-crate rebuild took 3m31s instead of 3m47s. build-rust-artifacts.sh gains SPACE_KIND_RUST_INCREMENTAL_CODEGEN: 1 turns on CARGO_PROFILE_RELEASE_INCREMENTAL in incremental mode, so rustc keeps its incremental data in the target directory and a rebuilt crate reuses the code generation its change did not touch. The default 0 keeps local builds as they are. rust-state-cache.sh keys the state on it, so a state built one way is never restored for the other. Kind CI's producer and Main CI's kind_rust_state both set it.
Collaborator
Author
|
Measurement done; closing this disposable trial. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disposable trial: the three open Kind CI PRs (#1391, #1393, #1394) on current main, to measure Pull CI with all three. The first push saves a Kind Rust build state on this PR, standing in for main's kind_rust_state. Later pushes measure a change with no Rust and one with Rust. Deleted after the measurement.
🤖 Generated with Claude Code