Repository navigation
test(nix): guard root dependency module identity - #64111
albertominetti wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
The tests aren't wired to what they guard against yet. I installed with the pre-#56889 Nix filters (--filter '!./' plus cli/desktop/app). On that layout the theme→spinner and single-OpenTUI tests fail with the original hex.replace is not a function crash, so they do catch the bug. The new CI step, though, runs them after the full bun install from setup-bun, not the filtered install in nix/node_modules.nix. There they pass no matter what the Nix install does: reverting the filter to !./ would still be green.
The two peer-context tests passed on both layouts. They guarded against the dedupe script from #56881, which this PR no longer has. They also fail if the lockfile ever dedupes Zod or the OpenTelemetry API. Details are inline.
Public API: unchanged. This only adds a test file and a CI step.
Related: #56856
| run: bun run lint:effect-simplifications | ||
|
|
||
| # nix/scripts is not a workspace, so turbo never runs its tests. | ||
| - name: Test Nix install layout |
There was a problem hiding this comment.
This runs after setup-bun's full bun install --frozen-lockfile, so it tests the dev layout rather than the Nix one. I checked locally: with the pre-#56889 filters (--filter '!./' --filter ./packages/cli --filter ./packages/desktop --filter ./packages/app) these tests fail with the original hex.replace is not a function. With the full install they pass whatever nix/node_modules.nix does, so a revert of the filter would stay green here. For this to guard the Nix install, it needs to run against that install. One way is to run it inside the derivation after the bun install in nix/node_modules.nix; check that this doesn't change the output hash. Another is a CI step in a clean checkout that uses the same filters as the derivation.
| } | ||
| }) | ||
|
|
||
| test.each([ |
There was a problem hiding this comment.
These two cases passed on the old filtered layout too, so they don't cover #56856. They guarded against the dedupe script from #56881, which is no longer part of this PR. They also only pass while the lockfile has several Zod / @opentelemetry/api versions under one package version (some(size > 1)). A lockfile update that dedupes those would fail CI even with no regression. I'd drop them and keep the theme→spinner and single-OpenTUI checks.
There was a problem hiding this comment.
Dropped both peer-context cases. The file now keeps only the theme-to-spinner checks and the single-OpenTUI-entrypoint check: 3 tests, 11 expectations, passing against the current install.
Cover the TUI startup crash from anomalyco#56856: theme colors must survive spinner conversion, every OpenTUI consumer must resolve one runtime entrypoint, and the install must keep distinct Zod and OpenTelemetry peer contexts now that root dependencies participate in the filtered install (anomalyco#56889). nix/scripts is not a workspace, so turbo never runs the test. The new checks.<system>.dependency-identity derivation overlays the node_modules output on the sources and runs the test there, and check-nix builds it, so an install-filter regression fails CI instead of the TUI. On the pre-anomalyco#56889 filter the test reproduces the reported failure ('hex.replace is not a function') and two identity assertions.
ab8c385 to
e4655c9
Compare
|
Rebased and upgraded so the check actually exercises the Nix install:
Evidence that it catches the regression: reproducing the derivation's filtered install locally with the pre-#56889 filter ( Nix is not installed here, so the derivation itself is unbuilt locally — |
There was a problem hiding this comment.
Thread 1 is fixed: the test now runs against the node_modules tree that nix/node_modules.nix builds, not the CI install. I couldn't build the check myself because Nix isn't available here.
New concern, inline: the check rebuilds the node_modules derivation with its fixed hash from nix/hashes.json. Those hashes are only refreshed by bot-nix-hashes after a push to dev. So check-nix will likely fail with a hash mismatch on any PR that changes dependencies, and on dev until the bot's hash PR merges. Today check-nix only evaluates and doesn't hit this.
The peer-context point (thread 2) is still open; the test file didn't change.
Public API: unchanged. This adds a flake check, a Nix derivation, a test and a CI step.
| # so a regression in the install filters fails here instead of at TUI startup. | ||
| - name: Check dependency identity | ||
| timeout-minutes: 20 | ||
| run: nix build .#checks.x86_64-linux.dependency-identity --print-build-logs |
There was a problem hiding this comment.
This builds nix/node_modules.nix, which is a fixed-output derivation pinned to nix/hashes.json. That file is only updated by bot-nix-hashes after a push to dev. Any PR that changes bun.lock or a package.json would then fail here with a hash mismatch, a failure the PR can't fix itself, and so would dev until the bot's hash PR lands. The check also rebuilds the whole dependency tree on every PR, hence the 15 → 40 minute timeout. Running the test in bot-nix-hashes might fit better: it already builds this install on each platform when dependencies change, so you could run the test there once the new hash is known. Another option is to run this step only when the hashes are expected to be current.
There was a problem hiding this comment.
Agreed and fixed. The check no longer runs in check-nix: bot-nix-hashes now builds checks.<system>.dependency-identity in the same job that computes the new hash, passing that hash, so a filter or lockfile change gets tested instead of failing on a stale pinned hash. check-nix is back to eval-only.
The check builds the fixed-output node_modules derivation, which is only pinned to the hash in nix/hashes.json. A pull request cannot update that hash (only bot/nix-hashes may), so check-nix would fail with a hash mismatch on any change to the install inputs - including a regression in the filters the test is meant to guard. Run it in bot-nix-hashes instead, right after the new hash is computed, building the check against that hash. Drop the two peer-context cases: they passed on the pre-anomalyco#56889 layout too, and they only hold while the lockfile keeps multiple Zod and OpenTelemetry API versions.
There was a problem hiding this comment.
Both earlier points are addressed. The peer-context tests are gone, and the check moved from check-nix (back to eval-only) into bot-nix-hashes, where it builds against the newly computed hash instead of a stale pinned one. I checked the change by reading it; Nix isn't available here, so I couldn't build checks.<system>.dependency-identity. The wiring looks right: the step output id matches, only hash.txt is uploaded, so the locally rewritten nix/hashes.json never reaches compute/publish, and check-bot-scope.yml and .github/nix-hashes/ are untouched.
One thing to be aware of: compute needs every hash job, so if this check fails on any one system, no hash PR is opened for any system. That is reasonable for a real filter regression, but an environment-specific failure (for example only on macOS) would block every hash update until it is fixed.
Public API: unchanged.
Related: #56856
|
Both review points are addressed: the peer-context cases are gone, and the dependency-identity check now runs in The workflows on this head are still |
Issue for this PR
Follow-up regression coverage for #56856. The production fix was independently merged in #56889 and is already in
dev; this PR does not change the Nix install or dependency hashes.Type of change
What does this PR do?
Adds a Nix check that runs against the filtered install the derivation produces, not against a developer or CI install:
hex.replace is not a functioncrash path),nix/scriptsis not a turbo workspace, so turbo never runs this file.checks.<system>.dependency-identityoverlaysnix/node_modules.nix's output on the sources and runs the test there.bot-nix-hashesbuilds that check right after it computes the new dependency hash, on all three systems. It is the only place that can: the check builds the fixed-output derivation, and onlybot/nix-hashesmay updatenix/hashes.json, so a pull request that changes the install filters or the lockfile cannot build it.On the pre-#56889 filter (
--filter '!./'), the same test reproduces the reported crash and fails all three tests, so the check catches a filter regression before it reaches a release.How did you verify your code works?
--filter './'):bun testinnix/scriptspasses, 3 tests / 11 expectations.--filter '!./'(pre-fix(nix): preserve root dependency peer context #56889): all three tests fail, includingTypeError: hex.replace is not a functionfrom@opentui/coreand two copies of@opentui/coreacross consumers.bun.lock,bunfig.toml,package.json, copiednode_modules): 3 pass.oxlint nix/scripts/dependency-identity.test.ts: 0 warnings, 0 errors; Prettier clean.nix build .#checks.x86_64-linux.dependency-identityhas not run here;bot-nix-hashesbuilds it on the next dependency change.Screenshots / recordings
Not applicable; this PR adds a check.
Checklist
Added by the opencode bot (AI-generated).
Related: #56856