Repository navigation
Validate published packages in CI - #1490
Draft
cristianrgreco wants to merge 3 commits into
Draft
cristianrgreco wants to merge 3 commits into
cristianrgreco wants to merge 3 commits into
Conversation
Add npm run check-packages, which builds each package from a clean build directory with its publish config, packs it, and fails if the tarball contains test-only files or if publint or @arethetypeswrong/cli report problems. Run it per changed package in a new Package job in the Checks workflow.
Packages already emit CommonJS. Declaring it stops Node.js from running module syntax detection on the package files, as publint suggests.
Modules depend on testcontainers, which requires Node.js >= 22.22, so declaring the same range on each module does not narrow support.
✅ Deploy Preview for testcontainers-node ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Nothing in CI checked what we actually publish. The smoke tests cover core's runtime under CJS, ESM, Jest and Bun, but nothing checked type resolution, package metadata or tarball contents for core and the 43 modules.
This PR adds
npm run check-packages [-- <package>...]and runs it per changed package in a newPackagejob inchecks.yml. For each package it:tsconfig.build.json, plus core'sprebuildand thepostbuildasset copies) from an emptybuilddirectory.npm pack, soprepackandfilesapply as they do on publish.*.test.*,*.spec.*,test-helper*,*-test-utils*,fixtures/,__tests__/,__mocks__/).publintwith--stricton the tarball, so errors and warnings fail the check.@arethetypeswrong/clion the tarball with the defaultstrictprofile (node10, node16-cjs, node16-esm, bundler) and--no-definitely-typed, so it checks only the types we ship.The tools follow the
check-enginespattern:npm exec --yes --package=<pkg>@<exact version>. They don't add devDependencies, so the lockfile and every CI install stay the same size. Both versions are older than themin-release-age=7window.The build has to start from an empty directory.
npm run check-compilesbuilds withtsconfig.json, which emits the tests into the samebuilddirectories. Also,tscskips re-emitting files that its.tsbuildinfosays are up to date, so deletingbuildon its own isn't enough. In a working tree wherecheck-compileshas run, a plainnpm pack -w packages/modules/postgresqlships 10*.test.js/*.test.d.tsfiles. Publishing isn't affected, because the publish workflow builds from a freshnpm ci. The check deletes bothbuildand*.tsbuildinfofirst.This automates the manual
AGENTS.mdrule that test-only helpers undersrcmust be excluded intsconfig.build.json, soAGENTS.mdnow points to the check and lists it in the PR checklist.Where it runs
The check needs the publish build. The
Compilejob builds core with--project tsconfig.json(which includes the tests) and skips lifecycle scripts, so reusing that job would mean a second clean build in it anyway. The newPackagejob instead copies theLint/Compilelayout: the samedetect-modulesmatrix, the samenpm-setupcache,needs: lint, and it's added toChecks complete. Changes under.github/scripts/already select all packages, so editing the script checks every package.Findings
Discovery ran on 12.2.0 (
mainat c528f29):npm ci,npm run build --ws, thennpm pack --wsand each tool against each of the 44 tarballs.testcontainers"type""type", no"engines.node"So the current tarballs are in good shape. The guardrail is there to keep them that way, for example when a new module adds a
*-test-utils.tsand forgets thetsconfig.build.jsonexclude, which today onlyAGENTS.mdprevents.Per-package results (before this PR)
testcontainerstype@testcontainers/arangodbtype,engines.node@testcontainers/azure-cosmosdb-emulatortype,engines.node@testcontainers/azureservicebustype,engines.node@testcontainers/azuritetype,engines.node@testcontainers/cassandratype,engines.node@testcontainers/chromadbtype,engines.node@testcontainers/clickhousetype,engines.node@testcontainers/cockroachdbtype,engines.node@testcontainers/couchbasetype,engines.node@testcontainers/couchdbtype,engines.node@testcontainers/elasticsearchtype,engines.node@testcontainers/etcdtype,engines.node@testcontainers/gcloudtype,engines.node@testcontainers/hivemqtype,engines.node@testcontainers/influxdbtype,engines.node@testcontainers/k3stype,engines.node@testcontainers/kafkatype,engines.node@testcontainers/kurrentdbtype,engines.node@testcontainers/localstacktype,engines.node@testcontainers/mariadbtype,engines.node@testcontainers/miniotype,engines.node@testcontainers/mockservertype,engines.node@testcontainers/mongodbtype,engines.node@testcontainers/mosquittotype,engines.node@testcontainers/mssqlservertype,engines.node@testcontainers/mysqltype,engines.node@testcontainers/natstype,engines.node@testcontainers/neo4jtype,engines.node@testcontainers/ollamatype,engines.node@testcontainers/opensearchtype,engines.node@testcontainers/oraclefreetype,engines.node@testcontainers/postgresqltype,engines.node@testcontainers/qdranttype,engines.node@testcontainers/rabbitmqtype,engines.node@testcontainers/redistype,engines.node@testcontainers/redpandatype,engines.node@testcontainers/s3mocktype,engines.node@testcontainers/scylladbtype,engines.node@testcontainers/seleniumtype,engines.node@testcontainers/toxiproxytype,engines.node@testcontainers/valkeytype,engines.node@testcontainers/vaulttype,engines.node@testcontainers/weaviatetype,engines.nodeFixed in this PR
Each fix is its own commit, so either can be dropped on its own.
"type": "commonjs"on all 44 packages (publint suggestion). The packages already emit CommonJS: without a"type"field Node already treats.jsas CommonJS, but it may also run module syntax detection on these files (publint mentions a small performance hit). Declaring the type turns that off and states the existing format. Thetscoutput is byte-identical before and after for all 44 packages."engines": { "node": ">= 22.22" }on the 43 modules (publint suggestion), copied from core.package-lock.jsonchanges only by the matching 43enginesentries.Deferred / not changed
exportsfield. attw and publint pass without it. Addingexportswould block deep imports (for exampletestcontainers/build/...), which is a breaking change. Recommendation for the next major: add"exports": { ".": "./build/index.js", "./package.json": "./package.json" }(plus explicit"types"if you like). This check would then validate the export map under every resolution mode.@testcontainers/gcloudleaks@google-cloud/spannertypes. attw and publint can't see this because they don't resolve other packages. I confirmed it with a clean consumer: install the packedtestcontainers,@testcontainers/gcloudand@testcontainers/postgresqltarballs, then runtscwithmodule: nodenextandskipLibCheck: false.build/spanner-emulator-helper.d.tsfails withTS2307: Cannot find module '@google-cloud/spanner'(and.../build/src/instance) unless the consumer installs spanner. Core and postgresql type-check cleanly. Declare missing module dependencies #1485 already defers this as a possible optionalpeerDependencybecause of npm conflicts across spanner majors. This result adds that users withskipLibCheck: falsehit it at compile time today. A consumer type-check like this could be a follow-up guardrail, but it needs registry installs per package, so I left it out of this job.npm-publish.yml. Every PR and push tomainruns the check on the same build config, so I left the release workflow unchanged.Verification
"src/test-helper.ts"frompackages/modules/kafka/tsconfig.build.json.npm run check-packages -- kafkathen fails withTest-only files in the tarball: build/test-helper.d.ts, build/test-helper.jsand exits 1. Restoring the exclude makes it pass.tsc -b packages/testcontainers packages/modules/postgresql(whatcheck-compilesruns),build/contains 10 test files.npm run check-packages -- postgresqlstill passes because it rebuilds from empty.npm ci --workspace packages/modules/gcloud --include-workspace-root(whatnpm-setupinstalls for a matrix entry), thennpm run check-packages -- gcloud. It passes, and the working tree stays clean.npm run check-compileswasn't needed because no TypeScript sources changed.generic-container-dockerfile.test.tsrun failed on the Reaper session label while other local sessions were using the same Docker daemon. It passed on 3 reruns with this change and on a run without it.Why this is not breaking
AGENTS.mdtext only affect contributors."type": "commonjs"is how Node, TypeScript (node16/nodenext), Bun and bundlers already read these packages, since a missing"type"means CommonJS. The emitted files are byte-identical, and the CJS and ESM smoke tests pass.engines.nodeon modules doesn't narrow support. Every module depends ontestcontainers, which already declares>= 22.22, and npm, Yarn and pnpm check engines across the whole dependency tree. An unsupported Node version already gets the same warning, or the same error underengine-strict. publint labels this "may be breaking" in general, but that doesn't apply when a dependency already requires the same range.The engines commit touches the same
package.jsonfiles and lockfile area as #1485 (azurite, k3s, selenium), so whichever PR merges second may need a trivial rebase.