From 59929868fd5d1c7522486cc0ca4e2eafae625a50 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sat, 5 Sep 2026 17:09:41 +0200 Subject: [PATCH 1/6] chore: back-merge main into develop (#1) * ci: install ffmpeg with apt in the provisioned e2e job too `ailoud setup` refused the sudo step, which is what it is supposed to do: with no terminal to answer a password prompt on it reports the exact command rather than hanging. A runner has no terminal, so the apt install is the workflow's job and only the rest -- the whisper release and the model files, neither needing sudo -- is left to setup. Also makes the back-merge workflow say so when there is no develop branch, instead of dying on `fatal: Not a valid object name origin/develop`. That is how its first run failed, and the copy it came from has the same weakness. * ci: put the provisioned binaries on PATH, and let Actions open the back-merge PR Two separate failures, both in what I added rather than in the product. The provisioned e2e job had every doctor check green and every transcribe spec exiting 3. The sandbox writes its own config naming only the MODEL and leaves `binary` at its default, so whisper-cli has to be on PATH -- while `setup` installs it under the data directory and records the absolute path in the user's config. Nothing was wrong with provisioning; the two configs simply disagreed about how the binary is found. The job now reads the installed paths back out of that config and adds their directories to PATH, read rather than spelled so bumping a pinned release cannot silently break it. The back-merge workflow could not open its PR: "GitHub Actions is not permitted to create or approve pull requests", a repository setting that is off by default. Enabled with default_workflow_permissions still `read`, so the token gains nothing beyond what this needs. Also adds the author (contact@lorem.dev) to all four manifests, the dev-tag skill and its spec, the branch and tag rules in AGENTS.md, and a publish-time check that every tarball carries the LICENSE and no source or tests -- pnpm copies the repository LICENSE into each workspace tarball, which is why no package holds its own copy, and an invariant worth relying on is worth checking. The `.agents/` skills were also missed by the rename and still said "laud". * docs: note which checks are required, and why not the provisioned one The provisioned end-to-end job runs only on push, so requiring it on a branch would leave every pull request waiting for a check that never arrives. * fix: restore the e2e test timeout, and lint the config files Every transcribe spec on CI failed with "Exceeded timeout of 5000 ms" -- Jest's default, not the 600 s the config asks for. Jest takes the per-test timeout from the GLOBAL config. Splitting jest.config into `projects` moved `testTimeout` into each project, where it appears in `configs[].testTimeout` while `globalConfig.testTimeout` stays undefined, so every test fell back to 5 s. Confirmed both ways with `jest --showConfig`: projects 600000, global undefined; with the value also at the root, global reads 600000. Locally this was invisible. The transcribe specs were already failing fast for a missing model -- mine are still under the pre-rename path -- so nothing ran long enough to hit a 5 s limit. On CI, where setup had just downloaded them, whisper takes tens of seconds and every spec timed out. The bug was mine and the runner was the only place it could be seen. Separately, the build's own config files were in eslint's ignore list, so every editor reported "File ignored because of a matching ignore pattern" on opening one, and a mistake in them was caught by nothing -- including the mistake above. They are linted now, with Node globals and the recommended rules rather than the type-aware config the packages use. Verified by planting an unused variable in jest.config.cjs and watching eslint catch it. * feat: fold pre-release changelog sections, and test the release scripts Cutting 1.0.0 after 1.0.0-dev.1, -dev.2 and -rc.1 left four changelog sections describing one release. `fold-prereleases.mjs` merges them into one, keeping the subsection grouping and dropping duplicates -- including a duplicate whose copies were wrapped differently, which is the common case across two dev tags. Only the same version's pre-releases fold, so an entry from an abandoned line cannot reappear under a release it was never part of. `check-changelog.mjs` is a required step on every tag, before the gate and before anything is built: a version number can never be reused and the unpublish window is 72 hours, so every reason to refuse is worth finding while refusing is free. It checks the section exists and has entries, is inside the hard limit, that nothing is stranded under Development, and that a final tag folded its pre-releases. Problems are collected and reported together, because somebody fixing a changelog wants the list rather than one round trip each. The limits were copied into three scripts, so they now live once in scripts/lib/changelog.mjs alongside the parsing all three need. A limit that differs between the script that warns and the script that refuses is worse than no limit: one of them is wrong and nobody knows which. The soft limit is a warning rather than an error -- console.warn locally, a GitHub annotation under Actions. It read as a failure before. The scripts have tests now: 20 over the shared library, and 22 driving all three end to end. Two of them WRITE, so the tests copy scripts/ into a throwaway directory beside a fixture CHANGES.md, and each asserts it is under the temp directory before running anything. One test reads the repository's own changelog afterwards and fails if it changed -- if a script ever resolves the wrong root, that is how it will be caught. They run in ci.yml, which triggers on branches and pull requests but not on tags, so a broken script is caught before a release depends on it. Also lints scripts/ and the build's config files, which were both in eslint's ignore list: every editor reported "File ignored because of a matching ignore pattern" on opening one, and `node --check` was the only gate on scripts/ -- which sees syntax, not an unused variable. Found while writing the tests: execFileSync discards stderr on success, so the soft-limit test could never have seen the warning it was asserting. spawnSync returns both streams. * feat: retire the pre-releases a final release supersedes Cutting a final tag leaves its `-dev.N` snapshots behind: still installable, still holding the `dev` dist-tag, still tagged. `scripts/retire-prereleases.mjs` clears them in one step, printing the plan and changing nothing without --yes. It deprecates rather than unpublishes -- npm allows unpublishing for 72 hours, the version number can never be reused after, and anyone who pinned it has their install broken. A deprecated version keeps working and says why. It deletes a tag only when the tag's commit is reachable from origin/main. The provenance of a published package names both the commit and the tag: losing the name costs convenience, but deleting a tag that holds the only reference to its commit lets the commit be collected, which costs the attestation its subject. The decision of which tags those are is a pure function in the shared lib, so it is tested without a repository; the CLI is tested against a throwaway one with a tag on each side of the line. Splitting the script tests one module per script also turned the single end-of-file "the real changelog is untouched" test into an afterEach that compares its bytes after every test, and catches a stray RELEASE_NOTES.md too -- verified by planting both. * fix: stop the script tests reading CI's environment, and retry the pnpm download Two failures on the runner that no local run could produce. The scripts read $GITHUB_REF_NAME as a tag fallback and print warnings as ::warning:: annotations on stdout when $GITHUB_ACTIONS is set. Both leaked into the tests through the inherited environment, so `check-changelog` with no argument found the tag `main` instead of failing, and the warning the retire test looked for on stderr had gone to stdout. The harness now scrubs every GITHUB_ variable, and tests that want that behaviour pass it explicitly -- which also gets both warning channels covered for the first time. Verified by running the suite with the runner's variables set: two failures before, none after. Separately, `corepack enable` only writes shims, leaving the pinned pnpm to be downloaded by whoever invokes it first -- setup-node's cache probe, where the download crashed on an undici assertion inside Node 24.20.0 and nothing could retry it. The setup is now one composite action, shared by all four jobs, that downloads pnpm in a step of our own and retries it three times. * fix: give the layer-boundary rule the half of its job it was silently skipping Chasing the resolver's "multiple projects" warning turned up something worse behind it: `boundaries/dependencies` never saw a cross-package import written as a package name. `@ailoud/providers` resolves through node_modules to `packages/providers/dist/index.js`, which matched no element pattern, so the rule classified the target as unknown and reported nothing -- while the same import written as `../../providers/src/index.js` was caught. Listing dist under the same type as src makes both forms the same violation. Verified by planting each one in packages/core: the package-name form was accepted before this change and is an error after it, the path form and the node:fs restriction still bite, and a clean tree lints silently. The resolver now reads the root tsconfig, which includes every package's sources, instead of a glob over the per-package ones -- one project rather than four, which is what it was asking for. * fix: install ffmpeg in the publish job, which runs the same gate as CI The v1.0.0-dev.1 release failed at "Run the gate", not at anything about releasing: the gate includes packages/providers/src/audio/ffmpeg.test.ts, which spawns the real binary, and this job never installed it. CI installs it in both jobs that run tests for exactly this reason, and the publish job runs the same gate. * test: stop the suite printing expected errors and a notice it acts on Every `pnpm test` printed "error: unknown option '--bogus'" twice, a full usage block, and one ExperimentalWarning per worker -- so a real error had to be picked out of noise the tests produce on purpose. Commander writes usage errors and no-argument help to stderr, which is right for a CLI. The four tests that provoke it now silence writeErr only; writeOut stays as buildProgram set it, because that is how help reaches context.write and one of the tests asserts on it. The SQLite notice is the flag the installed binary already carries in its shebang: node:sqlite is experimental and this project knowingly depends on it. Set through NODE_OPTIONS rather than the pool's execArgv -- that route works, but switching the pool to forks took doctor.test.ts from 3 seconds to a 225-second timeout. * chore: 1.0.0-dev.1 * fix: publish the tarballs by absolute path, not one npm reads as a repo `npm publish dist-npm/ailoud-core-1.0.0-dev.1.tgz` never looked at the file: npm parsed the slash as the `owner/repo` GitHub shorthand and ran `git ls-remote ssh://git@github.com/dist-npm/ailoud-core-1.0.0-dev.1.tgz.git`, which failed on a missing public key -- an error about ssh keys in a step that has no business talking to git. An absolute path cannot be read that way. The `ls` that found the file is gone too: it existed to expand a name that was never a glob, and it turned a missing tarball into a confusing npm error instead of saying which one was missing. * test: cover the video containers import accepts `ailoud audio import` takes video because a meeting recording usually is one, and domain/mime.ts maps four containers -- but nothing exercised any of them. The audio path was tested against a tone generated at run time; video needs a real container, so scripts/make-fixtures.mjs now wraps en-short.wav in each of the four and the fixtures are committed, through LFS like the audio. They are deliberately dull: 32x32 of black at 5 fps, which keeps each file under 14 kB while still being a real, decodable video stream. Each container gets the codec pair it carries in the wild, and one test per container asserts what actually matters -- that whatever went in, the 16 kHz mono WAV whisper.cpp needs comes out. The video length comes from the audio rather than from `-shortest`, which produced an 18-second mp4 from a 2.5-second source. * fix: resolve the release tag once, instead of letting a step default to a branch Re-running the publish workflow on v1.0.0-dev.1 failed with `no "## Version main" section`. The changelog check was the one step that passed no tag and let the script fall back to $GITHUB_REF_NAME -- which is the tag on a tag push but the BRANCH on a workflow_dispatch, so a manual re-run of a good tag checked the wrong version and could never pass. The tag is now resolved once into a job-level TAG and used by all three steps that need it, so there is no second answer to which tag is being released. * fix: let npm do the OIDC exchange by giving it no credentials to find Publishing failed with `404 PUT https://registry.npmjs.org/@ailoud%2fcore` even after the organization existed, and the log never mentioned trusted publishing -- because it never happened. setup-node's registry-url writes an .npmrc holding `_authToken=${NODE_AUTH_TOKEN}`; this workflow has no token secret by design, so npm found a credential, sent an empty one, and had no reason to reach for OIDC. The registry answered as it would to any stranger. Without that .npmrc npm sees it has nothing, and does the exchange the `id-token: write` permission is there for. The default registry is registry.npmjs.org regardless, so nothing else changes -- and the `Unknown user config "always-auth"` warning was setup-node's file too. * feat: allow a token for the bootstrap publish, and prefer OIDC once it can work npm answers ENEEDAUTH for @ailoud/core however complete the OIDC setup is -- npm 11.19.0, id-token: write, no stale .npmrc -- because a trusted publisher is attached to a package on npmjs.com and there is no page to attach it to until the package exists. The first version of each of the three has to go out on a credential; nothing about the workflow was wrong. So NPM_TOKEN is used when the secret is present and ignored when it is not, which makes removing the secret the whole of the switch to trusted publishing. Provenance is attached either way. The token is a bootstrap, not a fixture: once all three package pages have the publisher attached, deleting the secret leaves the arrangement with nothing to expire, which was the point of using OIDC in the first place. * docs: state the one case where a pre-release does move npm's latest tag Four places said only a final tag moves `latest`, without qualification. The first release proved otherwise: npm set `latest` to 1.0.0-dev.1 on all three packages, because it does that on a package's first publish whatever `--tag` says, and `latest` can be moved but never removed. So `npm install ailoud` returns the snapshot until 1.0.0 exists. Nothing in the workflow can prevent it, which is exactly why it belongs in the documentation rather than in a check. * fix: give every published package a README, and check that it has one All three npm pages read "This package does not have a README" after 1.0.0-dev.1 -- the first thing anyone arriving from a search saw. npm shows the README from inside the tarball, and no package directory had one. The two libraries get their own, short and specific: what the package is, how it relates to the other two, and that its interfaces are not stable and there is no reason to depend on it directly. The CLI's README is the repository's, so rather than keep a second copy in git that would drift, its prepack script copies the root file in at pack time -- verified identical in the packed tarball -- and the copy is gitignored. The packing guard now fails a release whose tarball has no README, beside the existing licence and no-source checks: the CLI's copy is a script that could stop working quietly, which is exactly the kind of thing that guard is for. * chore: 1.0.0-dev.2 * fix: report the installed version, not the literal 0.0.0 `ailoud --version` on the published 1.0.0-dev.1 answered `0.0.0`, and the MCP server told every client the same, because buildProgram passed commander a hardcoded string. Found by installing the release from the registry rather than by reading the code -- nothing in the repository disagreed with itself. The version now comes from the package's own manifest, resolved relative to the module so `dist/version.js` finds it one level up both here and in an installed package. The manifest is the one copy a release already updates and it ships inside the tarball, so it cannot go stale. The test asserts agreement with the manifest rather than a literal, so it needs no edit per release -- an edit per release is what would rot it into agreeing with whatever is there. Checked both ways: it fails against the old hardcoded value, and the packed tarballs installed into a scratch project report 1.0.0-dev.2. * feat: refuse a final release that would publish on a stored token The token exists to introduce each package to the registry, because a trusted publisher is attached to a package that already exists and there is no page to attach it to before the first publish. That bootstrap is over the moment the three packages exist -- but a token that keeps working is a token nobody gets round to removing, and the whole point of OIDC was having nothing stored. So the rule is enforced rather than remembered: a pre-release published on the secret logs a warning, and a final release refuses before anything is published and names the two steps that clear it. Publishing with no secret goes through OIDC exactly as before. Simulated all three paths -- pre-release with the token, final with the token, final without -- because the first version of this check read $version above the line that assigns it, which under `set -u` would have failed every release rather than only the ones it means to. * docs: note what the next snapshot is for 1.0.0-dev.2 cannot be republished -- npm's policy is that a version number is never reused, even after an unpublish -- so verifying that a release needs no stored credential takes a new number. Saying so in the changelog beats leaving someone to wonder what changed between two snapshots that are the same code. * chore: 1.0.0-dev.3 * feat: retire superseded pre-releases automatically, still with no stored token Deprecating the snapshots a release supersedes was a manual step because trusted publishing is defined for publishing: `npm deprecate` in the same job has nothing to authenticate with, and I was not willing to guess otherwise halfway through a release. Reading npm's own lib/utils/oidc.js settles it. Publishing gets its credential by exchanging the CI identity for a per-package token -- a GitHub id token with audience `npm:registry.npmjs.org`, posted to `/-/npm/v1/oidc/token/exchange/package/` -- and the exchange is an ordinary request anything can make. So the script makes it, and retiring a release needs no more stored credential than publishing one. The token never reaches a command line or a log: it goes into a temporary 0600 npmrc that is removed in a finally, which the tests check both ways. `retire.yml` carries this, called by publish.yml after a final release and dispatchable alone -- without `confirm` it exchanges a token, uses it for nothing, and reports whether it worked, so the credential path can be checked without waiting for a release to find out. * fix: retire pre-releases only as part of a production release A dispatchable retirement was two mistakes at once. Retiring snapshots means nothing unless something supersedes them, and the standalone run could not authenticate anyway: npm binds a trusted publisher to a workflow file, so a run entered through retire.yml is a different identity from one entered through publish.yml and the exchange is refused with `OIDC token exchange error - package not found`. The dry run proved that before a release depended on it. So retire.yml is workflow_call only, reached by publish.yml for a final tag, and its `confirm` input is gone -- it could only ever have been true, and a knob with one reachable value describes a choice that is not there. The script keeps its plan-first default for the laptop, where nothing has been decided. It runs after the publish rather than before. Deprecating the snapshots first would, if the publish then failed, leave every -dev.N pointing at a release that does not exist while `dev` is the only thing installable. * docs: describe the release rules once, and how the credential works The release rules had grown by accretion: credentials were explained in two places, the retirement rules were spread over four paragraphs written as each came up, and a sentence about the changelog check sat at the end of a section about tags. Anyone reading it would have had to assemble the rules themselves. Now "Branches and Tags" covers branches, tags and the changelog fold, and a "Publishing" section covers the rest: the OIDC exchange with the two calls verbatim, the three consequences that constrain anyone changing it, the bootstrap exception with the token as a table, retirement, and the npm facts none of it can work around -- a version number used up forever, `latest` set on first publish and never removable, and trusted publishing covering `npm publish` and nothing else. Every claim states what it costs to get wrong, because that is what makes a rule followed rather than looked up: why the entry workflow cannot be retire.yml, why deprecating happens after the publish and not before, why the token goes to a file and not a command line. * feat: check dependencies for advisories and age before a release A new `check-dependencies` skill: advisories first, funding second, updates last -- because the first two decide what an update is for. `pnpm audit` and `pnpm audit --prod` are read separately, since a high-severity advisory in a test runner cannot reach a user and one in `commander` is on their machine. The 14-day rule is a script rather than advice, because advice is what gets skipped at the moment it matters. `scripts/check-dependency-age.mjs` refuses any pinned direct dependency published less than 14 days ago: a compromised release is found by other people and that takes days, and there is no urgency in a patch that has been out two weeks that was not there on day one. The rule yields to a critical advisory -- two weeks with a known exploit is worse than a version nobody has audited yet -- through `scripts/dependency-age-exceptions.json`, where an exemption carries its advisory ID. That makes it a decision in the repository rather than an argument someone remembers to pass, and the check reports entries that have aged out so the file does not accumulate permanent holes. It runs second in `pre-release-check`, before the tests: an update it recommends changes what everything below it is testing. * feat: add Dependabot and CodeQL, both held to the 14-day rule Dependabot with `cooldown: default-days: 14`. Without it Dependabot opens a pull request the moment a version appears -- exactly the window check-dependency-age exists to refuse -- and the check would then fail on Dependabot's own branch, leaving the two arguing on every update. Security updates ignore cooldown, which is the behaviour we want and the same exception the age check records for a human: a known advisory beats an unaudited release. `versioning-strategy: increase` because exact pins are the convention here; `widen` would turn a pin into a range and hand the choice of version to whatever resolved last, which no age check can judge. The dev toolchain arrives as one grouped pull request since it cannot reach a user, while anything that ships gets its own. CodeQL is committed rather than enabled through the repository's default setup, for the reason every other check here is a file: what runs, when, and over what belongs in a diff. Weekly as well as per-push, because most findings arrive when the queries improve, not when the code changes. * fix: match provider hosts by hostname, and stop three patterns that backtrack CodeQL's first run found ten things; seven were real and this is them. Two places decided whether an endpoint is a hosted API by substring: `baseUrl.startsWith('https://api.openai.com')` and `/api\.(openai|anthropic)\.com/.test(baseUrl)`. Both answer yes for `https://api.openai.com.example.net/v1`, where the part of a hostname that decides where the request goes is the end of it, and yes again for `https://example.net/?upstream=api.openai.com`. They now share `isHostedLlm`, which parses the URL and compares the hostname exactly. Four `replace(/\/+$/, '')` calls stripped trailing slashes with a pattern that backtracks; on a value that is mostly slashes that is a denial of service, and the value comes from configuration. `withoutTrailingSlashes` is a loop, which is what the operation always was. Two of the four CodeQL did not flag -- same defect, below its threshold. `escapePackageName` used `replace('/', '%2f')`, which substitutes only the first match. A package name holds at most one slash, so it was right by accident rather than by what it said; `replaceAll` says it. The age check had its own copy of the same line and now imports the one function. The stale-lock takeover in setupLock had a real window: two runs can find the same stale lock, both remove it, and the loser's `open(path, 'wx')` failed with a raw EEXIST about a path the user has never heard of. It now refuses the way every other contended case does. Not covered by a test -- reproducing it needs two interleaved processes -- so it is one branch converting one error code. * chore: hold TypeScript at 6 until typescript-eslint can load against 7 Dependabot's first run proposed TypeScript 7.0.2, and lint failed outright: "typescript-eslint does not support TS 7.0. Please see ... to run typescript-eslint using the TS 6 API." The bump is blocked by a peer, not by anything here, and left alone it would return every Monday with the same failure. Ignored for majors only, with the tracking issue named, so patches and minors keep arriving. Drop the entry when typescript-eslint supports TS >= 7.1. * fix: take over a stale provisioning lock without deleting a live one CodeQL flagged this again after the last fix, and was right to. The retry I added only handled the case where the file still existed at the second `open`, which is not the dangerous one. The real sequence: we read a stale holder, another run takes the same stale lock and becomes a LIVE holder, and our unconditional `rm` then deletes ITS lock and we create our own. Both runs proceed -- the one outcome this file exists to prevent -- and the retry could never see it, because after the `rm` our `open` always succeeded. Takeover now writes the lock to a scratch path beside the target and renames over it. Rename is atomic and overwrites, so two takeovers both succeed at renaming, but only one of them is in the file afterwards; reading it back and finding another pid is how the loser learns it lost, and it refuses like every other contended case. The losing branch has no unit test -- reproducing it needs two interleaved processes -- so the tests cover what can be checked: the winner is recorded as the holder, and the scratch file is gone whether the rename worked or threw. * chore(deps): bump the actions group across 1 directory with 2 updates (#3) Bumps the actions group with 2 updates in the / directory: [actions/checkout](https://github.com/actions/checkout) and [actions/cache](https://github.com/actions/cache). Updates `actions/checkout` from 5 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v5...v7) Updates `actions/cache` from 4 to 6 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](https://github.com/actions/cache/compare/v4...v6) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: add a version badge to the README Taken from the npm registry rather than from GitHub releases, which is where skillkeeper's comes from: this repository has no releases at all, so that badge would render an error. npm is also where the README tells people to install from, so the number in the badge is the number they would get. It reads v1.0.0-dev.1 today, because npm set `latest` on the first publish of each package whatever `--tag` said and `latest` cannot be removed, only moved. Publishing 1.0.0 moves it. * feat: refuse a release that ships an unreviewed high-severity finding The dependency half of this was already enforced -- a --prod advisory or a version younger than 14 days blocks a release -- while what CodeQL finds in our own code reached the release path not at all. Today's first scan found seven real defects, including one that let two provisioning runs proceed at once, so that asymmetry was not theoretical. Only `state=open` counts. A finding that has been reviewed is `dismissed` with its reason attached and does not block; fixing it and dismissing it are both answers, and ignoring it is the one that is not. Three of today's ten are dismissed with reasons and correctly do not register. It runs before the manifest check, the gate and the pack, so a release that will be refused is refused in seconds rather than after publishing two of three packages. A failure to READ the results also refuses: a release that cannot tell whether it ships a known finding is not one to make blind. Verified against the live repository (0 open high/critical, 3 dismissed not counted) and both branches of the shell exercised on fixture JSON. * docs: cut what 1.0.0's notes should not carry Three entries out, on the changelog's own rules rather than on taste. The dev.3 entry began "Nothing for users" -- which is the rule for what stays out, quoted back at itself. It described why a snapshot existed, not anything anyone could do with it. The two "Fixed" entries described a missing README on npm and `--version` answering 0.0.0. Both were real, both were fixed, and both existed only in 1.0.0-dev.* snapshots. The rule is explicit that "before the first release, that is every fix", and 1.0.0 is the first release -- someone reading its notes is deciding whether to start using this, not whether to move off a snapshot. Judgement call: if you would rather dev users see them, they go back. The summarize command and the four engines behind it were two entries about one feature, which the rules say to merge; now one. Folding 1.0.0 in a sandbox yields 14 entries and passes, warning on the soft limit of 10 -- defensible for the release that introduces the whole tool. * docs: say what the CLI actually has, and stop claiming half of it is unbuilt AGENTS.md's overview described an early milestone: it listed `import`, `transcribe`, `ls`, `show`, `doctor` as the surface and said `search`, `summarize` and the rest "do not exist yet; do not document or assume commands beyond what M1 lists" -- while instructing the reader to treat that file as the authority. Both `search` and `summarize` shipped long ago, and are published on npm. Two independent reviews found this, and the binary settles it: `audio search --help` and `audio summarize --help` both answer. That combination is worse than a stale sentence. An agent following it would refuse to document working commands and would take the absence as fact. Replaced with the commands as they are, noun by noun, plus the instruction that matters more than the list: check the binary, which cannot be stale. CLAUDE.md said the same thing in its own words ("in a later milestone, will summarize") and now agrees. Also: docs/mcp.md carried its "your own files are safe" list twice in two wordings, a copy-paste that survived review; one copy left. And docs/usage/cli.md's list of top-level spellings omitted `search` -- checked each of the eight against the built binary rather than trusting either list. * fix: close the provisioning lock race properly, and stop the release failing open Five reviews of the 1.0.0 candidate; these are the confirmed defects. THE LOCK, third attempt and this time measured. Rename-then-read-back does not exclude anything: the interleaving A.rename, A.read, B.rename, B.read leaves each run reading its own pid, and both enter the body. A review demonstrated 34 overlaps in 60 runs against the compiled code. Exclusion now sits on the takeover itself, through a second lock created with `wx` -- one syscall, one winner, and the loser told to try again rather than left to guess. The re-create still uses `wx`, so a run that took the lock on the fast path in between wins it legitimately. Release only removes a lock that is still ours; removing it unconditionally let a third run in while a legitimate successor was working. There is now a concurrency test, and it fails on the previous implementation at round one with exactly that ENTER/ENTER trace. RETIREMENT no longer destroys what it could not replace. Every npm failure was a `::warning::`, which fails nothing, and the tag deletion ran regardless -- so a refused credential deleted the tags, deprecated nothing, left `dev` pointing at a snapshot, and reported success. Worse, the tags are what name the pre-releases, so a re-run found nothing to retire and could never repair it. Failures are collected now, and any of them keeps the tags. Deletion also goes to origin before local, because the reverse left the tag on origin with nothing locally to retry it by. The `dist-tag rm` had skipped the credential guard the deprecations use. THE CODE SCANNING GATE could pass on an empty answer: `jq length` printed nothing, `[ "" -gt 0 ]` errored, and a failing `if` condition is not caught by `set -e`, so the else branch announced no alerts. It now refuses a count it cannot read, and paginates -- an alert past the first hundred blocked nothing. THE TOKEN REFUSAL for a final release ran after twenty minutes of gate; it is now the first step, which is what the documentation already claimed. Tarball existence and a credential for all three packages are checked before the first publish, not per iteration: the loop publishes library, library, CLI, and a spent version number cannot be reused. Also: the age check reported `undefined@undefined` for an unpinned dependency, and treated a 429 or 502 from the registry as "unknown", which turned the 14-day rule into a no-op reporting success -- it fails closed now, and a 404 stays the one answer that means "no such package". The OIDC helper documents returning null rather than throwing, which did not hold for a transport failure, i.e. exactly the case the ambient-login fallback exists for. And retire.yml interpolated a tag name straight into a shell in a job holding contents: write. * feat: gate the docs on a successful publish, create the release, warn on fresh deps Three loose ends from the release review. DOCS no longer race the publish. Both workflows started on the tag push, so a publish that refused -- or failed partway through its three packages -- left the site advertising a version npm did not have. docs.yml now runs on `workflow_run` of Publish, checks out the SHA that was published, and takes the version from that commit's manifest rather than from `head_branch`: on a workflow_run that field is documented as a branch, and a release should not rest on what it happens to hold for a tag. publish.yml already refuses a tag that disagrees with any manifest, so the number is the same one either way. THE GITHUB RELEASE is created after the packages are on the registry, with the body extracted from CHANGES.md by the script that already existed for it, and `--prerelease` for a `-` version. Through `gh` rather than a third-party action: adding one to the release path would sit oddly beside a 14-day rule for everything else we depend on. Re-running a tag edits the notes instead of failing on an existing release. THE 14-DAY RULE now runs in CI, as a warning. Dependabot's version updates already respect the same window through `cooldown`, so the two do not argue -- but a security update deliberately ignores it, and a fresh version that closes an advisory is the one case where taking it beats waiting. Failing the build there would make the rule an obstacle to the thing it protects. * ci: exclude the one CodeQL rule that keeps mis-reading the provisioning lock js/file-system-race has fired three times, always on setupLock.ts, and its latest report blocked the back-merge PR -- the check is required now, so a finding on a file it cannot reason about stops merges. Two of the three were real and both are fixed: the first implementation deleted a live lock, the second let two runs each read back their own pid and proceed. What remains is genuinely a false positive. The analysis sees a read of a path followed by a write to it; it cannot see that the exclusion is held by a second lock file (`provisioning.lock.steal`, created with `wx`), because that is a different path. Excluded by id in .github/codeql/codeql-config.yml rather than dismissed alert by alert, so the next report does not block a merge again, with the reasoning in the config and at the site. What stands in for the rule is the two-process test that runs against the compiled lock and fails on either old implementation -- a measurement rather than a pattern match. The query set moves into the config too, so each choice sits next to its reason. * chore: 1.0.0 Also corrects the documented order of the two changelog steps, which was the wrong way round and would have failed this very release. `bump-version` promotes `## Development` into a `## Version ` section, so folding the pre-releases first leaves it an empty one to promote and produces two `## Version 1.0.0` headings -- `check-changelog` then reports "has no entries" against the empty one. Tried both orders in a sandbox; AGENTS.md and docs/development/releasing.md now say bump first, and say why. The release steps in the docs were also stale in two smaller ways: they still told the reader to extract RELEASE_NOTES.md by hand, which publish.yml now does, and their numbering had two step 3s. * fix: retire snapshots by hand, because the OIDC token cannot deprecate The 1.0.0 release settled this. The exchange works and returned a token; npm then refused it: `E404 ... or you do not have permission` on the first call and `E401 ... token is invalid` on every one after. It is publish-scoped and spent. Trusted publishing authenticates `npm publish` and nothing else -- which is what AGENTS.md said before I read npm's oidc.js, found the exchange endpoint, and talked myself out of it. So retire.yml is gone, publish.yml no longer calls it, and the script runs from a terminal under `npm login`. What the attempt leaves behind is worth keeping: npmOidc.mjs, which preflight-npm-auth.mjs uses to establish before publishing that npm will accept all three packages -- a question the exchange CAN answer. Two things the release proved that the docs had as guesses: npm matches `workflow_ref` (the entry workflow) and not `job_workflow_ref`, so a reusable workflow is not the problem; and the token's scope is. The earlier fix held, which is the point of it: the npm side failed, so the script kept the tags and exited non-zero instead of deleting the only record of which snapshots still need retiring. Its release ran green and shipped everything; only the retirement is outstanding. --------- Signed-off-by: dependabot[bot] Co-authored-by: Lorem Dev Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .agents/skills/bump-version/SKILL.md | 6 +- .agents/skills/check-dependencies/SKILL.md | 141 ++++++++++ .agents/skills/check-docs/SKILL.md | 4 +- .agents/skills/check-fixtures/SKILL.md | 14 +- .agents/skills/check-licenses/SKILL.md | 4 +- .agents/skills/dev-tag/SKILL.md | 96 +++++++ .agents/skills/pre-release-check/SKILL.md | 26 +- .agents/skills/run-tests-and-linters/SKILL.md | 2 +- .gitattributes | 9 + .github/actions/setup-node-pnpm/action.yml | 38 +++ .github/codeql/codeql-config.yml | 32 +++ .github/dependabot.yml | 59 +++++ .github/workflows/backmerge.yml | 10 +- .github/workflows/ci.yml | 107 +++++--- .github/workflows/codeql.yml | 51 ++++ .github/workflows/docs.yml | 44 +-- .github/workflows/publish.yml | 250 ++++++++++++++++-- .gitignore | 4 + AGENTS.md | 203 +++++++++++++- CHANGES.md | 10 +- CLAUDE.md | 8 +- README.md | 1 + apps/cli/package.json | 4 +- apps/cli/src/commands/doctor.ts | 8 +- apps/cli/src/commands/mcp.ts | 3 +- apps/cli/src/program.test.ts | 35 ++- apps/cli/src/program.ts | 3 +- apps/cli/src/setupLock.test.ts | 119 ++++++++- apps/cli/src/setupLock.ts | 103 ++++++-- apps/cli/src/version.ts | 28 ++ apps/cli/src/wiring.ts | 4 +- docs/development/releasing.md | 114 ++++++-- docs/mcp.md | 16 -- docs/usage/cli.md | 2 +- eslint.config.mjs | 68 ++++- fixtures/en-short.mkv | 3 + fixtures/en-short.mov | 3 + fixtures/en-short.mp4 | 3 + fixtures/en-short.webm | 3 + jest.config.cjs | 8 + package.json | 7 +- packages/core/README.md | 16 ++ packages/core/package.json | 3 +- packages/core/src/domain/llmHost.test.ts | 54 ++++ packages/core/src/domain/llmHost.ts | 37 +++ packages/core/src/index.ts | 1 + packages/providers/README.md | 20 ++ packages/providers/package.json | 3 +- packages/providers/src/audio/ffmpeg.test.ts | 56 +++- packages/providers/src/llm/anthropic.ts | 3 +- packages/providers/src/llm/models.ts | 5 +- .../providers/src/llm/openAiCompatible.ts | 3 +- scripts/check-changelog.mjs | 89 +++++++ scripts/check-changelog.test.mjs | 104 ++++++++ scripts/check-dependency-age.mjs | 110 ++++++++ scripts/fold-prereleases.mjs | 103 ++++++++ scripts/fold-prereleases.test.mjs | 114 ++++++++ scripts/lib/changelog.mjs | 152 +++++++++++ scripts/lib/changelog.test.mjs | 196 ++++++++++++++ scripts/lib/dependencyAge.mjs | 87 ++++++ scripts/lib/dependencyAge.test.mjs | 101 +++++++ scripts/lib/npmOidc.mjs | 150 +++++++++++ scripts/lib/npmOidc.test.mjs | 67 +++++ scripts/make-fixtures.mjs | 99 ++++++- scripts/preflight-npm-auth.mjs | 46 ++++ scripts/release-notes.mjs | 78 +++--- scripts/release-notes.test.mjs | 33 +++ scripts/retire-prereleases.mjs | 142 ++++++++++ scripts/retire-prereleases.test.mjs | 136 ++++++++++ scripts/testing/harness.mjs | 99 +++++++ vitest.config.ts | 10 +- 71 files changed, 3510 insertions(+), 260 deletions(-) create mode 100644 .agents/skills/check-dependencies/SKILL.md create mode 100644 .agents/skills/dev-tag/SKILL.md create mode 100644 .github/actions/setup-node-pnpm/action.yml create mode 100644 .github/codeql/codeql-config.yml create mode 100644 .github/dependabot.yml create mode 100644 .github/workflows/codeql.yml create mode 100644 apps/cli/src/version.ts create mode 100644 fixtures/en-short.mkv create mode 100644 fixtures/en-short.mov create mode 100644 fixtures/en-short.mp4 create mode 100644 fixtures/en-short.webm create mode 100644 packages/core/README.md create mode 100644 packages/core/src/domain/llmHost.test.ts create mode 100644 packages/core/src/domain/llmHost.ts create mode 100644 packages/providers/README.md create mode 100755 scripts/check-changelog.mjs create mode 100644 scripts/check-changelog.test.mjs create mode 100644 scripts/check-dependency-age.mjs create mode 100755 scripts/fold-prereleases.mjs create mode 100644 scripts/fold-prereleases.test.mjs create mode 100644 scripts/lib/changelog.mjs create mode 100644 scripts/lib/changelog.test.mjs create mode 100644 scripts/lib/dependencyAge.mjs create mode 100644 scripts/lib/dependencyAge.test.mjs create mode 100644 scripts/lib/npmOidc.mjs create mode 100644 scripts/lib/npmOidc.test.mjs create mode 100644 scripts/preflight-npm-auth.mjs create mode 100644 scripts/release-notes.test.mjs create mode 100644 scripts/retire-prereleases.mjs create mode 100644 scripts/retire-prereleases.test.mjs create mode 100644 scripts/testing/harness.mjs diff --git a/.agents/skills/bump-version/SKILL.md b/.agents/skills/bump-version/SKILL.md index 640105e..cfab96e 100644 --- a/.agents/skills/bump-version/SKILL.md +++ b/.agents/skills/bump-version/SKILL.md @@ -1,14 +1,14 @@ --- name: bump-version description: > - Set the laud version across every package.json and promote the CHANGES.md + Set the ailoud version across every package.json and promote the CHANGES.md Development section to a released Version section, then create the release commit. Does not tag or push. --- # bump-version -Bump the laud version and prepare the release commit. Tagging and pushing +Bump the ailoud version and prepare the release commit. Tagging and pushing remain separate, deliberate steps. ## Steps @@ -44,7 +44,7 @@ Run `git diff` and confirm the version bump touched exactly these files: `## Development`). No other files changed. There is no `Cargo.toml` and no -`tauri.conf.json` in this project -- laud is a pure TypeScript workspace. +`tauri.conf.json` in this project -- ailoud is a pure TypeScript workspace. ### 4. Create the release commit diff --git a/.agents/skills/check-dependencies/SKILL.md b/.agents/skills/check-dependencies/SKILL.md new file mode 100644 index 0000000..cdaf2b2 --- /dev/null +++ b/.agents/skills/check-dependencies/SKILL.md @@ -0,0 +1,141 @@ +--- +name: check-dependencies +description: > + Audit npm dependencies for known vulnerabilities, report who asks for + funding, and update what is behind -- refusing any version published less + than 14 days ago unless it fixes a critical advisory. Run before every + release and after any dependency change. +--- + +# check-dependencies + +Three questions, in this order: is anything we depend on known to be +vulnerable, who is asking to be paid for it, and what should be updated. +Updating comes last because the first two decide what an update is for. + +## Steps + +1. **Look for known vulnerabilities.** + + ```bash + pnpm audit # everything, including the dev toolchain + pnpm audit --prod # only what the three published packages ship + ``` + + Read them separately. A high-severity advisory in a test runner is a + different problem from one in `commander`: the first cannot reach a user, + the second is installed on their machine. Fix the `--prod` findings first + and say which list a finding came from. + +2. **Report funding.** + + ```bash + npm fund + ``` + + `pnpm` has no `fund` command; `npm fund` reads the installed tree and works + in this workspace. The list is long and almost all of it is the dev + toolchain. What is worth surfacing is the intersection with what actually + ships: + + ```bash + npm fund --json | node -e ' + let s = ""; + process.stdin.on("data", (d) => (s += d)).on("end", () => { + const names = []; + const rec = (n) => { + for (const [name, child] of Object.entries(n.dependencies ?? {})) { + names.push(name); + rec(child); + } + }; + rec(JSON.parse(s)); + const runtime = new Set(); + for (const p of ["packages/core", "packages/providers", "apps/cli"]) { + Object.keys(require(`./${p}/package.json`).dependencies ?? {}).forEach((k) => + runtime.add(k), + ); + } + console.log(names.filter((n) => runtime.has(n)).join(", ") || "(none)"); + })' + ``` + + Report the count and that intersection. Do not open funding pages or + install anything. + +3. **See what is behind.** + + ```bash + pnpm outdated -r + ``` + + Nothing here is urgent by default. A version bump is worth taking when it + fixes something this project hits, closes an advisory from step 1, or keeps + a major version from drifting far enough to become a project of its own. + "It is newer" is not a reason. + +4. **Apply the 14-day rule BEFORE updating anything.** + + ```bash + node scripts/check-dependency-age.mjs + ``` + + A compromised release is discovered by other people, and that takes days -- + every npm supply-chain incident in recent memory was caught within a week + or two of publication, after everyone who upgraded immediately had already + installed it. Waiting costs nothing: there is no urgency in a patch that + has been out for two weeks that was not there on day one. + + So a candidate version younger than 14 days is not taken. Pin the previous + one and come back to it. + + **Unless it fixes a critical or high advisory.** Then waiting is the worse + risk -- a known exploit beats an unaudited release. Record the decision in + `scripts/dependency-age-exceptions.json` with the advisory ID: + + ```json + { + "some-package@2.0.1": "fixes GHSA-xxxx-yyyy-zzzz (critical), 2026-09-05" + } + ``` + + The exception is a statement about one moment, so the check reports entries + that have aged out and should be deleted. Never add one to silence the + check for convenience; if there is no advisory ID, there is no exception. + +5. **Update carefully.** + + One concern per commit, so a regression is bisectable: + + ```bash + pnpm update --latest # or edit the manifest and pnpm install + ``` + + - Keep versions **exact**. This project pins them; a range hands the + decision to whatever resolved last, and the age check cannot judge it. + - Run the `check-licenses` skill if any `package.json` changed -- a new + version can change its licence. + - Run the `run-tests-and-linters` skill. A dependency update that passes + nothing is not an update. + - A user-visible consequence goes in CHANGES.md; a toolchain bump does not. + +6. **Verify the result.** + + ```bash + pnpm audit --prod + node scripts/check-dependency-age.mjs + ``` + + Both must pass. Report what changed, what was deliberately left behind and + why, and any advisory that remains open with the reason it cannot be closed + yet. + +## Do not + +- Do not run `pnpm audit --fix` or `npm audit fix`. They resolve to whatever + is newest, which is the version the 14-day rule exists to refuse. +- Do not update a major version as part of a release. It is its own change, + with its own testing. +- Do not treat a clean `pnpm audit` as proof of anything beyond "no advisory + has been published". Most of what the age rule protects against has no + advisory yet. diff --git a/.agents/skills/check-docs/SKILL.md b/.agents/skills/check-docs/SKILL.md index 2cc06fb..e5a275d 100644 --- a/.agents/skills/check-docs/SKILL.md +++ b/.agents/skills/check-docs/SKILL.md @@ -22,9 +22,9 @@ release, or right after adding or changing a CLI command or option. - Confirm every `pnpm` command shown in README.md exists as a script in the root `package.json` (`pnpm lint`, `pnpm typecheck`, `pnpm test:cov`, `pnpm build`, `pnpm format:check`, `pnpm test:e2e`, etc.). - - Confirm every `laud` CLI command shown (`import`, `transcribe`, `ls`, + - Confirm every `ailoud` CLI command shown (`import`, `transcribe`, `ls`, `show`, `doctor`) is a command M1 actually ships, per the "Project - Overview" section of AGENTS.md. `laud` has no `search`, `collection`, + Overview" section of AGENTS.md. `ailoud` has no `search`, `collection`, `tag`, `summarize`, `export`, or `config` command yet; flag any of those names if they appear in README.md or AGENTS.md. - Confirm every relative link in README.md and AGENTS.md resolves to a diff --git a/.agents/skills/check-fixtures/SKILL.md b/.agents/skills/check-fixtures/SKILL.md index 48451bd..6d27d26 100644 --- a/.agents/skills/check-fixtures/SKILL.md +++ b/.agents/skills/check-fixtures/SKILL.md @@ -1,7 +1,7 @@ --- name: check-fixtures description: > - Drive the built laud binary against fixtures/ end to end -- import, + Drive the built ailoud binary against fixtures/ end to end -- import, transcribe, ls, show, and doctor -- in a throwaway HOME, XDG_CONFIG_HOME, and XDG_DATA_HOME, and confirm the working tree stays clean afterward. --- @@ -10,7 +10,7 @@ description: > The unit tests cover the domain core against fakes (`MemFs`, `FakeClock`, `FakeIds`, `FakeStt`). This skill covers the layer they cannot: the real -`laud` binary, against real audio, writing to a real filesystem (inside a +`ailoud` binary, against real audio, writing to a real filesystem (inside a sandbox). It is the only check that would catch a regression living in the wiring between the CLI, the providers, and the filesystem -- for example a provider writing to the wrong data directory, or a pipeline that behaves @@ -28,14 +28,14 @@ right lens" below. Three short fixtures, each with a reference transcript: an English clip, a Russian clip, and a clip that mixes both languages. The suite drives: -- `laud doctor` against a sandbox with no config, and again after +- `ailoud doctor` against a sandbox with no config, and again after configuring the model. -- `laud import` against a fixture file, including the "already present" +- `ailoud import` against a fixture file, including the "already present" path on a repeat import. -- `laud transcribe`, checked by word error rate against the reference +- `ailoud transcribe`, checked by word error rate against the reference transcript rather than exact string equality -- a model or quantization change shifts wording by a word or two without being a regression. -- `laud show` in both `srt` and `json` formats, plus its error paths (a +- `ailoud show` in both `srt` and `json` formats, plus its error paths (a missing id, an unsupported `--format`). ## Isolation @@ -44,7 +44,7 @@ The suite must never touch the developer's machine state. Every invocation of the built binary sets all three of: - `XDG_CONFIG_HOME`, which relocates `config.yaml`. -- `XDG_DATA_HOME`, which relocates `laud.db` and the `media/` tree. +- `XDG_DATA_HOME`, which relocates `ailoud.db` and the `media/` tree. - `HOME`, so nothing the process resolves relative to the real home directory (for example a fallback default when an XDG variable is unset) can reach outside the sandbox. diff --git a/.agents/skills/check-licenses/SKILL.md b/.agents/skills/check-licenses/SKILL.md index e6bfe48..741aac9 100644 --- a/.agents/skills/check-licenses/SKILL.md +++ b/.agents/skills/check-licenses/SKILL.md @@ -11,7 +11,7 @@ description: > Verify that every direct npm dependency is license-compatible with Apache-2.0 and keep the Third-Party Notices section of `LICENSE` up to -date. `laud` is a pure TypeScript pnpm workspace -- there is no cargo +date. `ailoud` is a pure TypeScript pnpm workspace -- there is no cargo workspace to check, unlike the source repository this project inherits its conventions from. @@ -71,7 +71,7 @@ dependencies` subsection's table -- replace from its `| Package |` | | npm | | | | ``` - Every row's Ecosystem column is `npm` -- laud has no other dependency + Every row's Ecosystem column is `npm` -- ailoud has no other dependency ecosystem. List rows alphabetically by package name. Preserve everything above the table verbatim: the Apache 2.0 license text, the `## Third-Party Notices` heading, and the intro paragraph above the diff --git a/.agents/skills/dev-tag/SKILL.md b/.agents/skills/dev-tag/SKILL.md new file mode 100644 index 0000000..0fb0de3 --- /dev/null +++ b/.agents/skills/dev-tag/SKILL.md @@ -0,0 +1,96 @@ +--- +name: dev-tag +description: > + Cut a throwaway development tag (`v-dev.`) to publish a snapshot + to npm under the `dev` dist-tag and exercise the release pipeline. Publishes + no documentation and never moves `latest`. +--- + +# dev-tag + +A development tag publishes a snapshot nobody will get by accident, so a real +install of real code can be tried before a release is promised. + +## The three kinds of tag + +| Tag | Cut from | npm dist-tag | Docs published | +| -------------- | ---------- | ------------ | -------------- | +| `v1.2.3-dev.1` | any branch | `dev` | no | +| `v1.2.3-rc.1` | `develop` | `next` | no | +| `v1.2.3` | `main` | `latest` | yes | + +Only a final tag moves `latest` and publishes documentation. `npm install +ailoud` therefore never picks up a dev tag, and the site never describes a +version nobody can install. + +Except once per package: npm sets `latest` on a FIRST publish whatever `--tag` +says, and `latest` cannot be removed afterwards, only moved. If a dev tag is +what introduces a package to the registry -- as v1.0.0-dev.1 was -- then +`latest` points at that snapshot until a final version is published. + +## Cutting one + +1. The working tree must be clean and the gate green: + + ``` + pnpm build && pnpm format:check && pnpm lint && pnpm typecheck && pnpm test:cov + ``` + +2. Set the version across the manifests. `-dev.` is a pre-release, so it + sorts BELOW the release it precedes: + + ``` + node scripts/bump-version.mjs 1.2.3-dev.1 + ``` + + `bump-version` promotes the CHANGES.md `## Development` section, which a dev + tag does not want -- so for a dev tag, set the versions by hand and leave + the changelog alone: + + ``` + node -e "for (const p of ['package.json','packages/core/package.json','packages/providers/package.json','apps/cli/package.json']) { const fs=require('fs'); const d=JSON.parse(fs.readFileSync(p,'utf8')); d.version='1.2.3-dev.1'; fs.writeFileSync(p, JSON.stringify(d,null,2)+'\n'); }" + ``` + +3. Commit, tag and push. The publish workflow checks that every manifest + agrees with the tag, so these cannot drift: + + ``` + git commit -am "chore: 1.2.3-dev.1" + git tag -s v1.2.3-dev.1 -m "v1.2.3-dev.1" + git push origin HEAD --tags + ``` + +4. Try it from the registry, which is the whole point: + + ``` + npm install -g ailoud@dev + ailoud --version + ``` + +## What it does not do + +- It does not publish documentation. `docs.yml` starts only on a final tag and + refuses a pre-release twice over. +- It does not move `latest`. +- It is not a release candidate. An `-rc.` tag says "this is what the release + will be"; a `-dev.` tag says "this is a snapshot to try". + +## Cleaning up + +A dev version cannot be unpublished after 72 hours and its number can never be +reused, so use a fresh `-dev.` each time rather than retrying one. + +Once the final release is out, retire every snapshot it supersedes in one step: + +``` +node scripts/retire-prereleases.mjs 1.2.3 # prints the plan +node scripts/retire-prereleases.mjs 1.2.3 --yes # carries it out +``` + +It deprecates the versions rather than unpublishing them, drops the `dev` +dist-tag, and deletes the tags -- but only those whose commit is reachable from +`main`. The provenance of a published package names both the commit and the tag +it was built from: delete the tag and the name stops resolving, which costs +convenience; delete a tag holding the only reference to its commit and the +commit itself can be collected, which costs the attestation its subject. Tags +in the second case are reported and left alone. diff --git a/.agents/skills/pre-release-check/SKILL.md b/.agents/skills/pre-release-check/SKILL.md index d6dcaad..9b3287f 100644 --- a/.agents/skills/pre-release-check/SKILL.md +++ b/.agents/skills/pre-release-check/SKILL.md @@ -1,10 +1,10 @@ --- name: pre-release-check description: > - Gate a release by running check-licenses, run-tests-and-linters, - check-fixtures, check-docs, and check-changes, plus verifying the version - bump and that all commits since the last release follow the - conventional-commits format. + Gate a release by running check-licenses, check-dependencies, + run-tests-and-linters, check-fixtures, check-docs, and check-changes, plus + verifying the version bump and that all commits since the last release + follow the conventional-commits format. --- # pre-release-check @@ -21,8 +21,11 @@ details): 1. **check-licenses** -- must run first because a license failure is the most fundamental blocker. -2. **run-tests-and-linters** -- lint, typecheck, and coverage at 90%. -3. **check-fixtures** -- drive the built binary against `fixtures/` end to +2. **check-dependencies** -- advisories, funding, and the 14-day rule on any + version taken. Runs early: an update it recommends changes what everything + below is testing, so taking one afterwards invalidates the whole gate. +3. **run-tests-and-linters** -- lint, typecheck, and coverage at 90%. +4. **check-fixtures** -- drive the built binary against `fixtures/` end to end. This is the only check that exercises the real binary against real files, so it catches wiring regressions the unit tests (which run against in-memory fakes) cannot see. Six of the twelve specs need a real @@ -31,12 +34,14 @@ details): not a release blocker on its own -- attribute every failure (missing whisper-cli, fixture drift, product change, harness defect) before deciding whether it blocks. -4. **check-docs** -- README.md, cross-references, command accuracy, +5. **check-docs** -- README.md, cross-references, command accuracy, version references. -5. **check-changes** -- CHANGES.md Development section vs. commit history. +6. **check-changes** -- CHANGES.md Development section vs. commit history. -If check-licenses or run-tests-and-linters fails, report the failure and -stop. If check-fixtures fails for a reason other than the missing +If check-licenses, check-dependencies or run-tests-and-linters fails, report +the failure and stop. A `--prod` advisory or a version younger than 14 days +blocks a release: the first ships a known vulnerability, and the second ships +a version nobody has had time to find one in. If check-fixtures fails for a reason other than the missing `whisper-cli` binary (fixture drift, a product change, or a harness defect), treat it the same way -- a release must not ship while tests, licenses, or the end-to-end run are failing for a reason within this @@ -93,6 +98,7 @@ Produce a release-readiness summary: ``` check-licenses: PASS / FAIL +check-dependencies: PASS / FAIL (advisories: prod / dev only; ages: ok / N too young) run-tests-and-linters: PASS / FAIL check-fixtures: PASS / FAIL (attribute: missing whisper-cli / fixture drift / product change / harness defect) check-docs: PASS / FAIL diff --git a/.agents/skills/run-tests-and-linters/SKILL.md b/.agents/skills/run-tests-and-linters/SKILL.md index 3b42bdb..685906d 100644 --- a/.agents/skills/run-tests-and-linters/SKILL.md +++ b/.agents/skills/run-tests-and-linters/SKILL.md @@ -9,7 +9,7 @@ description: > # run-tests-and-linters -Run the full quality gate for laud before marking any task done. `laud` is +Run the full quality gate for ailoud before marking any task done. `ailoud` is a pure TypeScript pnpm workspace, so the gate has no Rust or native half to run alongside it. diff --git a/.gitattributes b/.gitattributes index 4ac1b58..f92e46c 100644 --- a/.gitattributes +++ b/.gitattributes @@ -5,4 +5,13 @@ *.mp3 filter=lfs diff=lfs merge=lfs -text *.m4a filter=lfs diff=lfs merge=lfs -text +# Video fixtures: `ailoud audio import` accepts video because a meeting +# recording usually is one, and the four containers it maps to a MIME type each +# get one. They are tiny (under 15 kB) but binary, so LFS keeps them out of the +# packfile like the audio. +*.mp4 filter=lfs diff=lfs merge=lfs -text +*.mov filter=lfs diff=lfs merge=lfs -text +*.mkv filter=lfs diff=lfs merge=lfs -text +*.webm filter=lfs diff=lfs merge=lfs -text + pnpm-lock.yaml linguist-generated=true diff --git a/.github/actions/setup-node-pnpm/action.yml b/.github/actions/setup-node-pnpm/action.yml new file mode 100644 index 0000000..3ac6725 --- /dev/null +++ b/.github/actions/setup-node-pnpm/action.yml @@ -0,0 +1,38 @@ +# The Node and pnpm setup every job shares, in one place because it is four +# jobs across two workflows and the corepack part needs care. +# +# `corepack enable` only writes shims; the pinned pnpm is downloaded lazily by +# whatever invokes it first, which used to be setup-node's cache probe. That +# download has failed there -- an undici assertion inside Node 24.20.0 when the +# connection ends mid-response -- and a failure inside another action's step +# cannot be retried. Downloading it here instead puts the network call in a +# step of our own, so a flake costs five seconds rather than the run. + +name: Set up Node.js and pnpm +description: Activate the pinned pnpm, then Node 24 with the pnpm store cached. + +runs: + using: composite + steps: + - name: Activate the pnpm pinned in package.json + shell: bash + env: + # Without this corepack prints a download notice that GitHub renders as + # an error annotation on a run that is fine. + COREPACK_ENABLE_DOWNLOAD_PROMPT: '0' + run: | + corepack enable + for attempt in 1 2 3; do + if corepack install; then break; fi + echo "corepack install failed (attempt ${attempt} of 3); retrying in 5s" + sleep 5 + done + # Fails the step if all three attempts did, rather than leaving the + # next job step to fail with something less specific. + pnpm --version + + - name: Set up Node.js 24 with pnpm cache + uses: actions/setup-node@v5 + with: + node-version: '24' + cache: pnpm diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml new file mode 100644 index 0000000..49afe3e --- /dev/null +++ b/.github/codeql/codeql-config.yml @@ -0,0 +1,32 @@ +# CodeQL configuration for AILoud. +# +# Queries are named here rather than in the workflow's `queries:` input so the +# reason for each exclusion can sit next to it. + +name: AILoud + +queries: + - uses: security-and-quality + +query-filters: + # js/file-system-race, three times over, always on apps/cli/src/setupLock.ts. + # + # That file's whole job is to be the thing a file-system race goes through. + # It acquires with `open(path, 'wx')` -- one atomic syscall -- and takes over + # a stale lock under a SECOND lock file (`provisioning.lock.steal`, also + # `wx`), which is what makes the takeover exclusive. The analysis sees a read + # of one path followed by a write to it and reports a race; it cannot see + # that the exclusion is held by a different file. + # + # Two of the three reports were nonetheless real, and both are fixed: the + # first version deleted a live lock, the second let two runs each read back + # their own pid and proceed. What stands in for this query now is a + # concurrency test (`setupLock.test.ts`, "under contention") that runs two + # processes against the compiled lock and fails on either old implementation + # -- an actual measurement rather than a pattern match. + # + # Excluded by id, so the rule is off everywhere. It has produced nothing but + # this file and test code in this repository; if that changes, the honest fix + # is to narrow this with a `paths` filter rather than keep the noise. + - exclude: + id: js/file-system-race diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..480e7d6 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,59 @@ +# Dependabot, held to the same rule as a human: nothing younger than 14 days. +# +# `cooldown` is what makes that true. Without it Dependabot opens a pull +# request the moment a version appears, which is precisely the window +# `scripts/check-dependency-age.mjs` exists to refuse -- the check would then +# fail on Dependabot's own branch and the two would be arguing. +# +# Security updates deliberately ignore cooldown, and that is the behaviour we +# want: a known advisory beats an unaudited release, which is the same +# exception `scripts/dependency-age-exceptions.json` records for a human. +# +# See .agents/skills/check-dependencies/SKILL.md for the reasoning, and +# AGENTS.md for how a dependency change reaches a release. + +version: 2 + +updates: + - package-ecosystem: npm + # The workspace root: one pnpm-lock.yaml covers all four manifests. + directory: / + schedule: + interval: weekly + day: monday + cooldown: + default-days: 14 + # Exact pins are this project's convention, and `increase` preserves them. + # `widen` would turn a pin into a range and hand the choice of version to + # whatever resolved last, which no age check can judge. + versioning-strategy: increase + open-pull-requests-limit: 5 + groups: + # The dev toolchain moves constantly and cannot reach a user, so it + # arrives as one reviewable pull request. Anything that ships gets its + # own, because a runtime dependency deserves its own reading. + dev-toolchain: + dependency-type: development + ignore: + # typescript-eslint 8.67.0 refuses to load against TS 7.0 outright + # ("typescript-eslint does not support TS 7.0"), so the bump Dependabot + # opened could not lint -- it is blocked by a peer, not by us. Tracked as + # typescript-eslint#10940; drop this entry when it supports TS >= 7.1. + - dependency-name: typescript + update-types: ['version-update:semver-major'] + labels: + - dependencies + + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + day: monday + cooldown: + default-days: 14 + groups: + actions: + patterns: + - '*' + labels: + - dependencies diff --git a/.github/workflows/backmerge.yml b/.github/workflows/backmerge.yml index 36a3d5d..e47c8f7 100644 --- a/.github/workflows/backmerge.yml +++ b/.github/workflows/backmerge.yml @@ -27,13 +27,21 @@ jobs: open-backmerge-pr: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 with: fetch-depth: 0 - name: Skip if develop already contains main id: check run: | + set -uo pipefail + # Said plainly rather than dying on `fatal: Not a valid object name`, + # which is what the first run did before develop existed. + if ! git rev-parse --verify --quiet origin/develop >/dev/null; then + echo "needs_pr=false" >> "$GITHUB_OUTPUT" + echo "::notice::there is no develop branch, so there is nothing to back-merge into." + exit 0 + fi if git merge-base --is-ancestor origin/main origin/develop; then echo "needs_pr=false" >> "$GITHUB_OUTPUT" echo "develop already contains main; nothing to do." diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8e8bfca..0efcdb4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -46,18 +46,12 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 with: lfs: true - - name: Enable pnpm via corepack - run: corepack enable - - - name: Set up Node.js 24 with pnpm cache - uses: actions/setup-node@v5 - with: - node-version: '24' - cache: pnpm + - name: Set up Node.js 24 and pnpm + uses: ./.github/actions/setup-node-pnpm - name: Install ffmpeg # packages/providers/src/audio/ffmpeg.test.ts spawns the real binary to @@ -96,18 +90,12 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 with: lfs: true - - name: Enable pnpm via corepack - run: corepack enable - - - name: Set up Node.js 24 with pnpm cache - uses: actions/setup-node@v5 - with: - node-version: '24' - cache: pnpm + - name: Set up Node.js 24 and pnpm + uses: ./.github/actions/setup-node-pnpm - name: Install dependencies (frozen lockfile) run: pnpm install --frozen-lockfile @@ -128,20 +116,14 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 with: # The audio fixtures are LFS-tracked; without this the suite # transcribes pointer files. lfs: true - - name: Enable pnpm via corepack - run: corepack enable - - - name: Set up Node.js 24 with pnpm cache - uses: actions/setup-node@v5 - with: - node-version: '24' - cache: pnpm + - name: Set up Node.js 24 and pnpm + uses: ./.github/actions/setup-node-pnpm - name: Install dependencies (frozen lockfile) run: pnpm install --frozen-lockfile @@ -149,20 +131,79 @@ jobs: - name: Cache the whisper models # Keyed on the catalogue, which holds the pinned file names, sizes and # URLs: a model change invalidates the cache, and nothing else does. - uses: actions/cache@v4 + uses: actions/cache@v6 with: path: ~/.local/share/ailoud/models key: ailoud-models-${{ runner.os }}-${{ hashFiles('packages/core/src/provision/catalogue.ts') }} + - name: Dependency age (advisory) + # Reports, never blocks. Dependabot's version updates already respect + # the same 14 days through `cooldown` in .github/dependabot.yml, so the + # two do not argue -- but a Dependabot SECURITY update deliberately + # ignores cooldown, and a fresh version that closes an advisory is the + # one case where taking it beats waiting. Failing the build there would + # make the rule an obstacle to the thing it exists to protect. + # + # Not run on tags: ci.yml does not trigger on them, and the release + # path asks the question through the check-dependencies skill instead. + run: | + if ! node scripts/check-dependency-age.mjs; then + echo "::warning::A direct dependency is younger than 14 days, or its age could not be" + echo "::warning::checked. If it closes a critical advisory, record it in" + echo "::warning::scripts/dependency-age-exceptions.json with the advisory ID; otherwise" + echo "::warning::pin the previous version. See .agents/skills/check-dependencies." + fi + - name: Build run: pnpm build - - name: Provision ffmpeg, whisper.cpp and the models - # The same command a user runs, which makes this a test of `setup` as - # well as a prerequisite for the suite. --llm skip because no spec here - # summarises, and it would otherwise download another 2.1 GB. + - name: Install ffmpeg + # Installed with apt rather than left to `setup`, which correctly + # refuses to run `sudo apt-get` with no terminal to answer a password + # prompt on -- it reports the exact command instead of hanging. There + # is no terminal on a runner, so the sudo step is ours to do. + run: sudo apt-get update -qq && sudo apt-get install -y -qq ffmpeg + + - name: Provision whisper.cpp and the models + # Everything that needs no sudo: the whisper release and the model + # files. Still the command a user runs, so this remains a test of + # `setup` and not only a prerequisite for the suite. --llm skip because + # no spec here summarises, and it would otherwise fetch another 2.1 GB. run: node apps/cli/dist/bin/ailoud.js setup --yes --llm skip + - name: Put the provisioned binaries on PATH + # The e2e sandbox writes its own config naming only the MODEL, and + # leaves `binary` at its default -- which means whisper-cli has to be + # found on PATH. `setup` installs it under the data directory and + # records the absolute path in the user's config instead, so doctor was + # entirely green while every transcribe spec exited 3. + # + # The directories are read back out of that config rather than spelled + # here, so bumping a pinned release does not silently break this. + run: | + set -euo pipefail + read_dir() { + node -e " + const { parseConfig } = require('./apps/cli/dist/config.js'); + const { readFileSync } = require('node:fs'); + const { homedir } = require('node:os'); + const { dirname } = require('node:path'); + const config = parseConfig( + readFileSync(homedir() + '/.config/ailoud/config.yaml', 'utf8'), + ); + const value = $1; + if (value && value.includes('/')) console.log(dirname(value)); + " + } + for dir in \ + "$(read_dir 'config.stt.whisperCpp.binary')" \ + "$(read_dir 'config.stt.diarization.binary')"; do + if [ -n "$dir" ]; then + echo "adding $dir to PATH" + echo "$dir" >> "$GITHUB_PATH" + fi + done + - name: Report what the machine now has # Printed whether or not the suite passes: a red e2e run is far quicker # to read next to doctor's account of what was actually installed. @@ -178,7 +219,7 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 - name: Install uv uses: astral-sh/setup-uv@v7 diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 0000000..c14bbe5 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,51 @@ +# CodeQL, GitHub's static analysis, over the TypeScript in this workspace. +# +# Committed rather than enabled through the repository's default setup, for the +# same reason every other check here is a file: what runs, when, and over what +# should be reviewable in a diff. +# +# The weekly run matters as much as the per-push one -- most findings arrive +# because the queries improved, not because the code changed. + +name: CodeQL + +on: + push: + branches: [main, develop] + pull_request: + schedule: + # Monday morning, after Dependabot has opened whatever it is going to. + - cron: '30 6 * * 1' + +concurrency: + group: codeql-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + analyze: + name: Analyze TypeScript + runs-on: ubuntu-latest + timeout-minutes: 30 + + permissions: + # security-events: write is how the results reach the Security tab; a run + # without it analyses the code and then throws the answer away. + security-events: write + contents: read + + steps: + - name: Checkout repository + uses: actions/checkout@v7 + + - name: Initialize CodeQL + uses: github/codeql-action/init@v4 + with: + languages: javascript-typescript + # The query set and the one rule this project excludes live in the + # config file, where each can carry its reason. security-and-quality + # over the default security-extended: this is a small codebase where + # the quality queries are worth reading rather than noise to filter. + config-file: .github/codeql/codeql-config.yml + + - name: Analyze + uses: github/codeql-action/analyze@v4 diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 2b17b60..03d7856 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -29,12 +29,13 @@ name: Docs on: - push: - tags: - # Final releases only: the negative pattern excludes any pre-release tag - # (a `-` qualifier), so an RC never starts this workflow at all. - - 'v*' - - '!v*-*' + # After a successful publish, not beside it. On the tag push both workflows + # started at once, so a publish that then refused -- or failed halfway + # through its three packages -- left the site advertising a version npm did + # not have. `workflow_run` waits for the answer. + workflow_run: + workflows: [Publish] + types: [completed] workflow_dispatch: # mike pushes commits to the gh-pages branch. @@ -50,11 +51,22 @@ jobs: deploy: name: Build and deploy versioned docs runs-on: ubuntu-latest + timeout-minutes: 20 + # A publish that failed publishes no documentation. Whether the commit is a + # final release is decided in the job, from the manifest in the commit + # itself -- `head_branch` on a workflow_run is documented as a branch, and + # a release must not rest on what it happens to hold for a tag. + if: >- + ${{ github.event_name == 'workflow_dispatch' || + github.event.workflow_run.conclusion == 'success' }} steps: - - name: Checkout repository - uses: actions/checkout@v5 + - name: Checkout the commit that was published + uses: actions/checkout@v7 with: + # The SHA, which is unambiguous. Without a ref this would check out + # the default branch, which is not necessarily what was published. + ref: ${{ github.event.workflow_run.head_sha || github.ref }} # mike needs full history plus the gh-pages branch it maintains. fetch-depth: 0 @@ -73,15 +85,13 @@ jobs: run: | set -euo pipefail - if [ "${GITHUB_REF_TYPE}" != "tag" ]; then - echo "::error::Docs publish only from a release tag; ${GITHUB_REF_NAME} is not one." - exit 1 - fi - - version="${GITHUB_REF_NAME#v}" - # The tag filter already keeps pre-releases out, but - # workflow_dispatch can be pointed at any ref -- so refuse here too - # rather than trusting the trigger alone. + # The version of the thing that was published, from the commit that + # was published. publish.yml refuses a tag that disagrees with any + # manifest, so this and the tag are the same number by then. + version="$(node -p "require('./package.json').version")" + # publish.yml runs for pre-release tags too, so this workflow starts + # for them and stops here. A pre-release must not move the site: the + # docs describe what `npm install ailoud` gives you. case "${version}" in *-*) echo "${version} is a pre-release; skipping docs publication." diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 226ccf9..354160f 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -43,8 +43,12 @@ on: # id-token is what makes trusted publishing work; without it npm has no OIDC # token to exchange and falls back to asking for a credential that is not there. permissions: - contents: read + # contents: write is for creating the GitHub release. + contents: write id-token: write + # Reading code scanning results, to refuse a release that ships a finding + # nobody has looked at. + security-events: read concurrency: group: npm-publish @@ -56,22 +60,29 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 30 + env: + # The tag being released, resolved once. `github.ref_name` is the tag on + # a tag push but the BRANCH on a workflow_dispatch, so a step that let it + # default -- the changelog check did -- looked for a "## Version main" + # section and failed a re-run of a perfectly good tag. + TAG: ${{ github.event.inputs.tag || github.ref_name }} + steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 with: ref: ${{ github.event.inputs.tag || github.ref }} lfs: true - - name: Enable pnpm via corepack - run: corepack enable - - - name: Set up Node.js 24 with pnpm cache - uses: actions/setup-node@v5 - with: - node-version: '24' - cache: pnpm - registry-url: 'https://registry.npmjs.org' + - name: Set up Node.js 24 and pnpm + # Deliberately WITHOUT setup-node's registry-url. Given one, it writes + # an .npmrc containing `_authToken=${NODE_AUTH_TOKEN}`; with no token + # secret to fill it, npm still saw credentials, sent an empty one, and + # never attempted the OIDC exchange -- the registry answered 404 on the + # PUT and the log said nothing about trusted publishing at all. With no + # .npmrc, npm finds it has no credentials and does the exchange. The + # default registry is registry.npmjs.org either way. + uses: ./.github/actions/setup-node-pnpm - name: Require an npm that can do trusted publishing # Trusted publishing landed in npm 11.5.1. An older npm fails by asking @@ -87,15 +98,103 @@ jobs: fi echo "npm $have" + - name: Refuse a final release while NPM_TOKEN is set + # First, not last. This used to live inside the publish step, after + # pnpm install, three checks, apt-get, the whole gate and the + # end-to-end suite -- around twenty minutes to learn a fact known at + # second zero. Nothing was ever published by it, so the cost was only + # time, but the documented order said otherwise and this is the most + # likely way a first production release fails. + # + # The bootstrap is over the moment the packages exist, and a token that + # keeps working is a token nobody gets round to removing. + env: + NPM_TOKEN: ${{ secrets.NPM_TOKEN }} + run: | + set -euo pipefail + version="${TAG#v}" + if [ -z "${NPM_TOKEN:-}" ]; then + echo "no NPM_TOKEN secret; this run will use trusted publishing" + exit 0 + fi + case "$version" in + *-*) + echo "::warning::publishing $version with the NPM_TOKEN secret; attach the trusted publisher and delete the secret" + ;; + *) + echo "::error::$version is a final release and NPM_TOKEN is set." + echo "::error::Attach the trusted publisher on all three package pages" + echo "::error::(organization lorem-dev, repository ailoud, workflow publish.yml," + echo "::error::environment empty), then delete the NPM_TOKEN secret and re-run." + exit 1 + ;; + esac + - name: Install dependencies (frozen lockfile) run: pnpm install --frozen-lockfile + - name: Check the changelog is fit to release + # First, and before the gate: a version number can never be reused and + # the unpublish window is 72 hours, so every reason to refuse should be + # found while refusing is still free. This checks the section exists and + # has entries, that it is inside the hard limit, that nothing is + # stranded under Development, and -- for a final tag -- that its + # pre-release sections were folded. + # + # The scripts' own tests run in ci.yml, on branches and pull requests + # rather than on tags, so a broken script is caught before a release + # depends on it. + run: node scripts/check-changelog.mjs "$TAG" + + - name: Refuse a release carrying unreviewed high-severity findings + # The dependency side of this is already enforced -- a --prod advisory + # or a version younger than 14 days blocks a release -- while what + # CodeQL finds in our own code did not reach here at all. This closes + # that asymmetry. + # + # Only `state=open` counts. A finding that was reviewed and explained + # is `dismissed` with its reason attached, and does not block; one + # nobody has looked at does. Fixing it and dismissing it are both + # answers -- ignoring it is not. + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + # --paginate, because an alert past the first hundred blocked nothing. + # jq -s 'add' folds the pages into one array. + if ! alerts=$(gh api --paginate \ + "repos/${{ github.repository }}/code-scanning/alerts?state=open&per_page=100" \ + | jq -s 'add // [] | [.[] | select(.rule.security_severity_level == "high" or .rule.security_severity_level == "critical")]'); then + echo "::error::could not read code scanning results. A release that cannot check" + echo "::error::whether it ships a known finding is not one to make blind; fix the" + echo "::error::access or the analysis, then re-run." + exit 1 + fi + count=$(jq 'length' <<< "$alerts" || true) + # An empty or malformed answer used to pass: `jq length` printed + # nothing, `[ "" -gt 0 ]` errored, and a failing `if` condition is not + # caught by `set -e`, so the else branch reported "no alerts". + case "$count" in + '' | *[!0-9]*) + echo "::error::could not count the code scanning alerts (got '${count}')." + echo "::error::Refusing rather than assuming there are none." + exit 1 + ;; + esac + if [ "$count" -gt 0 ]; then + jq -r '.[] | "::error::\(.rule.security_severity_level) \(.rule.id) at \(.most_recent_instance.location.path):\(.most_recent_instance.location.start_line)"' <<< "$alerts" + echo "::error::${count} open high or critical code scanning alert(s). Fix them, or" + echo "::error::dismiss each with a reason, then re-run this workflow on the tag." + exit 1 + fi + echo "no open high or critical code scanning alerts" + - name: Check that the tag matches the version in the manifests # A tag that disagrees with package.json publishes a version nobody # asked for, under a tag that points at different code. run: | set -euo pipefail - tag="${{ github.event.inputs.tag || github.ref_name }}" + tag="$TAG" want="${tag#v}" for manifest in packages/core packages/providers apps/cli; do have=$(node -p "require('./$manifest/package.json').version") @@ -106,6 +205,13 @@ jobs: done echo "all three manifests are at $want" + - name: Install ffmpeg + # The gate includes packages/providers/src/audio/ffmpeg.test.ts, which + # spawns the real binary. CI installs it for the same reason; this job + # runs the same gate, so it needs the same tool. Without it the release + # failed here rather than on anything about the release. + run: sudo apt-get update -qq && sudo apt-get install -y -qq ffmpeg + - name: Run the gate # Nothing is published that has not built, linted, typechecked and # passed its tests on this runner. An unpublish window is 72 hours and @@ -130,28 +236,134 @@ jobs: done ls -l dist-npm + - name: Check each tarball carries the licence and no source + # pnpm copies the repository's LICENSE into every workspace tarball, + # which is why no package directory holds its own copy -- three copies + # of a 224-line file would drift. That makes it an invariant worth + # checking rather than assuming: if pnpm ever stops, this fails the + # release instead of publishing an unlicensed package. + # + # The same step guards the other packing mistake: shipping `src` and + # the tests. Before `files` was set, the CLI tarball carried 82 source + # files and 32 test files. + run: | + set -euo pipefail + for tarball in dist-npm/*.tgz; do + echo "--- $tarball" + contents=$(tar -tzf "$tarball") + if ! grep -qiE '^package/LICEN[SC]E' <<< "$contents"; then + echo "::error::$tarball has no LICENSE." + exit 1 + fi + if grep -qE '^package/src/' <<< "$contents"; then + echo "::error::$tarball ships src/; check the \`files\` field." + exit 1 + fi + if grep -qE '\.test\.' <<< "$contents"; then + echo "::error::$tarball ships tests; check the \`files\` field." + exit 1 + fi + # 1.0.0-dev.1 went out with no README, so all three npm pages read + # "This package does not have a README" -- the first thing anyone + # arriving from a search sees. The libraries hold their own; the + # CLI's is the repository's, copied in by its prepack script, which + # is the part that could silently stop working. + if ! grep -qiE '^package/README' <<< "$contents"; then + echo "::error::$tarball has no README." + exit 1 + fi + echo "licence, readme, no source, no tests" + done + - name: Publish, dependencies first # Order matters: the CLI's manifest names exact versions of the two # libraries, so they have to exist first. --provenance records which # workflow and which commit produced each tarball, verifiable from the - # package page. + # package page -- it works under either credential below. + env: + NPM_TOKEN: ${{ secrets.NPM_TOKEN }} run: | set -euo pipefail - tag="${{ github.event.inputs.tag || github.ref_name }}" + tag="$TAG" version="${tag#v}" - # A pre-release goes out under `next`, so `npm install ailoud` keeps - # returning the last stable version until a final tag moves `latest`. + # BOOTSTRAP ONLY. Trusted publishing is configured per package on + # npmjs.com, which cannot be done for a package that does not exist + # yet -- so the first version of each of the three has to go out on a + # token, and npm answers ENEEDAUTH without one however complete the + # OIDC setup is. Once the trusted publisher is attached to all three + # package pages, delete the NPM_TOKEN secret: with it gone this + # branch is skipped and npm does the OIDC exchange instead, which is + # the arrangement with nothing to expire. + # Whether a token is allowed here at all was settled in the first + # step of this job; this only applies it. + if [ -n "${NPM_TOKEN:-}" ]; then + npm config set //registry.npmjs.org/:_authToken "$NPM_TOKEN" + fi + # Three kinds of tag, three destinations. See the dev-tag skill. + # -dev.N a snapshot to try -> dev + # -rc.N what the release will be -> next + # final the release -> latest + # Only the last moves `latest`, so `npm install ailoud` never picks up + # a pre-release by accident -- with one exception nothing here can + # prevent: npm sets `latest` on a package's first publish whatever + # --tag says, and `latest` can be moved but never removed. case "$version" in - *-*) dist_tag=next ;; - *) dist_tag=latest ;; + *-dev.*) dist_tag=dev ;; + *-*) dist_tag=next ;; + *) dist_tag=latest ;; esac + # Everything checkable, checked before the first publish. The loop + # goes library, library, CLI, and a version number npm has seen can + # never be reused -- so a fault found on the third iteration leaves + # two packages published at a version the release can no longer use. + for name in ailoud-core ailoud-providers ailoud; do + # An ABSOLUTE path. Given `dist-npm/ailoud-core-1.0.0-dev.1.tgz`, + # npm read the slash as the `owner/repo` GitHub shorthand and tried + # to `git ls-remote ssh://git@github.com/dist-npm/...git`, which + # failed on a missing public key rather than on anything to do with + # publishing. + if [ ! -f "$PWD/dist-npm/${name}-${version}.tgz" ]; then + echo "::error::dist-npm/${name}-${version}.tgz was not packed." + exit 1 + fi + done + node scripts/preflight-npm-auth.mjs + echo "publishing $version under dist-tag $dist_tag" for name in ailoud-core ailoud-providers ailoud; do - tarball=$(ls dist-npm/${name}-${version}.tgz) + tarball="$PWD/dist-npm/${name}-${version}.tgz" echo "--- $tarball" npm publish "$tarball" --provenance --access public --tag "$dist_tag" done + - name: Create the GitHub release from CHANGES.md + # After the publish, so a release object never advertises a version the + # registry refused. `gh release create` rather than a third-party + # action: adding one to the release path would sit oddly beside a + # 14-day rule for everything else we depend on. + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + version="${TAG#v}" + # Writes RELEASE_NOTES.md from the `## Version ` section, + # and refuses if that section is missing, empty or over the limit. + node scripts/release-notes.mjs "$TAG" + prerelease="" + case "$version" in + *-*) prerelease="--prerelease" ;; + esac + if gh release view "$TAG" >/dev/null 2>&1; then + echo "release $TAG already exists; updating its notes" + gh release edit "$TAG" --notes-file RELEASE_NOTES.md + else + gh release create "$TAG" \ + --title "$TAG" \ + --notes-file RELEASE_NOTES.md \ + --verify-tag \ + $prerelease + fi + - name: Report what was published if: always() run: ls -l dist-npm || true diff --git a/.gitignore b/.gitignore index 6c04168..7352d59 100644 --- a/.gitignore +++ b/.gitignore @@ -26,3 +26,7 @@ docs/superpowers/**/scratch/ RELEASE_NOTES.md # npm tarballs built by the publish workflow dist-npm/ + +# Copied from the repository root by apps/cli prepack, so the npm page for +# `ailoud` shows the real README without a second copy in git. +apps/cli/README.md diff --git a/AGENTS.md b/AGENTS.md index 9652edd..5c28229 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,12 +18,25 @@ resulting text. No microphone or system-audio capture; import only. The CLI is the only front end -- there is no GUI, and the interface is English-only. The multilingual part of this project is the audio, not the UI. -M1, the current milestone, ships `import`, `transcribe`, `ls`, `show`, and -`doctor`. The rest of the CLI surface (`search`, `collection`, `tag`, -`summarize`, `export`, `config`) belongs to later milestones and does not -exist yet; do not document or assume commands beyond what M1 lists. The full -design lives in the maintainer's planning notes under `.superpowers/`, which -is not tracked in git, so treat this file as the authority on what exists. +The commands, grouped by the noun they act on: + +| Command | Does | +| ---------------------------------------------------------- | ----------------------------------------- | +| `audio import\|transcribe\|annotate\|search\|ls\|show\|rm` | the library and everything over it | +| `audio summarize` | writes a summary and saves it as a report | +| `report ls\|show\|rm` | saved reports | +| `template ls\|new` | what shape a summary of a kind takes | +| `mcp` and `mcp install\|uninstall\|update` | serve the library to an agent | +| `doctor`, `setup` | check and provision the machine | + +The verbs also answer at the top level (`ailoud search`, `ailoud transcribe`), +and each second-level verb has a one-letter alias. + +Do not trust a list of commands in prose over the binary. This section was +wrong for a while -- it described an early milestone and said `search` and +`summarize` "do not exist yet" long after both shipped -- so check with +`node apps/cli/dist/bin/ailoud.js --help` after a build, which cannot be +stale. --- @@ -257,6 +270,180 @@ Run the `check-docs` skill after changing any command or option. --- +## Branches and Tags + +| Branch | Is | +| --------- | --------------------------------------- | +| `main` | the release branch | +| `develop` | integration; feature branches land here | + +**While the project is pre-release, work happens directly on `main`** and +`develop` follows it: a push to `main` opens a back-merge PR +(`.github/workflows/backmerge.yml`). Once releases start that inverts -- +features land on `develop`, only release commits reach `main` -- and the same +workflow keeps `develop` from falling behind either way. Both branches are +protected against deletion and force-push with CI required; an admin can still +push directly, which is what makes the pre-release flow possible. + +The three required checks are the ones that run on a pull request. The +provisioned end-to-end job is deliberately NOT required: it runs only on push, +so requiring it would leave every PR waiting for a check that never arrives. + +| Tag | Cut from | npm dist-tag | Docs | Retires snapshots | +| -------------- | ---------- | ------------ | ---- | ----------------- | +| `v1.2.3-dev.1` | any branch | `dev` | no | no | +| `v1.2.3-rc.1` | `develop` | `next` | no | no | +| `v1.2.3` | `main` | `latest` | yes | yes | + +Only a final tag moves `latest`, publishes the site and retires what it +supersedes. `publish.yml` refuses a tag that disagrees with any manifest +version. + +Use the `dev-tag` skill for a snapshot; `bump-version` then `pre-release-check` +for a release. + +Cutting a final tag folds its pre-release sections back into one. The order +matters, and it is the opposite of what reads naturally: + +``` +node scripts/bump-version.mjs 1.0.0 # FIRST: promotes Development +node scripts/fold-prereleases.mjs 1.0.0 # then merges the 1.0.0-dev.* sections in +node scripts/check-changelog.mjs v1.0.0 # refuses if anything is left over +``` + +`bump-version` promotes `## Development` into a `## Version ` section, so +folding first leaves it a second, empty one to create -- two `## Version 1.0.0` +headings, and `check-changelog` then fails on the empty one with "has no +entries". Tested both ways in a sandbox before the first release. + +`publish.yml` runs that check on every tag before it builds anything. The +limits live in `scripts/lib/changelog.mjs` and are quoted, not restated, +everywhere else -- a limit that differs between the script that warns and the +script that refuses is worse than no limit. The scripts have tests +(`scripts/**/*.test.mjs`), which run in CI on branches and pull requests but +not on tags. + +--- + +## Publishing + +Nothing long-lived is stored. Publishing authenticates by exchanging the CI +job's OIDC identity for a short-lived, per-package npm token. Retiring the +snapshots a release supersedes does NOT -- see below; it is a manual step. + +### How the exchange works + +`npm publish` does this for itself, and `scripts/lib/npmOidc.mjs` makes the +same two calls (read out of npm's `lib/utils/oidc.js`) so that +`preflight-npm-auth.mjs` can establish before publishing that npm will accept +every package: + +``` +GET $ACTIONS_ID_TOKEN_REQUEST_URL&audience=npm:registry.npmjs.org + Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN -> { value } +POST /-/npm/v1/oidc/token/exchange/package/ + Authorization: Bearer -> { token } +``` + +Three consequences worth knowing before changing any of this: + +- **`id-token: write` is required**, or GitHub mints no identity and the first + call is skipped entirely. +- **The token it returns authenticates `npm publish` and nothing else.** It is + publish-scoped and spent: `npm deprecate` with it answered `E404 ... or you +do not have permission`, then `E401 ... token is invalid` on every call + after. Measured on the 1.0.0 release. So the exchange can answer "will npm + accept us", and cannot be used to do anything but publish. +- **A trusted publisher is bound to a workflow FILE.** Entering through a + reusable workflow leaves `workflow_ref` pointing at the caller and + `job_workflow_ref` at the callee; npm matches the caller, which the 1.0.0 + release also confirmed. +- **The token never reaches a command line or a log.** It goes into a + temporary 0600 `.npmrc` removed in a `finally`; `withNpmToken` is tested + both for the normal path and for a body that throws. The id token's claims + ARE logged -- `sub`, `workflow_ref`, `job_workflow_ref` -- because a + rejection is otherwise indistinguishable from a missing package. + +### The bootstrap exception + +A trusted publisher is attached to a package that already exists, so the FIRST +version of each package cannot use one: npm answers `ENEEDAUTH` however +complete the setup is. A token in the `NPM_TOKEN` secret introduces the +packages to the registry, and deleting the secret is the whole of the switch -- +`publish.yml` uses it when present and OIDC when not. + +The rule is enforced rather than remembered, because a token that keeps working +is a token nobody removes: + +| Tag | `NPM_TOKEN` set | Result | +| ----------- | --------------- | -------------------------- | +| pre-release | yes | publishes, logs a warning | +| final | yes | **refuses**, names the fix | +| any | no | OIDC exchange | + +### What blocks a release + +Enforced by `publish.yml`, all of it before anything is built or published: + +| Refuses when | Why | +| ---------------------------------------------- | ------------------------------------------- | +| the changelog is unfit (`check-changelog.mjs`) | a release nobody can read the notes for | +| an open high or critical code scanning alert | a finding nobody has looked at, shipped | +| a manifest disagrees with the tag | publishes a version nobody asked for | +| `NPM_TOKEN` is set on a final tag | a stored credential outliving its bootstrap | +| the gate or the end-to-end suite fails | the ordinary reasons | +| a tarball lacks its LICENCE or README | what 1.0.0-dev.1 shipped | + +A code scanning finding counts only while it is `open`. One reviewed and +explained is `dismissed` with its reason attached and does not block; fixing +and dismissing are both answers, ignoring is not. The dependency side of the +same question is the `check-dependencies` skill, which a human runs before +tagging. + +### Retiring superseded snapshots + +Only a production release retires anything, and it happens after the publish +succeeds -- deprecating first would, if the publish then failed, leave every +`-dev.N` pointing at a release that does not exist while `dev` is the only +installable thing. + +``` +node scripts/retire-prereleases.mjs 1.0.0 # prints the plan +node scripts/retire-prereleases.mjs 1.0.0 --yes # carries it out +``` + +This is a manual step, run under `npm login`. Automating it was tried and +removed: the OIDC token cannot deprecate (above). If the npm side does not +complete, the script leaves the tags alone and exits non-zero -- the tags are +what name which pre-releases to retire, so deleting them after a failed +deprecation would destroy the only record of what was missed. + +It deprecates every `1.0.0-dev.*` of all three packages, drops the `dev` +dist-tag, and deletes the tags. Two things it deliberately does not do: + +- **Unpublish.** Allowed for 72 hours only, the version number can never be + reused after, and anyone who pinned it has their install broken. A deprecated + version keeps working and prints a notice. +- **Delete a tag whose commit is not on `origin/main`.** The published + provenance attests that commit; unreachable, it can be collected, leaving the + attestation pointing at nothing. Those tags are reported and kept. + +### npm facts that constrain all of the above + +None of these can be worked around, so design around them: + +- **A version number is used up forever.** `1.0.0-dev.1` cannot be republished + even after an unpublish. A botched snapshot needs a new number, not a retry. +- **`latest` is set on a package's first publish** whatever `--tag` says, and + `latest` can be moved but never removed. A package introduced by a + pre-release answers `npm install ` with it until a final version + exists. +- **`npm publish` is the only thing trusted publishing authenticates.** + `deprecate` and `dist-tag` need a real credential, which in practice means a + human at a terminal. + +--- + ## The Changelog `CHANGES.md` is for someone deciding whether to upgrade. It is not a commit @@ -319,7 +506,7 @@ about to add an entry will actually see them. ## Local Development Skills -Seven skills live under `.agents/skills/`. Invoke them when the situation +Nine skills live under `.agents/skills/`. Invoke them when the situation calls for it: | Skill | When to use | @@ -330,7 +517,9 @@ calls for it: | `check-licenses` | After editing any `package.json` -- verify all npm dependencies are license-compliant and update LICENSE. | | `run-tests-and-linters` | Before marking any task done -- run the full gate (build, format check, lint, typecheck, test:cov at 90%). | | `check-fixtures` | After touching import, transcribe, or the audio/STT providers -- drive the built binary against `fixtures/` end to end, in a throwaway `HOME`, `XDG_CONFIG_HOME`, and `XDG_DATA_HOME`, and confirm the working tree stays clean. | +| `check-dependencies` | Before every release and after any dependency change -- audit for advisories, report funding, and update what is behind, refusing any version published less than 14 days ago unless it fixes a critical advisory. | | `pre-release-check` | Before cutting a release -- runs the `check-*` and `run-tests-and-linters` skills above (not `bump-version`) plus version-bump and commit-format checks. | +| `dev-tag` | To publish a snapshot to npm under the `dev` dist-tag without promising a release -- cuts a `v-dev.` tag. | --- diff --git a/CHANGES.md b/CHANGES.md index b43f25e..b145ee8 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -40,6 +40,8 @@ ## Development +## Version 1.0.0 + ### Added - `ailoud audio import` adds audio and video files, or whole directories, to a @@ -55,10 +57,10 @@ as a report. `--template` shapes it for the kind of conversation -- a 1:1, a performance review, an architecture discussion, a decision between solutions -- and `--context` supplies what the transcript does not say. Templates are - editable YAML files under the config directory. -- Four summarisation engines behind one setting: a local GGUF model through - llama.cpp, Claude by subscription through the Claude Code CLI, Claude by API, - and any OpenAI-compatible endpoint including Ollama and LM Studio. + editable YAML files under the config directory. Four engines sit behind one + setting: a local GGUF model through llama.cpp, Claude by subscription through + the Claude Code CLI, Claude by API, and any OpenAI-compatible endpoint, + including Ollama and LM Studio. - `ailoud report ls|show|rm` lists, prints and deletes saved reports. - `ailoud mcp` serves the library to an AI agent over MCP: sixteen tools, three prompts, and transcripts as addressable resources. Deleting takes two calls, diff --git a/CLAUDE.md b/CLAUDE.md index 305d1be..999534e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,10 +1,10 @@ # CLAUDE.md -- ailoud `ailoud` is a command-line tool that transcribes audio and video into a local -recording library, and, in a later milestone, will summarize and answer -questions over it through a large language model. Speech-to-text and the -LLM are separate engine layers behind stable ports. There is no GUI; the -CLI is the only front end, and the interface is English-only. +recording library, searches it, and summarizes and answers questions over it +through a large language model. Speech-to-text and the LLM are separate engine +layers behind stable ports. There is no GUI; the CLI is the only front end, and +the interface is English-only. **Must read before touching code:** diff --git a/README.md b/README.md index 1e9c411..5f73ed4 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@

AILoud

+ npm Documentation License Coverage diff --git a/apps/cli/package.json b/apps/cli/package.json index ef55947..dea0ef5 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -1,6 +1,6 @@ { "name": "ailoud", - "version": "0.0.0", + "version": "1.0.0", "type": "module", "bin": { "ailoud": "./dist/bin/ailoud.js" @@ -18,10 +18,12 @@ "zod": "4.4.3" }, "scripts": { + "prepack": "node -e \"require('node:fs').copyFileSync('../../README.md','README.md')\"", "build": "tsc -b tsconfig.build.json && chmod +x dist/bin/ailoud.js", "typecheck": "tsc -p tsconfig.json --noEmit" }, "description": "Transcribe audio and video into a local library, then search and summarise it", + "author": "Lorem Dev ", "license": "Apache-2.0", "homepage": "https://lorem-dev.github.io/ailoud/", "bugs": { diff --git a/apps/cli/src/commands/doctor.ts b/apps/cli/src/commands/doctor.ts index cc55640..694428f 100644 --- a/apps/cli/src/commands/doctor.ts +++ b/apps/cli/src/commands/doctor.ts @@ -1,6 +1,6 @@ import { access, constants, stat } from 'node:fs/promises'; import type { Command } from 'commander'; -import { EnvironmentError, installHint } from '@ailoud/core'; +import { EnvironmentError, installHint, isHostedLlm } from '@ailoud/core'; import type { Remedy } from '@ailoud/core'; import { run } from '@ailoud/providers'; import type { CliContext } from '../wiring.js'; @@ -334,8 +334,10 @@ export async function checkLanguageModel( const key = apiKeyFrom(env, variable); const settings = llm.provider === 'anthropic' ? llm.anthropic : llm.openaiCompatible; // A local OpenAI-compatible server needs no key, so its absence is only a - // problem when the endpoint is a hosted one. - const hosted = /api\.(openai|anthropic)\.com/.test(settings.baseUrl); + // problem when the endpoint is a hosted one. By hostname: the pattern this + // replaced matched `https://api.openai.com.example.net` and + // `https://example.net/?x=api.openai.com` alike. + const hosted = isHostedLlm(settings.baseUrl); if (key === undefined && hosted) { return { name, diff --git a/apps/cli/src/commands/mcp.ts b/apps/cli/src/commands/mcp.ts index f63efca..0c9effd 100644 --- a/apps/cli/src/commands/mcp.ts +++ b/apps/cli/src/commands/mcp.ts @@ -3,6 +3,7 @@ import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js' import type { CliContext } from '../wiring.js'; import { buildMcpServer } from '../mcp/server.js'; import { registerMcpInstall } from './mcpInstall.js'; +import { VERSION } from '../version.js'; export function registerMcp(program: Command, context: CliContext): void { const mcp = program @@ -25,7 +26,7 @@ export function registerMcp(program: Command, context: CliContext): void { // channel. A single stray line of human-facing text would corrupt the // JSON-RPC stream and the client would drop the connection. Every other // command in this codebase writes through context.ui; this one must not. - const { server, close } = buildMcpServer(context, program.version() ?? '0.0.0'); + const { server, close } = buildMcpServer(context, program.version() ?? VERSION); const transport = new StdioServerTransport(); const shutdown = async (): Promise => { diff --git a/apps/cli/src/program.test.ts b/apps/cli/src/program.test.ts index 9e68e16..d11bfa2 100644 --- a/apps/cli/src/program.test.ts +++ b/apps/cli/src/program.test.ts @@ -1,3 +1,4 @@ +import type { Command } from 'commander'; import { afterEach, describe, expect, it } from 'vitest'; import { parseConfig } from './config.js'; import { EnvironmentError, FailureError, UsageError } from '@ailoud/core'; @@ -14,6 +15,8 @@ import { SqliteStore } from '@ailoud/providers'; import { buildProgram, exitCodeFor } from './program.js'; import type { CliContext } from './wiring.js'; import { PlainUi } from './ui/plain.js'; +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; describe('exitCodeFor', () => { it('maps each domain error to its documented code', () => { @@ -154,15 +157,41 @@ describe('buildProgram', () => { expect(program.description()).toContain('audio-to-text'); }); + it('reports the version in its own manifest', () => { + // It used to report the literal 0.0.0, so the published 1.0.0-dev.1 + // answered `ailoud --version` with 0.0.0 and told MCP clients the same. + // Asserting against the manifest rather than a literal keeps this test + // from needing an edit at every release -- which is what would make it + // rot into agreeing with whatever is there. + const manifest: unknown = JSON.parse( + readFileSync(fileURLToPath(new URL('../package.json', import.meta.url)), 'utf8'), + ); + const version = (manifest as { version: string }).version; + expect(version).toMatch(/^\d+\.\d+\.\d+/); + expect(buildProgram(makeContext()).version()).toBe(version); + }); + + /** + * Commander writes its usage errors straight to stderr, which is right for + * a CLI and wrong for a test run: two of the tests below made every + * `pnpm test` print "error: unknown option", so a real error in the log had + * to be picked out of expected noise. Only stderr is silenced -- writeOut is + * how buildProgram routes help through context.write, which one of these + * tests asserts on. + */ + function quiet(program: Command): Command { + return program.configureOutput({ writeErr: () => {} }); + } + it('throws instead of exiting the process on an unknown flag', async () => { - const program = buildProgram(makeContext()); + const program = quiet(buildProgram(makeContext())); await expect(program.parseAsync(['node', 'ailoud', '--bogus'])).rejects.toMatchObject({ code: 'commander.unknownOption', }); }); it('maps an unknown flag to exit code 2 end to end', async () => { - const program = buildProgram(makeContext()); + const program = quiet(buildProgram(makeContext())); const error: unknown = await program .parseAsync(['node', 'ailoud', '--bogus']) .catch((caught: unknown) => caught); @@ -186,7 +215,7 @@ describe('buildProgram', () => { // raise 'commander.help' with its own exitCode of 1. That is a usage // error (nothing was told what to do), not a normal failure, so it must // map to 2 here, not fall through as 1. - const program = buildProgram(makeContext()); + const program = quiet(buildProgram(makeContext())); const error: unknown = await program .parseAsync(['node', 'ailoud']) .catch((caught: unknown) => caught); diff --git a/apps/cli/src/program.ts b/apps/cli/src/program.ts index 64f36d6..0f91abc 100644 --- a/apps/cli/src/program.ts +++ b/apps/cli/src/program.ts @@ -15,6 +15,7 @@ import { registerMcp } from './commands/mcp.js'; import { attachLetters, group, inGroupAndTopLevel } from './commands/groups.js'; import { registerTranscribe } from './commands/transcribe.js'; import type { CliContext } from './wiring.js'; +import { VERSION } from './version.js'; /** * Reads the commander error code off an unknown thrown value without @@ -57,7 +58,7 @@ export function buildProgram(context: CliContext): Command { program .name('ailoud') .description('Multilingual audio-to-text with a local recording library') - .version('0.0.0') + .version(VERSION) .exitOverride(); // throw instead of calling process.exit // Commander's own help/version text stays plain in both modes: routed // through context.write, not context.ui, so it is never decorated. diff --git a/apps/cli/src/setupLock.test.ts b/apps/cli/src/setupLock.test.ts index 78e5e25..60c6f00 100644 --- a/apps/cli/src/setupLock.test.ts +++ b/apps/cli/src/setupLock.test.ts @@ -1,5 +1,8 @@ import { describe, expect, it } from 'vitest'; -import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'; +import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'; +import { spawn } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { lockPath, withProvisioningLock } from './setupLock.js'; @@ -72,6 +75,41 @@ describe('withProvisioningLock', () => { await expect(withProvisioningLock(dir, async () => 'took over')).resolves.toBe('took over'); }); + it('leaves no scratch file behind after taking over a stale lock', async () => { + // The takeover writes its own lock beside the path and renames over it, + // because `rm` then create loses the race it looks like it wins. A rename + // that failed, or a path built wrong, would leave the scratch file in the + // data directory. + const dir = await dataDir(); + await writeFile( + lockPath(dir), + JSON.stringify({ pid: 4_194_304, startedAt: '2020-01-01T00:00:00.000Z' }), + 'utf8', + ); + let duringBody: string[] = []; + await withProvisioningLock(dir, async () => { + duringBody = await readdir(dir); + }); + expect(duringBody).toEqual(['provisioning.lock']); + expect(await readdir(dir)).toEqual([]); + }); + + it('records the taking-over process as the holder, not the stale one', async () => { + const dir = await dataDir(); + await writeFile( + lockPath(dir), + JSON.stringify({ pid: 4_194_304, startedAt: '2020-01-01T00:00:00.000Z' }), + 'utf8', + ); + let held: unknown; + await withProvisioningLock(dir, async () => { + held = JSON.parse(await readFile(lockPath(dir), 'utf8')); + }); + // The read-back after the rename is what makes a losing takeover + // detectable; this is the winning side of it. + expect(held).toMatchObject({ pid: process.pid }); + }); + it('treats an empty lock file as stale', async () => { const dir = await dataDir(); // A run that died between creating the lock and writing to it. @@ -97,3 +135,82 @@ describe('withProvisioningLock', () => { await rm(dir, { recursive: true, force: true }); }); }); + +/** + * Two runs at once, against the COMPILED lock, with a stale lock in place. + * + * Every other test here is single-threaded, which is why none of them caught + * that the first two takeover implementations let both runs proceed -- a + * concurrency test found 34 overlaps in 60 attempts. This is that test, small + * enough to keep in the suite. + */ +describe('withProvisioningLock under contention', () => { + const ITERATIONS = 12; + const HOLD_MS = 120; + + it('lets exactly one of two simultaneous runs into the body', async () => { + const dist = fileURLToPath(new URL('../dist/setupLock.js', import.meta.url)); + expect(existsSync(dist)).toBe(true); // the gate builds before it tests + + const scratch = await mkdtemp(join(tmpdir(), 'ailoud-lock-race-')); + const worker = join(scratch, 'worker.mjs'); + await writeFile( + worker, + `import { appendFileSync } from "node:fs"; + import { withProvisioningLock } from ${JSON.stringify(dist)}; + const [dir, log] = process.argv.slice(2); + try { + await withProvisioningLock(dir, async () => { + appendFileSync(log, \`ENTER \${process.pid}\n\`); + await new Promise((r) => setTimeout(r, ${HOLD_MS})); + appendFileSync(log, \`LEAVE \${process.pid}\n\`); + }); + } catch { + appendFileSync(log, \`REFUSED \${process.pid}\n\`); + }`, + 'utf8', + ); + + let refusals = 0; + for (let round = 0; round < ITERATIONS; round += 1) { + const dir = join(scratch, `round-${round}`); + await mkdir(dir, { recursive: true }); + // A stale lock: pid 2^22 is above every default pid_max, so nothing + // holds it. This is the state a Ctrl-C or a crash leaves behind, and the + // only state in which a takeover happens at all. + await writeFile( + lockPath(dir), + JSON.stringify({ pid: 4_194_304, startedAt: '2020-01-01T00:00:00.000Z' }), + 'utf8', + ); + const log = join(dir, 'log.txt'); + await writeFile(log, '', 'utf8'); + + await Promise.all( + [0, 1].map( + () => + new Promise((resolve) => { + const child = spawn(process.execPath, [worker, dir, log], { stdio: 'ignore' }); + child.on('exit', () => resolve()); + }), + ), + ); + + const lines = (await readFile(log, 'utf8')).split('\n').filter(Boolean); + refusals += lines.filter((l) => l.startsWith('REFUSED')).length; + // Nesting is the failure: ENTER, ENTER means both are inside the body. + let inside = 0; + for (const line of lines) { + if (line.startsWith('ENTER')) inside += 1; + if (line.startsWith('LEAVE')) inside -= 1; + expect(inside, `round ${round}: ${lines.join(' | ')}`).toBeLessThanOrEqual(1); + } + expect(lines.filter((l) => l.startsWith('ENTER'))).toHaveLength(1); + } + + // Without this the test could pass by never contending at all. + expect(refusals).toBeGreaterThan(0); + + await rm(scratch, { recursive: true, force: true }); + }, 60_000); +}); diff --git a/apps/cli/src/setupLock.ts b/apps/cli/src/setupLock.ts index 014d7e8..de40670 100644 --- a/apps/cli/src/setupLock.ts +++ b/apps/cli/src/setupLock.ts @@ -65,6 +65,12 @@ async function readHolder(path: string): Promise { * would leave a window for the other process to win in between, which is * exactly the race being closed. * + * Taking over a stale lock cannot use `wx`, since the file is there, so the + * takeover runs under a second lock (`provisioning.lock.steal`) created with + * `wx`. That makes the takeover itself exclusive -- one winner, and the loser + * told to try again. Checking afterwards who won is not enough: two runs can + * each check after their own write and each see themselves. + * * A live lock is refused immediately rather than waited on. Provisioning is * interactive and can sit on a consent prompt for minutes, so a queued * second run would look like a hang. The refusal names the holder's pid and @@ -78,36 +84,99 @@ export async function withProvisioningLock(dataDir: string, body: () => Promi const path = lockPath(dataDir); await mkdir(dirname(path), { recursive: true }); - let handle; + const holder: LockHolder = { pid: process.pid, startedAt: new Date().toISOString() }; + const mine = JSON.stringify(holder); + try { - handle = await open(path, 'wx'); + const handle = await open(path, 'wx'); + try { + await handle.writeFile(mine, 'utf8'); + } finally { + await handle.close(); + } } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; - const holder = await readHolder(path); - if (holder !== null && isRunning(holder.pid)) { + const existing = await readHolder(path); + if (existing !== null && isRunning(existing.pid)) { throw new FailureError( - `another ailoud provisioning run is already in progress (pid ${holder.pid}, started ` + - `${holder.startedAt}). Wait for it to finish, or stop it, then try again.`, + `another ailoud provisioning run is already in progress (pid ${existing.pid}, started ` + + `${existing.startedAt}). Wait for it to finish, or stop it, then try again.`, ); } + // Stale: the holder is gone, or never finished writing who it was. - await rm(path, { force: true }); - handle = await open(path, 'wx'); - } + // + // Two earlier attempts at this were wrong in the same way -- they made the + // takeover look exclusive without making it exclusive. `rm` then create + // let a second run become a live holder in between and then deleted its + // lock. Rename-then-read-back looked safer but is not: the interleaving + // A.rename, A.read, B.rename, B.read leaves each run reading its own pid, + // and both proceed. A concurrency test found 34 overlaps in 60 runs. + // + // Exclusion has to be on the takeover itself, so it runs through a second + // lock created with `wx` -- one atomic syscall, one winner. The loser is + // told to try again rather than being allowed to guess. + const steal = `${path}.steal`; + let stealHandle; + try { + stealHandle = await open(steal, 'wx'); + } catch (stealError) { + if ((stealError as NodeJS.ErrnoException).code !== 'EEXIST') throw stealError; + throw new FailureError( + 'another ailoud provisioning run is taking over a stale lock right now. Try again.', + ); + } - try { - const holder: LockHolder = { pid: process.pid, startedAt: new Date().toISOString() }; - await handle.writeFile(JSON.stringify(holder), 'utf8'); - } finally { - await handle.close(); + try { + // Re-read under the steal lock: between the check above and here, the + // stale lock may have been taken by a run that is now alive. + const current = await readHolder(path); + if (current !== null && isRunning(current.pid)) { + throw new FailureError( + `another ailoud provisioning run is already in progress (pid ${current.pid}, started ` + + `${current.startedAt}). Wait for it to finish, or stop it, then try again.`, + ); + } + await rm(path, { force: true }); + // Still `wx`: a run on the fast path can create the lock in the instant + // after that `rm`, and it is then the holder. Losing to it is the + // correct outcome, not something to overwrite. + // + // CodeQL reads this as a check-then-use race (js/file-system-race) and + // cannot see that the exclusion is held by `${path}.steal` above. The + // rule is excluded in .github/codeql/codeql-config.yml, where the + // reasoning lives; what actually guards this is the two-process test in + // setupLock.test.ts. + try { + const handle = await open(path, 'wx'); + try { + await handle.writeFile(mine, 'utf8'); + } finally { + await handle.close(); + } + } catch (createError) { + if ((createError as NodeJS.ErrnoException).code !== 'EEXIST') throw createError; + throw new FailureError( + 'another ailoud provisioning run took the lock at the same moment. Try again.', + ); + } + } finally { + await stealHandle.close(); + await rm(steal, { force: true }); + } } try { return await body(); } finally { - // force: a lock already gone is the outcome we wanted anyway, and - // failing to clean up must never mask what the body was doing. - await rm(path, { force: true }); + // Only our own lock. Removing it unconditionally would delete the lock of + // a run that legitimately took over after ours was declared stale, letting + // a third run in while that one is still working. + // + // force: a lock already gone is the outcome we wanted anyway, and failing + // to clean up must never mask what the body was doing. + const held = await readHolder(path); + if (held === null || held.pid === process.pid) await rm(path, { force: true }); } } diff --git a/apps/cli/src/version.ts b/apps/cli/src/version.ts new file mode 100644 index 0000000..e199fbc --- /dev/null +++ b/apps/cli/src/version.ts @@ -0,0 +1,28 @@ +import { readFileSync } from 'node:fs'; + +/** + * The CLI's own version, read from its manifest. + * + * It used to be the literal `0.0.0` passed to commander, so the published + * 1.0.0-dev.1 answered `ailoud --version` with 0.0.0 and told every MCP client + * the same. The manifest is the one place a release already updates, and it + * ships inside the tarball, so it is the only copy that cannot go stale. + * + * Resolved relative to this module rather than to the working directory: + * `dist/version.js` sits one level below the package root both in the + * repository and in an installed package. + */ +export const VERSION: string = readVersion(); + +function readVersion(): string { + const manifest = new URL('../package.json', import.meta.url); + const parsed: unknown = JSON.parse(readFileSync(manifest, 'utf8')); + const version = + typeof parsed === 'object' && parsed !== null + ? (parsed as { version?: unknown }).version + : undefined; + if (typeof version !== 'string' || version === '') { + throw new Error(`no version in ${manifest.pathname}`); + } + return version; +} diff --git a/apps/cli/src/wiring.ts b/apps/cli/src/wiring.ts index ac35d5a..aa21b0f 100644 --- a/apps/cli/src/wiring.ts +++ b/apps/cli/src/wiring.ts @@ -11,7 +11,7 @@ import type { TranscriptionProvider, } from '@ailoud/core'; import { existsSync, statSync } from 'node:fs'; -import { EnvironmentError } from '@ailoud/core'; +import { EnvironmentError, isHostedLlm } from '@ailoud/core'; import { AnthropicSummarizer, ClaudeCliSummarizer, @@ -165,7 +165,7 @@ export async function createContext( if (llm.provider === 'openai-compatible') { const settings = llm.openaiCompatible; const apiKey = apiKeyFrom(env, 'OPENAI_API_KEY'); - if (settings.baseUrl.startsWith('https://api.openai.com') && apiKey === undefined) { + if (isHostedLlm(settings.baseUrl) && apiKey === undefined) { throw new EnvironmentError( 'No API key for the language model. Set AILOUD_LLM_API_KEY (or OPENAI_API_KEY) in ' + 'your environment. It is read from the environment on purpose and never from ' + diff --git a/docs/development/releasing.md b/docs/development/releasing.md index 3cd0233..e271d94 100644 --- a/docs/development/releasing.md +++ b/docs/development/releasing.md @@ -10,10 +10,16 @@ ## Tags -| Tag | Means | -| ------------------- | ---------------------------------------- | -| `v-rc.` | a release candidate, tagged on `develop` | -| `v` | a final release, tagged on `main` only | +| Tag | Means | npm dist-tag | +| -------------------- | ---------------------------------------- | ------------ | +| `v-dev.` | a snapshot, tagged on any branch | `dev` | +| `v-rc.` | a release candidate, tagged on `develop` | `next` | +| `v` | a final release, tagged on `main` only | `latest` | + +Only a final tag moves `latest`, so `npm install ailoud` never returns a +pre-release -- except for a package's very first publish, where npm sets +`latest` whatever `--tag` says. `latest` can be moved but not removed, so that +first pre-release answers `npm install ailoud` until a final version exists. ## Changelog limits @@ -28,26 +34,39 @@ bug, there is nothing to say. See ## Steps 1. Run the `pre-release-check` skill. It runs the whole gate plus the - documentation, changelog and version checks. + documentation, dependency, changelog and version checks. + 2. Run the `bump-version` skill. It sets the version across every - `package.json`, promotes the CHANGES.md Development section, and makes the - release commit. It does not tag or push. -3. Extract the release body: + `package.json` and promotes the CHANGES.md Development section. It does not + tag or push. + +3. Fold the pre-release sections in. `bump-version` comes first: ``` - node scripts/release-notes.mjs v1.2.3 + node scripts/fold-prereleases.mjs 1.2.3 + node scripts/check-changelog.mjs v1.2.3 ``` - It reads the `## Version 1.2.3` section and writes `RELEASE_NOTES.md`. It - exits non-zero if that section is missing, empty, or over the hard limit. + Folding first would leave `bump-version` an empty Development section to + promote, giving a second `## Version 1.2.3` heading that fails the check. + + To read the notes as they will appear, `node scripts/release-notes.mjs +v1.2.3` writes them to `RELEASE_NOTES.md`. The release itself does not need + this -- `publish.yml` runs the same script. -4. Merge to `main` and tag: +4. Commit, tag and push: ``` + git commit -am "chore: 1.2.3" git tag -s v1.2.3 -m "v1.2.3" - git push origin main --tags + git push origin main + git push origin v1.2.3 ``` + The tag is what starts everything else: `publish.yml` publishes the three + packages, creates the GitHub release from CHANGES.md, and retires the + superseded snapshots; `docs.yml` then publishes the site. + ## Publishing to npm Pushing a final tag also runs @@ -59,6 +78,19 @@ mints a short-lived OIDC token for the run, npm exchanges it for a credential good for minutes, and provenance is attached automatically. Nothing long-lived is stored, so there is no 90-day expiry to renew. +Except once, per package. A trusted publisher is attached to a package on +npmjs.com, and there is no page to attach it to until the package exists, so +the first version of each has to go out on a token in the `NPM_TOKEN` secret -- +npm answers `ENEEDAUTH` without one however complete the OIDC setup is. The +workflow uses the secret when it is present and OIDC when it is not, so +deleting the secret is the whole of the switch. + +It will not let that drift: a **pre-release** published on the token logs a +warning, and a **final release** with the secret still set fails before +publishing anything. Attaching the publisher (organization `lorem-dev`, +repository `ailoud`, workflow `publish.yml`, environment empty) on all three +package pages and deleting the secret clears it. + One-time setup on npmjs.com, per package -- Package, then Settings, then Trusted publisher, then GitHub Actions: @@ -74,24 +106,58 @@ half of what is needed: `pnpm pack` rewrites the `workspace:*` dependencies into real versions, which npm requires and will not do itself, and `npm publish` is the one with OIDC and provenance. -Before publishing it checks that all three manifests agree with the tag, then -runs the whole gate. A pre-release tag publishes under the `next` dist-tag, so -`npm install ailoud` keeps returning the last stable version. +Before publishing, and before anything is built, it refuses a release that +should not happen: a changelog unfit to release, an open high or critical code +scanning alert, a manifest that disagrees with the tag, or the `NPM_TOKEN` +secret still set on a final tag. Then it runs the whole gate. + +A code scanning finding blocks only while it is open. One that has been +reviewed is dismissed with its reason and does not block. + +## Retiring pre-releases + +After a final release, retire the snapshots it supersedes: + +``` +node scripts/retire-prereleases.mjs 1.0.0 # prints the plan +node scripts/retire-prereleases.mjs 1.0.0 --yes # carries it out +``` + +Deprecating, not unpublishing: a deprecated version keeps every pinned install +working and prints a notice on the next one. It also drops the `dev` dist-tag, +and deletes the tags -- but only those whose commit is reachable from `main`, +because the published provenance attests that commit. The rest are reported and +left in place. + +This is a manual step, run under `npm login`. Automating it was tried and does +not work: trusted publishing authenticates `npm publish` and nothing else. The +OIDC exchange succeeds, but the token it returns is refused by `npm deprecate` +-- `E404 ... or you do not have permission`, then `E401 ... token is invalid` +on every call after. Measured on the 1.0.0 release. + +If the npm side does not complete, the script leaves the tags alone and exits +non-zero. The tags are what name which pre-releases to retire, so deleting them +after a failed deprecation would destroy the only record of what was missed. ## What a tag triggers -Pushing a final tag runs -[`.github/workflows/docs.yml`](https://github.com/lorem-dev/ailoud/blob/main/.github/workflows/docs.yml), -which publishes the documentation for that version to the `gh-pages` branch -with [mike](https://github.com/jimporter/mike) and moves the `latest` alias -that the site root redirects to. +Pushing a tag runs `publish.yml`. When it succeeds, +[`.github/workflows/docs.yml`](https://github.com/lorem-dev/ailoud/blob/main/.github/workflows/docs.yml) +runs on its completion and publishes the documentation for that version to the +`gh-pages` branch with [mike](https://github.com/jimporter/mike), moving the +`latest` alias that the site root redirects to. `publish.yml` also creates the +GitHub release, with the body taken from the `## Version ` section of +CHANGES.md by `scripts/release-notes.mjs`. + +The order matters: the two used to start together on the tag push, so a publish +that then refused left the site advertising a version npm did not have. Nothing else publishes documentation. A push to a branch publishes nothing, so what is online always describes a version someone can install. -A pre-release tag (`v1.2.3-rc.1`, or any tag with a `-` qualifier) publishes -nothing either. The workflow refuses it twice: the tag filter never starts it, -and the job checks again in case `workflow_dispatch` was pointed at one. +A pre-release publishes nothing either. It reaches docs.yml -- `publish.yml` +runs for pre-releases too -- and the job stops once it reads the version from +the published commit's manifest and finds a `-` in it. ## One-time repository setup diff --git a/docs/mcp.md b/docs/mcp.md index c8c8a31..533c7fd 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -76,22 +76,6 @@ Your own files are safe: Comments in a JSON or `.jsonc` MCP config do not survive an edit -- the file is parsed and re-serialised. TOML and YAML keep theirs. -Your own config files are safe: - -- Only the text between the two markers is ever rewritten. A file that merely - _mentions_ `` in prose keeps everything around it. -- A `config.toml` that already defines an `ailoud` server some other way is - refused, not edited. Two definitions of one key is a TOML error, and it - would break your whole Codex config rather than just this server. -- A Hermes `config.yaml` with your own settings or comments is rewritten, never - deleted. Only a file holding nothing but AILoud's own keys is removed. -- A trailing comma or a byte-order mark in a `.jsonc` is tolerated. A file that - is not JSON at all is refused with a message, not rewritten. - -!!! note -Comments in a JSON or `.jsonc` MCP config do not survive an edit -- the -file is parsed and re-serialised. TOML and YAML keep theirs. - ### Supported agents | Agent | Scopes | Config | Rules file | diff --git a/docs/usage/cli.md b/docs/usage/cli.md index 7a343e9..dd4c587 100644 --- a/docs/usage/cli.md +++ b/docs/usage/cli.md @@ -44,7 +44,7 @@ ailoud report l The old top-level spellings still work: `ailoud ls`, `ailoud show`, `ailoud rm`, `ailoud annotate`, `ailoud import`, `ailoud transcribe`, -`ailoud summarize`. +`ailoud summarize`, `ailoud search`. ## audio import diff --git a/eslint.config.mjs b/eslint.config.mjs index f263d02..5a63ff7 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -12,19 +12,44 @@ const LAYER_MAY_IMPORT = { export default tseslint.config( { - // scripts/** holds plain Node utility scripts (release tooling, not - // part of the typed packages/apps source tree); node --check is their - // syntax gate instead. ignores: [ '**/dist/**', '**/coverage/**', '**/node_modules/**', // mkdocs build output: third-party minified JS, not ours to lint. 'site/**', - '**/*.config.{js,mjs,cjs,ts}', - 'scripts/**', ], }, + { + // The build's configuration and the release scripts. Both were ignored, + // which meant every editor reported "File ignored because of a matching + // ignore pattern" on opening one, and a mistake in them was caught by + // nothing -- `node --check` was the only gate on scripts/, and it sees + // syntax, not an unused variable or a misspelled identifier. They are + // Node, not part of the typed source tree, so they get the recommended + // rules and Node globals rather than the type-aware config. + files: ['*.config.{js,mjs,cjs,ts}', '**/*.config.{js,mjs,cjs,ts}', 'scripts/**/*.mjs'], + languageOptions: { + // The Node globals these files actually use. Spelled out rather than + // pulled from a globals package: it is a short list, and a new name + // appearing here should be a deliberate addition. + globals: { + console: 'readonly', + process: 'readonly', + module: 'writable', + require: 'readonly', + __dirname: 'readonly', + Buffer: 'readonly', + URL: 'readonly', + fetch: 'readonly', + }, + }, + rules: { + // A .cjs file uses require/module, which the type-aware rules would + // otherwise flag in a "type": "module" workspace. + '@typescript-eslint/no-require-imports': 'off', + }, + }, js.configs.recommended, ...tseslint.configs.recommended, { @@ -40,13 +65,28 @@ export default tseslint.config( files: ['packages/*/src/**/*.ts', 'apps/*/src/**/*.ts'], plugins: { boundaries }, settings: { + // Both src and dist. A cross-package import can be written two ways: + // `../../providers/src/index.js` resolves to a source file, but + // `@ailoud/providers` resolves through node_modules to the built + // `dist/index.js` -- which matched no element, so the rule classified it + // as unknown and said nothing at all. Listing dist under the same type + // makes both forms the same violation. It does mean the second form is + // only caught once dist exists, which is why the gate and CI both build + // before they lint. 'boundaries/elements': [ - { type: 'core', pattern: 'packages/core/src/**' }, - { type: 'providers', pattern: 'packages/providers/src/**' }, - { type: 'cli', pattern: 'apps/cli/src/**' }, + { type: 'core', pattern: ['packages/core/src/**', 'packages/core/dist/**'] }, + { + type: 'providers', + pattern: ['packages/providers/src/**', 'packages/providers/dist/**'], + }, + { type: 'cli', pattern: ['apps/cli/src/**', 'apps/cli/dist/**'] }, ], + // The root tsconfig, which includes every package's sources, rather + // than a glob over the per-package ones: the resolver warns about + // multiple projects, and one project covering the whole workspace is + // both quieter and faster. 'import/resolver': { - typescript: { project: ['packages/*/tsconfig.json', 'apps/*/tsconfig.json'] }, + typescript: { project: 'tsconfig.json' }, }, }, rules: { @@ -54,10 +94,12 @@ export default tseslint.config( 'error', { default: 'disallow', - policies: Object.entries(LAYER_MAY_IMPORT).map(([from, to]) => ({ - from: { element: { type: from } }, - allow: { to: { element: { types: { anyOf: to } } } }, - })), + policies: [ + ...Object.entries(LAYER_MAY_IMPORT).map(([from, to]) => ({ + from: { element: { type: from } }, + allow: { to: { element: { types: { anyOf: to } } } }, + })), + ], }, ], }, diff --git a/fixtures/en-short.mkv b/fixtures/en-short.mkv new file mode 100644 index 0000000..0e38a2d --- /dev/null +++ b/fixtures/en-short.mkv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f05f1d481205693d249df834dfb8fd7ef706db41932d07afca49a7cc3bdf2e3 +size 9879 diff --git a/fixtures/en-short.mov b/fixtures/en-short.mov new file mode 100644 index 0000000..24e5ea0 --- /dev/null +++ b/fixtures/en-short.mov @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc8f99f5540c772a26017328a69c05f6a8c4d07e6f105fb00b03a3cd10176214 +size 13266 diff --git a/fixtures/en-short.mp4 b/fixtures/en-short.mp4 new file mode 100644 index 0000000..2009cac --- /dev/null +++ b/fixtures/en-short.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c984e208272ef1ae391e07d0a57fd5fa505ca40f4a0932429a366b968b414de6 +size 13215 diff --git a/fixtures/en-short.webm b/fixtures/en-short.webm new file mode 100644 index 0000000..57a0da4 --- /dev/null +++ b/fixtures/en-short.webm @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c77373b0cd1330b4e64d1c120828efd3306a3856382c2737e175a73091d791f2 +size 9117 diff --git a/jest.config.cjs b/jest.config.cjs index 7fb320e..c0af394 100644 --- a/jest.config.cjs +++ b/jest.config.cjs @@ -39,6 +39,14 @@ const shared = { module.exports = { maxWorkers: 2, detectOpenHandles: true, + // Also at the root, not only inside each project. Jest takes the per-test + // timeout from the GLOBAL config, and with `projects` a value set only on a + // project reaches `configs[].testTimeout` while `globalConfig.testTimeout` + // stays undefined -- so every test silently fell back to Jest's 5 s default. + // Locally that was masked: the transcribe specs were failing fast for a + // missing model, so nothing ran long enough to hit it. On CI, where the + // model is there, whisper takes tens of seconds and every one timed out. + testTimeout: 600_000, projects: [ { ...shared, diff --git a/package.json b/package.json index 78a53db..bdfe21e 100644 --- a/package.json +++ b/package.json @@ -1,9 +1,10 @@ { "name": "ailoud-workspace", - "version": "0.0.0", + "version": "1.0.0", "private": true, "type": "module", "description": "Multilingual audio-to-text CLI with a recording library and LLM summaries", + "author": "Lorem Dev ", "license": "Apache-2.0", "packageManager": "pnpm@11.9.0", "engines": { @@ -18,9 +19,9 @@ "lint": "eslint .", "lint:fix": "eslint . --fix", "typecheck": "pnpm -r typecheck", - "test": "vitest run", + "test": "NODE_OPTIONS=--disable-warning=ExperimentalWarning vitest run", "test:watch": "vitest", - "test:cov": "vitest run --coverage", + "test:cov": "NODE_OPTIONS=--disable-warning=ExperimentalWarning vitest run --coverage", "test:e2e": "pnpm build && jest --config jest.config.cjs", "test:e2e:no-tools": "pnpm build && jest --config jest.config.cjs --selectProjects no-tools" }, diff --git a/packages/core/README.md b/packages/core/README.md new file mode 100644 index 0000000..cec11c7 --- /dev/null +++ b/packages/core/README.md @@ -0,0 +1,16 @@ +# @ailoud/core + +The domain layer of [AILoud](https://github.com/lorem-dev/ailoud): the model, +the ports, and the pure logic that does no I/O. + +Every effect reaches this package as a port -- `Fs`, `Clock`, `AudioTool`, +`TranscriptionProvider`, `RecordingStore`, `Summarizer` -- implemented in +[`@ailoud/providers`](https://www.npmjs.com/package/@ailoud/providers) and wired +together by the [`ailoud`](https://www.npmjs.com/package/ailoud) CLI. A lint +rule fails the build if anything here imports `node:fs`, `node:child_process` +or a network module. + +Published because the CLI depends on it. The interfaces are not stable yet, and +there is no reason to depend on this package directly. + +Documentation: diff --git a/packages/core/package.json b/packages/core/package.json index bd9f524..9fa81d1 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,6 +1,6 @@ { "name": "@ailoud/core", - "version": "0.0.0", + "version": "1.0.0", "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts", @@ -19,6 +19,7 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "description": "AILoud domain model, ports and pure logic: the layer that does no I/O", + "author": "Lorem Dev ", "license": "Apache-2.0", "homepage": "https://lorem-dev.github.io/ailoud/", "bugs": { diff --git a/packages/core/src/domain/llmHost.test.ts b/packages/core/src/domain/llmHost.test.ts new file mode 100644 index 0000000..61f8050 --- /dev/null +++ b/packages/core/src/domain/llmHost.test.ts @@ -0,0 +1,54 @@ +import { describe, expect, it } from 'vitest'; +import { isHostedLlm, withoutTrailingSlashes } from './llmHost.js'; + +describe('isHostedLlm', () => { + it('recognises the two hosted APIs', () => { + expect(isHostedLlm('https://api.openai.com/v1')).toBe(true); + expect(isHostedLlm('https://api.anthropic.com')).toBe(true); + }); + + it('is not fooled by a host that merely starts with one', () => { + // The defect this replaced: `startsWith('https://api.openai.com')` and + // `/api\.(openai|anthropic)\.com/` both said yes to these, and the part + // of a hostname that decides where a request goes is the end of it. + expect(isHostedLlm('https://api.openai.com.example.net/v1')).toBe(false); + expect(isHostedLlm('https://api.anthropic.com.example.net')).toBe(false); + }); + + it('is not fooled by the name appearing elsewhere in the URL', () => { + expect(isHostedLlm('https://example.net/?upstream=api.openai.com')).toBe(false); + expect(isHostedLlm('https://example.net/api.openai.com')).toBe(false); + }); + + it('treats a local server as not hosted', () => { + expect(isHostedLlm('http://localhost:11434/v1')).toBe(false); + expect(isHostedLlm('http://127.0.0.1:8080')).toBe(false); + }); + + it('ignores case in the hostname, as DNS does', () => { + expect(isHostedLlm('https://API.OpenAI.com/v1')).toBe(true); + }); + + it('says no to something that is not a URL', () => { + // Reported as a configuration error elsewhere. Calling it hosted here + // would demand an API key for a value that cannot address anything. + expect(isHostedLlm('api.openai.com')).toBe(false); + expect(isHostedLlm('')).toBe(false); + }); +}); + +describe('withoutTrailingSlashes', () => { + it('strips one slash and many', () => { + expect(withoutTrailingSlashes('https://x.test/')).toBe('https://x.test'); + expect(withoutTrailingSlashes('https://x.test/v1///')).toBe('https://x.test/v1'); + }); + + it('leaves a value with none alone', () => { + expect(withoutTrailingSlashes('https://x.test/v1')).toBe('https://x.test/v1'); + }); + + it('handles a value that is only slashes without backtracking', () => { + // The reason this is a loop and not `replace(/\/+$/, '')`. + expect(withoutTrailingSlashes('/'.repeat(50_000))).toBe(''); + }); +}); diff --git a/packages/core/src/domain/llmHost.ts b/packages/core/src/domain/llmHost.ts new file mode 100644 index 0000000..358750e --- /dev/null +++ b/packages/core/src/domain/llmHost.ts @@ -0,0 +1,37 @@ +/** + * Whether a language-model endpoint is one of the hosted APIs. + * + * By hostname, compared exactly. Two places used to ask this with a substring + * -- `baseUrl.startsWith('https://api.openai.com')` and + * `/api\.(openai|anthropic)\.com/.test(baseUrl)` -- and both answered yes for + * `https://api.openai.com.example.net/v1`, where the interesting part of the + * name is what follows. A local server, which is the case these checks exist + * to spare, is any other host. + */ +const HOSTED_HOSTS: readonly string[] = ['api.openai.com', 'api.anthropic.com']; + +export function isHostedLlm(baseUrl: string): boolean { + let parsed; + try { + parsed = new URL(baseUrl); + } catch { + // Not a URL at all. Reported elsewhere as a configuration error; treating + // it as hosted here would demand an API key for a value that cannot + // address anything. + return false; + } + return HOSTED_HOSTS.includes(parsed.hostname.toLowerCase()); +} + +/** + * `baseUrl` without trailing slashes, so a path can be appended to it. + * + * Written as a loop rather than `replace(/\/+$/, '')`: on a value ending in + * many slashes that pattern backtracks, which is a denial of service when the + * value comes from anywhere but us. There is no regex worth that here. + */ +export function withoutTrailingSlashes(baseUrl: string): string { + let end = baseUrl.length; + while (end > 0 && baseUrl[end - 1] === '/') end -= 1; + return baseUrl.slice(0, end); +} diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 2eb2e66..6631bc6 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -62,6 +62,7 @@ export { export { AiloudError, FailureError, UsageError, EnvironmentError } from './domain/errors.js'; export { encodeUlid } from './domain/ulid.js'; +export { isHostedLlm, withoutTrailingSlashes } from './domain/llmHost.js'; export { mimeForPath } from './domain/mime.js'; diff --git a/packages/providers/README.md b/packages/providers/README.md new file mode 100644 index 0000000..2cf98c5 --- /dev/null +++ b/packages/providers/README.md @@ -0,0 +1,20 @@ +# @ailoud/providers + +The adapters of [AILoud](https://github.com/lorem-dev/ailoud): each port +declared in [`@ailoud/core`](https://www.npmjs.com/package/@ailoud/core), +implemented against something real. + +| Port | Implemented with | +| ----------------------- | ----------------------------------------------------------- | +| `AudioTool` | ffmpeg and ffprobe | +| `TranscriptionProvider` | whisper.cpp | +| `RecordingStore` | SQLite (`node:sqlite`) with FTS5 search | +| `Summarizer` | llama.cpp, the Claude CLI, or the Anthropic and OpenAI APIs | + +These talk to binaries and files on the machine, so they are covered by the +end-to-end suite rather than by mocks. + +Published because the CLI depends on it. The interfaces are not stable yet, and +there is no reason to depend on this package directly. + +Documentation: diff --git a/packages/providers/package.json b/packages/providers/package.json index 246edc5..b94dbff 100644 --- a/packages/providers/package.json +++ b/packages/providers/package.json @@ -1,6 +1,6 @@ { "name": "@ailoud/providers", - "version": "0.0.0", + "version": "1.0.0", "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts", @@ -18,6 +18,7 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "description": "AILoud port implementations: ffmpeg, sqlite, whisper.cpp, and the LLM adapters", + "author": "Lorem Dev ", "license": "Apache-2.0", "homepage": "https://lorem-dev.github.io/ailoud/", "bugs": { diff --git a/packages/providers/src/audio/ffmpeg.test.ts b/packages/providers/src/audio/ffmpeg.test.ts index 0b75a77..65e503c 100644 --- a/packages/providers/src/audio/ffmpeg.test.ts +++ b/packages/providers/src/audio/ffmpeg.test.ts @@ -1,6 +1,7 @@ import { mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; import { afterAll, beforeAll, describe, expect, it } from 'vitest'; import { run } from '../process/run.js'; import { FfmpegAudioTool } from './ffmpeg.js'; @@ -28,6 +29,22 @@ afterAll(async () => { await rm(dir, { recursive: true, force: true }); }); +/** The first audio stream's sample rate and channel count, as ffprobe sees it. */ +async function audioStreamOf(path: string): Promise<{ sample_rate: string; channels: number }> { + const probe = await run('ffprobe', [ + '-v', + 'error', + '-select_streams', + 'a:0', + '-show_entries', + 'stream=sample_rate,channels', + '-of', + 'json', + path, + ]); + return JSON.parse(probe.stdout).streams[0]; +} + describe('FfmpegAudioTool', () => { it('probes the duration', async () => { const { durationMs } = await new FfmpegAudioTool().probe(source); @@ -38,18 +55,7 @@ describe('FfmpegAudioTool', () => { it('converts to 16 kHz mono wav', async () => { const output = join(dir, 'out.wav'); await new FfmpegAudioTool().toWav16kMono(source, output); - const probe = await run('ffprobe', [ - '-v', - 'error', - '-select_streams', - 'a:0', - '-show_entries', - 'stream=sample_rate,channels', - '-of', - 'json', - output, - ]); - const stream = JSON.parse(probe.stdout).streams[0]; + const stream = await audioStreamOf(output); expect(stream.sample_rate).toBe('16000'); expect(stream.channels).toBe(1); }); @@ -77,3 +83,29 @@ describe('FfmpegAudioTool', () => { await expect(new FfmpegAudioTool().slice(bad, join(dir, 'o.wav'), 0, 1000)).rejects.toThrow(); }); }); + +/** + * The video containers domain/mime.ts knows. `ailoud audio import` accepts + * video because a meeting recording usually is one, and only the audio track + * matters -- so each fixture is the same clip of speech wrapped in a different + * container by scripts/make-fixtures.mjs, and the assertion is that what comes + * out is the 16 kHz mono WAV whisper.cpp needs, whatever went in. + */ +describe.each(['mp4', 'mov', 'mkv', 'webm'])('a %s recording', (container) => { + const fixture = fileURLToPath( + new URL(`../../../../fixtures/en-short.${container}`, import.meta.url), + ); + + it('probes like audio and converts to 16 kHz mono wav', async () => { + const tool = new FfmpegAudioTool(); + const { durationMs } = await tool.probe(fixture); + expect(durationMs).toBeGreaterThan(2000); + expect(durationMs).toBeLessThan(3000); + + const output = join(dir, `${container}.wav`); + await tool.toWav16kMono(fixture, output); + const stream = await audioStreamOf(output); + expect(stream.sample_rate).toBe('16000'); + expect(stream.channels).toBe(1); + }); +}); diff --git a/packages/providers/src/llm/anthropic.ts b/packages/providers/src/llm/anthropic.ts index c097e19..8474029 100644 --- a/packages/providers/src/llm/anthropic.ts +++ b/packages/providers/src/llm/anthropic.ts @@ -1,5 +1,6 @@ import type { Summarizer } from '@ailoud/core'; import { EnvironmentError, FailureError } from '@ailoud/core'; +import { withoutTrailingSlashes } from '@ailoud/core'; /** As for the OpenAI adapter: a hosted call that has not returned in five minutes is not going to. */ const REQUEST_TIMEOUT_MS = 5 * 60_000; @@ -71,7 +72,7 @@ export class AnthropicSummarizer implements Summarizer { } public async complete(prompt: string): Promise { - const url = `${this.options.baseUrl.replace(/\/+$/, '')}/messages`; + const url = `${withoutTrailingSlashes(this.options.baseUrl)}/messages`; const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); diff --git a/packages/providers/src/llm/models.ts b/packages/providers/src/llm/models.ts index 9971b12..c96f35d 100644 --- a/packages/providers/src/llm/models.ts +++ b/packages/providers/src/llm/models.ts @@ -1,4 +1,5 @@ import { EnvironmentError, FailureError } from '@ailoud/core'; +import { withoutTrailingSlashes } from '@ailoud/core'; /** As elsewhere in this directory: a hosted call that has not answered in a minute is not going to. */ const REQUEST_TIMEOUT_MS = 60_000; @@ -77,7 +78,7 @@ export async function listOpenAiModels( apiKey: string | undefined, fetchImpl: typeof fetch = fetch, ): Promise { - const url = `${baseUrl.replace(/\/+$/, '')}/models`; + const url = `${withoutTrailingSlashes(baseUrl)}/models`; const body = (await getJson( url, apiKey === undefined ? {} : { authorization: `Bearer ${apiKey}` }, @@ -104,7 +105,7 @@ export async function listAnthropicModels( apiKey: string, fetchImpl: typeof fetch = fetch, ): Promise { - const root = `${baseUrl.replace(/\/+$/, '')}/models`; + const root = `${withoutTrailingSlashes(baseUrl)}/models`; const headers = { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }; const collected: ModelOption[] = []; let after: string | undefined; diff --git a/packages/providers/src/llm/openAiCompatible.ts b/packages/providers/src/llm/openAiCompatible.ts index 69c547c..ac91d01 100644 --- a/packages/providers/src/llm/openAiCompatible.ts +++ b/packages/providers/src/llm/openAiCompatible.ts @@ -1,5 +1,6 @@ import type { Summarizer } from '@ailoud/core'; import { EnvironmentError, FailureError } from '@ailoud/core'; +import { withoutTrailingSlashes } from '@ailoud/core'; /** * A hosted model does not get the hour a local one does. If a request has not @@ -61,7 +62,7 @@ export class OpenAiCompatibleSummarizer implements Summarizer { } public async complete(prompt: string): Promise { - const url = `${this.options.baseUrl.replace(/\/+$/, '')}/chat/completions`; + const url = `${withoutTrailingSlashes(this.options.baseUrl)}/chat/completions`; const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); diff --git a/scripts/check-changelog.mjs b/scripts/check-changelog.mjs new file mode 100755 index 0000000..c452029 --- /dev/null +++ b/scripts/check-changelog.mjs @@ -0,0 +1,89 @@ +#!/usr/bin/env node +// Verify CHANGES.md is fit to release a given tag. +// +// Usage: node scripts/check-changelog.mjs +// The tag falls back to $GITHUB_REF_NAME; a single leading `v` is stripped. +// +// Run on every tag before anything is published. A version number can never +// be reused and the unpublish window is 72 hours, so the changelog is worth +// checking while refusing is still free. +import { + HARD_LIMIT, + SOFT_LIMIT, + baseVersion, + countBullets, + escapeForRegExp, + fail, + isPrerelease, + readChanges, + splitSections, + versionFromTag, + versionHeading, + warn, +} from './lib/changelog.mjs'; + +const SCOPE = 'check-changelog'; + +const rawTag = process.argv[2] ?? process.env.GITHUB_REF_NAME; +if (!rawTag) fail(SCOPE, 'no tag given (pass one, or set $GITHUB_REF_NAME)'); + +const version = versionFromTag(rawTag); +const { sections } = splitSections(readChanges()); + +// Collected rather than thrown one at a time: someone fixing a changelog +// before a release wants the whole list, not one round trip per problem. +const problems = []; + +const own = sections.find((section) => versionHeading(version).test(section.heading)); +if (own === undefined) { + problems.push(`no "## Version ${version}" section. Promote it with bump-version first.`); +} + +const entries = own === undefined ? 0 : countBullets(own.body); +if (own !== undefined && entries === 0) { + problems.push(`the "## Version ${version}" section has no entries.`); +} +if (entries > HARD_LIMIT) { + problems.push( + `${entries} entries in this version; the hard limit is ${HARD_LIMIT}. Merge or cut some.`, + ); +} + +// A final tag must not leave its own pre-release sections behind: they +// describe the same release, and a reader of 1.0.0's notes should not have to +// read 1.0.0-dev.1's as well. fold-prereleases exists to merge them. +if (!isPrerelease(version)) { + const base = baseVersion(version); + const leftovers = sections + .map((section) => section.heading) + .filter((heading) => new RegExp(`^## Version ${escapeForRegExp(base)}-`).test(heading)); + if (leftovers.length > 0) { + problems.push( + `${leftovers.length} pre-release section(s) for ${base} are still present ` + + `(${leftovers.map((h) => h.replace('## Version ', '')).join(', ')}). ` + + `Run: node scripts/fold-prereleases.mjs ${base}`, + ); + } +} + +// Nothing may be stranded in Development at release time: an entry left there +// is a change that shipped and went unmentioned. +const development = sections.find((section) => /^## Development$/.test(section.heading)); +if (development !== undefined) { + const stranded = countBullets(development.body); + if (stranded > 0) { + problems.push( + `${stranded} entr${stranded === 1 ? 'y is' : 'ies are'} still under "## Development".`, + ); + } +} + +if (problems.length > 0) { + for (const problem of problems) console.error(`${SCOPE}: ${problem}`); + process.exit(1); +} + +if (entries > SOFT_LIMIT) { + warn(`${SCOPE}: ${entries} entries, over the soft limit of ${SOFT_LIMIT}.`); +} +console.log(`${SCOPE}: ${version} is ready (${entries} entries).`); diff --git a/scripts/check-changelog.test.mjs b/scripts/check-changelog.test.mjs new file mode 100644 index 0000000..83e418e --- /dev/null +++ b/scripts/check-changelog.test.mjs @@ -0,0 +1,104 @@ +import { describe, expect, it } from 'vitest'; +import { changes, entries, makeSandbox, run, useSandboxes } from './testing/harness.mjs'; + +useSandboxes(); + +describe('check-changelog', () => { + it('passes a version with entries and nothing stranded', () => { + const dir = makeSandbox(changes(`## Development\n\n## Version 1.0.0\n\n### Added\n\n- One.\n`)); + const result = run(dir, 'check-changelog.mjs', ['v1.0.0']); + expect(result.code).toBe(0); + expect(result.stdout).toMatch(/1\.0\.0 is ready \(1 entries\)/); + }); + + it('refuses a version with no section', () => { + const dir = makeSandbox(changes('## Development\n\n- Something.\n')); + const result = run(dir, 'check-changelog.mjs', ['v9.9.9']); + expect(result.code).toBe(1); + expect(result.stderr).toMatch(/no "## Version 9\.9\.9" section/); + }); + + it('refuses a section with no entries', () => { + const dir = makeSandbox(changes('## Development\n\n## Version 1.0.0\n\n### Added\n')); + expect(run(dir, 'check-changelog.mjs', ['1.0.0']).stderr).toMatch(/has no entries/); + }); + + it('refuses entries left stranded under Development', () => { + // An entry left there is a change that shipped and went unmentioned. + const dir = makeSandbox( + changes('## Development\n\n- Forgotten.\n\n## Version 1.0.0\n\n- One.\n'), + ); + const result = run(dir, 'check-changelog.mjs', ['1.0.0']); + expect(result.code).toBe(1); + expect(result.stderr).toMatch(/still under "## Development"/); + }); + + it('refuses a final tag whose pre-release sections were never folded', () => { + const dir = makeSandbox( + changes( + '## Development\n\n## Version 1.0.0\n\n- One.\n\n## Version 1.0.0-dev.1\n\n- Older.\n', + ), + ); + const result = run(dir, 'check-changelog.mjs', ['1.0.0']); + expect(result.code).toBe(1); + expect(result.stderr).toMatch(/pre-release section\(s\) for 1\.0\.0 are still present/); + expect(result.stderr).toMatch(/fold-prereleases\.mjs 1\.0\.0/); + }); + + it('allows a PRE-RELEASE tag to coexist with its siblings', () => { + // Only a final tag has to have folded them. + const dir = makeSandbox( + changes( + '## Development\n\n## Version 1.0.0-dev.2\n\n- Two.\n\n## Version 1.0.0-dev.1\n\n- One.\n', + ), + ); + expect(run(dir, 'check-changelog.mjs', ['v1.0.0-dev.2']).code).toBe(0); + }); + + it('reports every problem at once, not one per run', () => { + const dir = makeSandbox(changes('## Development\n\n- Stranded.\n')); + const result = run(dir, 'check-changelog.mjs', ['1.0.0']); + expect(result.stderr).toMatch(/no "## Version 1\.0\.0" section/); + expect(result.stderr).toMatch(/still under "## Development"/); + }); + + it('refuses past the hard limit and only warns past the soft one', () => { + const over = makeSandbox(changes(`## Development\n\n## Version 1.0.0\n\n${entries(51)}\n`)); + const hard = run(over, 'check-changelog.mjs', ['1.0.0']); + expect(hard.code).toBe(1); + expect(hard.stderr).toMatch(/hard limit is 50/); + + const soft = makeSandbox(changes(`## Development\n\n## Version 1.0.0\n\n${entries(12)}\n`)); + const result = run(soft, 'check-changelog.mjs', ['1.0.0']); + expect(result.code).toBe(0); + expect(`${result.stdout}${result.stderr}`).toMatch(/soft limit of 10/); + }); + + it('refuses with no tag at all', () => { + const dir = makeSandbox(changes('## Development\n')); + expect(run(dir, 'check-changelog.mjs').stderr).toMatch(/no tag given/); + }); + + it('falls back to $GITHUB_REF_NAME, which is how the workflow calls it', () => { + const dir = makeSandbox(changes('## Development\n\n## Version 1.0.0\n\n- One.\n')); + const result = run(dir, 'check-changelog.mjs', [], { env: { GITHUB_REF_NAME: 'v1.0.0' } }); + expect(result.code).toBe(0); + expect(result.stdout).toMatch(/1\.0\.0 is ready/); + }); + + it('warns as a GitHub annotation when it runs on a runner', () => { + // ::warning:: goes to stdout and shows up on the run summary; the plain + // warning goes to stderr. Both paths are exercised because CI once saw + // one of them and no test ever had. + const body = `## Development\n\n## Version 1.0.0\n\n${entries(12)}\n`; + const annotated = run(makeSandbox(body), 'check-changelog.mjs', ['1.0.0'], { + env: { GITHUB_ACTIONS: 'true' }, + }); + expect(annotated.stdout).toMatch(/::warning::.*soft limit of 10/); + expect(annotated.stderr).toBe(''); + + const plain = run(makeSandbox(body), 'check-changelog.mjs', ['1.0.0']); + expect(plain.stderr).toMatch(/soft limit of 10/); + expect(plain.stdout).not.toContain('::warning::'); + }); +}); diff --git a/scripts/check-dependency-age.mjs b/scripts/check-dependency-age.mjs new file mode 100644 index 0000000..d02f43a --- /dev/null +++ b/scripts/check-dependency-age.mjs @@ -0,0 +1,110 @@ +#!/usr/bin/env node +// Refuse dependency versions that are too fresh to trust. +// +// Usage: node scripts/check-dependency-age.mjs [--days 14] +// +// WHY +// +// A compromised release is discovered by other people, and that takes days: +// the npm supply-chain incidents of recent years were all caught within a week +// or two of publication, after the malicious version had already been installed +// by everyone who upgraded immediately. Waiting is the whole mitigation, and it +// costs nothing -- there is no urgency in a patch that has been out for two +// weeks that was not there on day one. +// +// Checked against the version each manifest PINS, not what is installed: this +// project pins exact versions, so the manifest is the decision and the lockfile +// only records it. A spec that is not an exact version is reported rather than +// aged, because a range means the decision was left to whatever resolved last. +import { existsSync, readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { escapePackageName } from './lib/npmOidc.mjs'; +import { + DEFAULT_DAYS, + EXACT, + classify, + collectDependencies, + staleExceptions, +} from './lib/dependencyAge.mjs'; + +const SCOPE = 'check-dependency-age'; +const REGISTRY = 'https://registry.npmjs.org'; +const EXCEPTIONS = 'scripts/dependency-age-exceptions.json'; + +const root = join(dirname(fileURLToPath(import.meta.url)), '..'); +const read = (path) => readFileSync(join(root, path), 'utf8'); + +/** + * When `version` was published, or null if the registry does not know. + * + * A 404 means the registry has no such package, which is a real answer. Any + * other refusal -- a 429, a 502 -- is the registry declining to answer, and + * returning null for it turned this whole check into a no-op that reported + * success: every entry became "unknown", nothing was young, exit 0. So those + * throw, and the rule fails closed. + */ +async function publishedAt(name, version) { + const response = await fetch(`${REGISTRY}/${escapePackageName(name)}`); + if (response.status === 404) return null; + if (!response.ok) { + throw new Error(`the registry answered ${response.status} for ${name}; cannot judge its age`); + } + const { time } = await response.json(); + const stamp = time?.[version]; + if (typeof stamp !== 'string') return null; + const parsed = Date.parse(stamp); + // An unparseable timestamp is not an age. Reported as unknown rather than + // compared as NaN, which produced "published NaN days ago". + return Number.isFinite(parsed) ? parsed : null; +} + +const daysFlag = process.argv.indexOf('--days'); +const days = daysFlag === -1 ? DEFAULT_DAYS : Number(process.argv[daysFlag + 1]); +if (!Number.isFinite(days) || days < 0) { + console.error(`${SCOPE}: --days needs a non-negative number`); + process.exit(1); +} + +const exceptions = existsSync(join(root, EXCEPTIONS)) ? JSON.parse(read(EXCEPTIONS)) : {}; +const dependencies = collectDependencies(read); +const unpinned = [...dependencies].filter(([, spec]) => !EXACT.test(spec)); +const pinned = [...dependencies].filter(([, spec]) => EXACT.test(spec)); + +const entries = await Promise.all( + pinned.map(async ([name, spec]) => [name, spec, await publishedAt(name, spec)]), +); +const now = Date.now(); +const { young, exempt, unknown } = classify(entries, now, days, exceptions); + +console.log(`${SCOPE}: ${pinned.length} pinned direct dependencies, minimum age ${days} days`); + +for (const [name, spec] of unpinned) { + console.log(` ${name}@${spec} is not an exact version; its age was not checked`); +} +for (const { name, spec } of unknown) { + console.log(` ${name}@${spec}: the registry reported no publish time`); +} +for (const { name, spec, ageDays, reason } of exempt) { + console.log(` ${name}@${spec} is ${ageDays.toFixed(1)} days old, allowed: ${reason}`); +} +for (const key of staleExceptions(exceptions, entries, now, days)) { + // Not a failure: a hole that has closed is only clutter. Left unreported it + // would still be there the next time something needed exempting. + console.log(` ${EXCEPTIONS} no longer needs its entry for ${key}`); +} + +if (young.length === 0) { + console.log(`${SCOPE}: nothing newer than ${days} days.`); + process.exit(0); +} + +for (const { name, spec, ageDays } of young) { + console.error(`${SCOPE}: ${name}@${spec} was published ${ageDays.toFixed(1)} days ago`); +} +console.error( + `${SCOPE}: wait until each is ${days} days old, or pin the previous version. ` + + 'A compromised release is found by other people, and that takes days. ' + + `If one fixes a critical advisory, add it to ${EXCEPTIONS} with the advisory ID.`, +); +process.exit(1); diff --git a/scripts/fold-prereleases.mjs b/scripts/fold-prereleases.mjs new file mode 100755 index 0000000..2cce9a4 --- /dev/null +++ b/scripts/fold-prereleases.mjs @@ -0,0 +1,103 @@ +#!/usr/bin/env node +// Fold the pre-release sections of a version back into one released section. +// +// Usage: node scripts/fold-prereleases.mjs +// +// Cutting 1.0.0 after 1.0.0-dev.1, -dev.2 and -rc.1 leaves four changelog +// sections describing one release: three pre-release ones plus whatever +// accumulated in `## Development`. A reader of the released notes wants one. +// This merges them into `## Version `, keeps the subsection grouping, +// drops duplicates, and removes the pre-release sections. +// +// Only the SAME version's pre-releases are folded: 1.0.0-dev.1 goes into 1.0.0 +// and never into 1.1.0. That is what stops an entry from an abandoned line +// reappearing under a release it was never part of. +import { + HARD_LIMIT, + SOFT_LIMIT, + escapeForRegExp, + fail, + fingerprint, + groupBullets, + readChanges, + splitSections, + versionFromTag, + versionHeading, + warn, + writeChanges, +} from './lib/changelog.mjs'; + +const SCOPE = 'fold-prereleases'; + +const version = versionFromTag(process.argv[2] ?? ''); +if (!/^\d+\.\d+\.\d+$/.test(version)) { + fail(SCOPE, `expected a released version like 1.0.0, got "${process.argv[2] ?? ''}"`); +} + +const { head, sections } = splitSections(readChanges()); + +const isOwnPrerelease = (heading) => + new RegExp(`^## Version ${escapeForRegExp(version)}-`).test(heading); +const isDevelopment = (heading) => /^## Development$/.test(heading); +const isTarget = (heading) => versionHeading(version).test(heading); +const isFolded = (heading) => + isOwnPrerelease(heading) || isDevelopment(heading) || isTarget(heading); + +const folded = sections.filter((section) => isFolded(section.heading)); +if (folded.length === 0) fail(SCOPE, 'nothing to fold: no Development and no matching sections'); +const prereleaseCount = folded.filter((section) => isOwnPrerelease(section.heading)).length; + +// Sections are read in document order, which puts Development first and the +// pre-releases below it, newest to oldest. First occurrence of a duplicate +// wins, so the newest wording of an entry is the one kept. +const merged = new Map(); +const seen = new Set(); +for (const section of folded) { + for (const [name, entries] of groupBullets(section.body)) { + for (const entry of entries) { + const key = fingerprint(entry); + if (key === '' || seen.has(key)) continue; + seen.add(key); + if (!merged.has(name)) merged.set(name, []); + merged.get(name).push(entry); + } + } +} + +const total = [...merged.values()].reduce((sum, entries) => sum + entries.length, 0); +if (total === 0) fail(SCOPE, 'every folded section was empty'); +if (total > HARD_LIMIT) { + fail( + SCOPE, + `the folded section would have ${total} entries; the hard limit is ${HARD_LIMIT}. ` + + 'Merge related entries in CHANGES.md before cutting the release.', + ); +} +if (total > SOFT_LIMIT) { + warn(`${SCOPE}: ${total} entries, over the soft limit of ${SOFT_LIMIT}.`); +} + +const body = [...merged.entries()] + .filter(([, entries]) => entries.length > 0) + .map(([name, entries]) => [`### ${name}`, '', ...entries.map((e) => e.join('\n'))].join('\n')) + .join('\n\n'); + +const kept = sections.filter((section) => !isFolded(section.heading)); + +writeChanges( + [ + head.replace(/\n+$/, ''), + '## Development', + `## Version ${version}`, + body, + ...kept.map((section) => [section.heading, section.body.replace(/\n+$/, '')].join('\n')), + ] + .filter((part) => part !== '') + .join('\n\n') + .replace(/\n{3,}/g, '\n\n'), +); + +console.log( + `${SCOPE}: folded ${prereleaseCount} pre-release section(s) and Development into ` + + `## Version ${version} (${total} entries)`, +); diff --git a/scripts/fold-prereleases.test.mjs b/scripts/fold-prereleases.test.mjs new file mode 100644 index 0000000..8e78ea0 --- /dev/null +++ b/scripts/fold-prereleases.test.mjs @@ -0,0 +1,114 @@ +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { changes, entries, makeSandbox, run, useSandboxes } from './testing/harness.mjs'; + +useSandboxes(); + +describe('fold-prereleases', () => { + const withPrereleases = changes( + [ + '## Development', + '', + '### Added', + '', + '- Newest, from Development.', + '', + '## Version 1.0.0-dev.2', + '', + '### Added', + '', + '- From dev.2.', + '- A duplicate that', + ' wraps across lines.', + '', + '### Fixed', + '', + '- Only dev.2 had this.', + '', + '## Version 1.0.0-dev.1', + '', + '### Added', + '', + '- From dev.1.', + '- A duplicate that wraps across lines.', + '', + '## Version 0.9.0', + '', + '### Added', + '', + '- An older release.', + '', + ].join('\n'), + ); + + it('merges the pre-releases and Development into one released section', () => { + const dir = makeSandbox(withPrereleases); + const result = run(dir, 'fold-prereleases.mjs', ['1.0.0']); + expect(result.code).toBe(0); + const out = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + expect(out).toContain('## Version 1.0.0'); + expect(out).not.toContain('1.0.0-dev.1'); + expect(out).not.toContain('1.0.0-dev.2'); + }); + + it('drops a duplicate even when one copy was rewrapped', () => { + const dir = makeSandbox(withPrereleases); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + const out = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + expect(out.match(/duplicate that/g)).toHaveLength(1); + }); + + it('keeps the subsection grouping', () => { + const dir = makeSandbox(withPrereleases); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + const out = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + expect(out).toContain('### Added'); + expect(out).toContain('### Fixed'); + expect(out).toContain('- Only dev.2 had this.'); + }); + + it('leaves an unrelated release alone', () => { + // 1.0.0-dev.1 folds into 1.0.0 and never into another version. + const dir = makeSandbox(withPrereleases); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + const out = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + expect(out).toContain('## Version 0.9.0'); + expect(out).toContain('- An older release.'); + }); + + it('leaves an empty Development heading for the next cycle', () => { + const dir = makeSandbox(withPrereleases); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + const out = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + const development = out.slice(out.indexOf('## Development'), out.indexOf('## Version 1.0.0')); + expect(development).not.toMatch(/^- /m); + }); + + it('is idempotent: folding twice changes nothing more', () => { + const dir = makeSandbox(withPrereleases); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + const once = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + run(dir, 'fold-prereleases.mjs', ['1.0.0']); + expect(readFileSync(join(dir, 'CHANGES.md'), 'utf8')).toBe(once); + }); + + it('refuses anything that is not a released version', () => { + const dir = makeSandbox(withPrereleases); + for (const bad of ['1.0.0-dev.1', 'latest', '1.0', '']) { + const result = run(dir, 'fold-prereleases.mjs', bad === '' ? [] : [bad]); + expect(result.code, bad).toBe(1); + expect(result.stderr, bad).toMatch(/expected a released version/); + } + }); + + it('refuses past the hard limit rather than writing an oversized section', () => { + const dir = makeSandbox(changes(`## Development\n\n${entries(51)}\n`)); + const before = readFileSync(join(dir, 'CHANGES.md'), 'utf8'); + const result = run(dir, 'fold-prereleases.mjs', ['1.0.0']); + expect(result.code).toBe(1); + expect(result.stderr).toMatch(/hard limit is 50/); + // And left the file exactly as it was. + expect(readFileSync(join(dir, 'CHANGES.md'), 'utf8')).toBe(before); + }); +}); diff --git a/scripts/lib/changelog.mjs b/scripts/lib/changelog.mjs new file mode 100644 index 0000000..879b9fa --- /dev/null +++ b/scripts/lib/changelog.mjs @@ -0,0 +1,152 @@ +// Shared vocabulary for the release scripts: the limits, the two ways of +// reporting, and the CHANGES.md parsing all three of them need. +// +// Extracted because the numbers and the parsing were copied into every script +// that touched the changelog. Copied limits drift, and a limit that differs +// between the script that warns and the script that refuses is worse than no +// limit -- one of them is then wrong and nobody knows which. +import { readFileSync, writeFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +/** + * The published packages, in dependency order. + * + * The CLI is last because the other two are its dependencies: publish and + * deprecate both have to walk them in this order. + */ +export const PACKAGES = ['@ailoud/core', '@ailoud/providers', 'ailoud']; + +/** + * Which pre-release tags belong to a released version, and which may be deleted. + * + * Pure: it takes the tag list and an "is this commit on main" predicate rather + * than running git, so the decision is testable without a repository. + */ +export function planRetirement(version, tags, isOnMain) { + const prefix = `v${version}-`; + const mine = tags.filter((tag) => tag.startsWith(prefix)).sort(); + return { + versions: mine.map((tag) => versionFromTag(tag)), + deletable: mine.filter((tag) => isOnMain(tag)), + kept: mine.filter((tag) => !isOnMain(tag)), + }; +} + +/** Entries per version section. Stated once here, quoted in AGENTS.md and CHANGES.md. */ +export const SOFT_LIMIT = 10; +export const HARD_LIMIT = 50; + +/** Where the repository root is, relative to scripts/lib/. */ +export const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..'); +export const CHANGES_PATH = join(ROOT, 'CHANGES.md'); + +/** Stops with a message the caller owns. */ +export function fail(scope, message) { + console.error(`${scope}: ${message}`); + process.exit(1); +} + +/** + * A warning, not an error. + * + * `console.error` made a soft-limit notice look like a failure in a terminal + * and in a CI log alike. Under GitHub Actions this becomes an annotation, + * which is what a warning should be there. + */ +export function warn(message) { + if (process.env.GITHUB_ACTIONS === 'true') { + console.log(`::warning::${message}`); + return; + } + console.warn(message); +} + +/** `v1.2.3` and `1.2.3` both mean 1.2.3. */ +export function versionFromTag(tag) { + return String(tag).replace(/^v/, ''); +} + +/** The release part of a version: `1.0.0-dev.2` -> `1.0.0`. */ +export function baseVersion(version) { + return version.split('-')[0]; +} + +export function isPrerelease(version) { + return version.includes('-'); +} + +/** Safe inside a RegExp built from a version string. */ +export function escapeForRegExp(value) { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +export function readChanges() { + return readFileSync(CHANGES_PATH, 'utf8'); +} + +export function writeChanges(text) { + writeFileSync(CHANGES_PATH, `${text.trimEnd()}\n`); +} + +/** Every `## ` section as {heading, body}, plus whatever precedes the first one. */ +export function splitSections(text) { + const lines = text.split('\n'); + const starts = []; + lines.forEach((line, at) => { + if (line.startsWith('## ')) starts.push(at); + }); + const head = lines.slice(0, starts[0] ?? lines.length).join('\n'); + const sections = starts.map((start, index) => ({ + heading: lines[start], + body: lines.slice(start + 1, starts[index + 1] ?? lines.length).join('\n'), + })); + return { head, sections }; +} + +/** + * Bullets of a section body, grouped by their `### ` subsection. + * + * A continuation line is attached to the bullet above it rather than dropped, + * which is what keeps an 80-column entry whole -- every wrapped entry would + * otherwise lose everything after its first line. + */ +export function groupBullets(body) { + const groups = new Map(); + let current = 'Added'; + for (const line of body.split('\n')) { + if (line.startsWith('### ')) { + current = line.slice(4).trim(); + if (!groups.has(current)) groups.set(current, []); + continue; + } + if (/^\s*- /.test(line)) { + if (!groups.has(current)) groups.set(current, []); + groups.get(current).push([line]); + continue; + } + const entries = groups.get(current); + if (entries !== undefined && entries.length > 0 && line.trim() !== '') { + entries[entries.length - 1].push(line); + } + } + return groups; +} + +export function countBullets(body) { + return body.split('\n').filter((line) => /^\s*- /.test(line)).length; +} + +/** Whitespace-insensitive, so the same entry rewrapped is still the same entry. */ +export function fingerprint(entry) { + return entry.join(' ').replace(/\s+/g, ' ').trim().toLowerCase(); +} + +/** The body of the section whose heading matches, or null. */ +export function findSection(sections, pattern) { + return sections.find((section) => pattern.test(section.heading)) ?? null; +} + +export function versionHeading(version) { + return new RegExp(`^## Version ${escapeForRegExp(version)}(\\s|$)`); +} diff --git a/scripts/lib/changelog.test.mjs b/scripts/lib/changelog.test.mjs new file mode 100644 index 0000000..a65495b --- /dev/null +++ b/scripts/lib/changelog.test.mjs @@ -0,0 +1,196 @@ +import { describe, expect, it } from 'vitest'; +import { + HARD_LIMIT, + SOFT_LIMIT, + baseVersion, + countBullets, + escapeForRegExp, + fingerprint, + groupBullets, + isPrerelease, + planRetirement, + splitSections, + versionFromTag, + versionHeading, +} from './changelog.mjs'; + +describe('the limits', () => { + it('are stated once, and the soft one is below the hard one', () => { + // Copied limits drift, and a limit that differs between the script that + // warns and the script that refuses is worse than no limit at all. + expect(SOFT_LIMIT).toBe(10); + expect(HARD_LIMIT).toBe(50); + expect(SOFT_LIMIT).toBeLessThan(HARD_LIMIT); + }); +}); + +describe('versionFromTag', () => { + it('strips one leading v and nothing else', () => { + expect(versionFromTag('v1.2.3')).toBe('1.2.3'); + expect(versionFromTag('1.2.3')).toBe('1.2.3'); + expect(versionFromTag('v1.2.3-dev.1')).toBe('1.2.3-dev.1'); + // Not a recursive strip: a version does not start with v twice. + expect(versionFromTag('vv1.2.3')).toBe('v1.2.3'); + }); +}); + +describe('baseVersion and isPrerelease', () => { + it('splits a pre-release from its release', () => { + expect(baseVersion('1.0.0-dev.2')).toBe('1.0.0'); + expect(baseVersion('1.0.0')).toBe('1.0.0'); + expect(isPrerelease('1.0.0-dev.2')).toBe(true); + expect(isPrerelease('1.0.0-rc.1')).toBe(true); + expect(isPrerelease('1.0.0')).toBe(false); + }); +}); + +describe('escapeForRegExp and versionHeading', () => { + it('does not let a dot in a version match any character', () => { + // Unescaped, `1.0.0` matches `1x0y0` -- and a heading for another version. + expect(escapeForRegExp('1.0.0')).toBe('1\\.0\\.0'); + expect(versionHeading('1.0.0').test('## Version 1.0.0')).toBe(true); + expect(versionHeading('1.0.0').test('## Version 1x0y0')).toBe(false); + }); + + it('matches a heading with a trailing date but not a longer version', () => { + expect(versionHeading('1.0.0').test('## Version 1.0.0 -- 2026-09-05')).toBe(true); + expect(versionHeading('1.0.0').test('## Version 1.0.0-dev.1')).toBe(false); + expect(versionHeading('1.0.0').test('## Version 1.0.10')).toBe(false); + }); +}); + +describe('splitSections', () => { + const text = [ + '# Title', + '', + 'preamble', + '', + '## Development', + '', + '- one', + '', + '## Version 1.0.0', + '', + '- two', + ].join('\n'); + + it('keeps whatever precedes the first section', () => { + expect(splitSections(text).head).toContain('preamble'); + }); + + it('returns each section with its heading and body', () => { + const { sections } = splitSections(text); + expect(sections.map((s) => s.heading)).toEqual(['## Development', '## Version 1.0.0']); + expect(sections[0].body).toContain('- one'); + expect(sections[1].body).toContain('- two'); + }); + + it('handles a file with no sections at all', () => { + const { head, sections } = splitSections('# Just a title\n'); + expect(sections).toEqual([]); + expect(head).toContain('Just a title'); + }); + + it('does not treat a ### subsection as a section', () => { + const { sections } = splitSections('## Version 1.0.0\n\n### Added\n\n- x\n'); + expect(sections).toHaveLength(1); + expect(sections[0].body).toContain('### Added'); + }); +}); + +describe('groupBullets', () => { + it('groups by subsection, in first-seen order', () => { + const groups = groupBullets('### Added\n\n- a\n\n### Fixed\n\n- b\n'); + expect([...groups.keys()]).toEqual(['Added', 'Fixed']); + expect(groups.get('Added')).toEqual([['- a']]); + }); + + it('keeps a wrapped entry whole', () => { + // Every 80-column entry wraps; dropping the continuation would lose + // everything after the first line. + const groups = groupBullets('### Added\n\n- first line\n second line\n'); + expect(groups.get('Added')).toEqual([['- first line', ' second line']]); + }); + + it('defaults to Added when a body has no subsection heading', () => { + expect([...groupBullets('- loose entry\n').keys()]).toEqual(['Added']); + }); + + it('ignores a blank line between entries', () => { + expect(groupBullets('- a\n\n- b\n').get('Added')).toEqual([['- a'], ['- b']]); + }); + + it('records a subsection that has no entries', () => { + const groups = groupBullets('### Added\n\n### Fixed\n\n- b\n'); + expect(groups.get('Added')).toEqual([]); + }); +}); + +describe('fingerprint', () => { + it('sees the same entry rewrapped as the same entry', () => { + // The reason a duplicate across two dev tags is caught even after one of + // them was reflowed. + expect(fingerprint(['- a duplicated entry that', ' wraps across two lines.'])).toBe( + fingerprint(['- a duplicated entry that wraps across two lines.']), + ); + }); + + it('is case-insensitive', () => { + expect(fingerprint(['- Same Thing'])).toBe(fingerprint(['- same thing'])); + }); + + it('keeps genuinely different entries apart', () => { + expect(fingerprint(['- one'])).not.toBe(fingerprint(['- two'])); + }); +}); + +describe('countBullets', () => { + it('counts entries, not their continuation lines', () => { + expect(countBullets('- a\n continued\n- b\n')).toBe(2); + }); + + it('counts an indented entry', () => { + expect(countBullets(' - nested\n')).toBe(1); + }); + + it('counts nothing in prose', () => { + expect(countBullets('### Added\n\nsome prose\n')).toBe(0); + }); +}); + +describe('planRetirement', () => { + const tags = [ + 'v1.0.0-dev.2', + 'v1.0.0-dev.1', + 'v1.0.0-rc.1', + 'v1.1.0-dev.1', + 'v1.0.0', + 'v10.0.0-dev.1', + ]; + + it('takes the pre-releases of one version and no others', () => { + // v1.1.0-dev.1 belongs to a version that has not been released, and + // v10.0.0-dev.1 shares a prefix with "v1" only as text. + const { versions } = planRetirement('1.0.0', tags, () => true); + expect(versions).toEqual(['1.0.0-dev.1', '1.0.0-dev.2', '1.0.0-rc.1']); + }); + + it('leaves the final release itself alone', () => { + const { versions } = planRetirement('1.0.0', tags, () => true); + expect(versions).not.toContain('1.0.0'); + }); + + it('splits the tags by whether their commit is on main', () => { + // A tag whose commit is not reachable from main cannot be deleted: the + // commit could then be collected, and the published provenance attests it. + const onMain = (tag) => tag !== 'v1.0.0-dev.2'; + const { deletable, kept } = planRetirement('1.0.0', tags, onMain); + expect(deletable).toEqual(['v1.0.0-dev.1', 'v1.0.0-rc.1']); + expect(kept).toEqual(['v1.0.0-dev.2']); + }); + + it('plans nothing for a version that never had a pre-release', () => { + const plan = planRetirement('2.0.0', tags, () => true); + expect(plan).toEqual({ versions: [], deletable: [], kept: [] }); + }); +}); diff --git a/scripts/lib/dependencyAge.mjs b/scripts/lib/dependencyAge.mjs new file mode 100644 index 0000000..08a88fe --- /dev/null +++ b/scripts/lib/dependencyAge.mjs @@ -0,0 +1,87 @@ +// The dependency-age rule, apart from the command that applies it. +// +// Pure and I/O-free so the rule is testable without the network, and so the +// file that exports it can be imported without running a CLI. +export const DEFAULT_DAYS = 14; +export const DAY_MS = 24 * 60 * 60 * 1000; + +/** An exact version, as this project pins them. Anything else is not aged. */ +export const EXACT = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; + +/** The manifests whose direct dependencies are this project's decisions. */ +export const MANIFESTS = [ + 'package.json', + 'packages/core/package.json', + 'packages/providers/package.json', + 'apps/cli/package.json', +]; + +/** + * Every direct dependency across the manifests, as name -> spec. + * + * Workspace siblings are skipped: `workspace:*` is not a registry version, and + * the packages it names are ours to trust or not on other grounds. + */ +export function collectDependencies(read) { + const found = new Map(); + for (const manifest of MANIFESTS) { + const parsed = JSON.parse(read(manifest)); + for (const field of ['dependencies', 'devDependencies', 'optionalDependencies']) { + for (const [name, spec] of Object.entries(parsed[field] ?? {})) { + if (spec.startsWith('workspace:')) continue; + found.set(name, spec); + } + } + } + return found; +} + +/** + * Sorts `entries` into what fails the rule, what is exempt, and what cannot be + * judged. + * + * `entries` is `[name, spec, publishedAtMs]`; a null time means the registry + * reported none, which is surfaced rather than treated as old -- treating it + * as old would hide exactly the version whose metadata is odd. + * + * `exceptions` maps `name@version` to a reason. The rule has to yield to a + * critical advisory: waiting two weeks with a known exploit is worse than + * installing a version nobody has audited yet. Exempting one is a decision + * that belongs in the repository with its reason attached, not an argument + * someone remembers to pass. + */ +export function classify(entries, nowMs, days, exceptions = {}) { + const cutoff = nowMs - days * DAY_MS; + const young = []; + const exempt = []; + const unknown = []; + for (const [name, spec, publishedAtMs] of entries) { + const reason = exceptions[`${name}@${spec}`]; + if (publishedAtMs === null) { + unknown.push({ name, spec }); + continue; + } + if (publishedAtMs <= cutoff) continue; + const ageDays = (nowMs - publishedAtMs) / DAY_MS; + if (reason !== undefined) exempt.push({ name, spec, ageDays, reason }); + else young.push({ name, spec, ageDays }); + } + return { young, exempt, unknown }; +} + +/** + * Exceptions that no longer apply, because the version they cover has aged + * past the rule or is no longer a dependency. + * + * Reported so the file does not accumulate permanent holes: an exception is a + * statement about one moment, and it stops being true. + */ +export function staleExceptions(exceptions, entries, nowMs, days) { + const cutoff = nowMs - days * DAY_MS; + const live = new Map(entries.map(([name, spec, at]) => [`${name}@${spec}`, at])); + return Object.keys(exceptions).filter((key) => { + if (!live.has(key)) return true; + const at = live.get(key); + return at !== null && at <= cutoff; + }); +} diff --git a/scripts/lib/dependencyAge.test.mjs b/scripts/lib/dependencyAge.test.mjs new file mode 100644 index 0000000..8d41f6c --- /dev/null +++ b/scripts/lib/dependencyAge.test.mjs @@ -0,0 +1,101 @@ +import { describe, expect, it } from 'vitest'; +import { + DAY_MS, + DEFAULT_DAYS, + classify, + collectDependencies, + staleExceptions, +} from './dependencyAge.mjs'; + +const NOW = Date.parse('2026-09-05T00:00:00Z'); +const at = (daysAgo) => NOW - daysAgo * DAY_MS; + +describe('the window', () => { + it('is fourteen days, stated once', () => { + expect(DEFAULT_DAYS).toBe(14); + }); +}); + +describe('collectDependencies', () => { + const manifests = { + 'package.json': { devDependencies: { prettier: '3.9.6' } }, + 'packages/core/package.json': { dependencies: {} }, + 'packages/providers/package.json': { + dependencies: { '@ailoud/core': 'workspace:*', yaml: '2.9.0' }, + }, + 'apps/cli/package.json': { + dependencies: { commander: '15.0.0' }, + optionalDependencies: { fsevents: '2.3.3' }, + }, + }; + const read = (path) => JSON.stringify(manifests[path]); + + it('collects direct dependencies of every kind across the manifests', () => { + const found = collectDependencies(read); + expect([...found]).toEqual([ + ['prettier', '3.9.6'], + ['yaml', '2.9.0'], + ['commander', '15.0.0'], + ['fsevents', '2.3.3'], + ]); + }); + + it('skips workspace siblings, which are not registry versions', () => { + expect(collectDependencies(read).has('@ailoud/core')).toBe(false); + }); +}); + +describe('classify', () => { + it('flags a version published inside the window', () => { + const { young } = classify([['left-pad', '1.0.0', at(3)]], NOW, 14); + expect(young).toHaveLength(1); + expect(young[0].ageDays).toBeCloseTo(3); + }); + + it('accepts one published outside it, and one exactly at the boundary', () => { + // The rule is "at least this old", so 14 days old passes at 14 days. + expect(classify([['a', '1.0.0', at(15)]], NOW, 14).young).toEqual([]); + expect(classify([['a', '1.0.0', at(14)]], NOW, 14).young).toEqual([]); + }); + + it('exempts a version named in the exceptions, and keeps its reason', () => { + // The rule yields to a critical advisory: two weeks with a known exploit + // is worse than a version nobody has audited yet. + const { young, exempt } = classify([['left-pad', '2.0.0', at(1)]], NOW, 14, { + 'left-pad@2.0.0': 'fixes GHSA-xxxx-yyyy-zzzz (critical)', + }); + expect(young).toEqual([]); + expect(exempt[0].reason).toContain('GHSA-xxxx-yyyy-zzzz'); + }); + + it('does not let an exception cover a different version of the same package', () => { + const { young } = classify([['left-pad', '2.0.1', at(1)]], NOW, 14, { + 'left-pad@2.0.0': 'fixes something else', + }); + expect(young).toHaveLength(1); + }); + + it('reports an unknown publish time instead of assuming it is old', () => { + const { young, unknown } = classify([['left-pad', '1.0.0', null]], NOW, 14); + expect(young).toEqual([]); + expect(unknown).toEqual([{ name: 'left-pad', spec: '1.0.0' }]); + }); +}); + +describe('staleExceptions', () => { + it('names an exception whose version has since aged past the rule', () => { + const entries = [['left-pad', '2.0.0', at(30)]]; + expect(staleExceptions({ 'left-pad@2.0.0': 'was urgent' }, entries, NOW, 14)).toEqual([ + 'left-pad@2.0.0', + ]); + }); + + it('names an exception for something no longer depended on', () => { + expect(staleExceptions({ 'gone@1.0.0': 'was urgent' }, [], NOW, 14)).toEqual(['gone@1.0.0']); + }); + + it('leaves an exception that is still doing work', () => { + const entries = [['left-pad', '2.0.0', at(2)]]; + expect(staleExceptions({ 'left-pad@2.0.0': 'urgent' }, entries, NOW, 14)).toEqual([]); + }); +}); diff --git a/scripts/lib/npmOidc.mjs b/scripts/lib/npmOidc.mjs new file mode 100644 index 0000000..3f3cbda --- /dev/null +++ b/scripts/lib/npmOidc.mjs @@ -0,0 +1,150 @@ +// Exchange a CI OIDC identity for a short-lived npm token. +// +// This is what `npm publish` does for itself and exposes to nothing else: +// trusted publishing is defined for publishing, so `npm deprecate` and +// `npm dist-tag` in the same job have nothing to authenticate with. Doing the +// two calls here means retiring a release needs no stored credential either. +// +// Read out of npm's own implementation (lib/utils/oidc.js in npm 11): +// +// GET $ACTIONS_ID_TOKEN_REQUEST_URL&audience=npm:registry.npmjs.org +// Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN -> { value } +// POST /-/npm/v1/oidc/token/exchange/package/ +// Authorization: Bearer -> { token } +// +// The token is minted per package and is never printed or written anywhere but +// the temporary npmrc the caller hands to npm. +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const REGISTRY = 'https://registry.npmjs.org'; + +/** + * npm's escaping: a scope's slash becomes %2f, everything else is literal. + * + * `replaceAll`, though a package name holds at most one slash: `replace` with a + * string argument substitutes only the first match, so the single-slash case + * was right by accident rather than by what the code said. + */ +export function escapePackageName(name) { + return name.replaceAll('/', '%2f'); +} + +/** True when this process is a GitHub Actions job with `id-token: write`. */ +export function canExchange() { + return Boolean( + process.env.ACTIONS_ID_TOKEN_REQUEST_URL && process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN, + ); +} + +/** + * A short-lived npm token for one package, or null with the reason logged. + * + * Returns null rather than throwing: a caller that cannot get a token should + * be able to fall back on an ambient `npm login`, which is how this runs from + * a laptop. + */ +export async function tokenForPackage(name, log = console.error) { + if (!canExchange()) { + log('npm-oidc: not a GitHub Actions job with id-token: write'); + return null; + } + const idUrl = new URL(process.env.ACTIONS_ID_TOKEN_REQUEST_URL); + idUrl.searchParams.set('audience', `npm:${new URL(REGISTRY).hostname}`); + + let idResponse; + try { + idResponse = await fetch(idUrl, { + headers: { + accept: 'application/json', + authorization: `Bearer ${process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN}`, + }, + }); + } catch (error) { + // The contract above says null, not a throw: a caller that can fall back + // on an ambient `npm login` should get the chance, and a DNS or TLS + // failure is exactly the transient case that fallback is for. + log(`npm-oidc: could not reach GitHub for an id token: ${error.message}`); + return null; + } + if (!idResponse.ok) { + log(`npm-oidc: GitHub refused the id token (${idResponse.status})`); + return null; + } + const { value: idToken } = await idResponse.json(); + if (typeof idToken !== 'string' || idToken === '') { + log('npm-oidc: GitHub returned no id token'); + return null; + } + + // The claims, not the token. npm binds a trusted publisher to a workflow + // file, so a rejection usually means the identity is right and the workflow + // is not the one configured -- which is invisible unless the claims are + // printed. They are public metadata; the token they came in is not. + log(`npm-oidc: identity ${describeClaims(idToken)}`); + + const exchange = `${REGISTRY}/-/npm/v1/oidc/token/exchange/package/${escapePackageName(name)}`; + let response; + try { + response = await fetch(exchange, { + method: 'POST', + headers: { authorization: `Bearer ${idToken}`, accept: 'application/json' }, + }); + } catch (error) { + log(`npm-oidc: could not reach the registry to exchange: ${error.message}`); + return null; + } + if (!response.ok) { + // The body carries npm's reason -- usually that no trusted publisher is + // attached to this package -- and holds no secret. + const body = await response.text(); + log(`npm-oidc: ${name} exchange failed (${response.status}): ${body.slice(0, 200)}`); + return null; + } + const { token } = await response.json(); + if (typeof token !== 'string' || token === '') { + log(`npm-oidc: ${name} exchange returned no token`); + return null; + } + return token; +} + +/** + * The claims npm matches a trusted publisher against, as one line. + * + * `workflow_ref` is the workflow the run entered through; `job_workflow_ref` + * is the reusable workflow the job itself is defined in. They differ exactly + * when one workflow calls another, which is the case this exists to explain. + */ +function describeClaims(idToken) { + try { + const [, payload] = idToken.split('.'); + const claims = JSON.parse(Buffer.from(payload, 'base64url').toString('utf8')); + return [ + `sub=${claims.sub}`, + `workflow_ref=${claims.workflow_ref}`, + `job_workflow_ref=${claims.job_workflow_ref}`, + ].join(' '); + } catch { + return '(claims unreadable)'; + } +} + +/** + * Runs `body(env)` with a temporary npmrc holding the token, then removes it. + * + * A file rather than an argument or an env var: a token on a command line is + * visible to every process on the machine, and npm's env form of this key + * needs a variable name containing slashes and a colon. + */ +export function withNpmToken(token, body) { + const dir = mkdtempSync(join(tmpdir(), 'ailoud-npmrc-')); + const file = join(dir, '.npmrc'); + try { + writeFileSync(file, `//registry.npmjs.org/:_authToken=${token}\n`, { mode: 0o600 }); + return body({ ...process.env, NPM_CONFIG_USERCONFIG: file }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} diff --git a/scripts/lib/npmOidc.test.mjs b/scripts/lib/npmOidc.test.mjs new file mode 100644 index 0000000..e327632 --- /dev/null +++ b/scripts/lib/npmOidc.test.mjs @@ -0,0 +1,67 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; +import { canExchange, escapePackageName, withNpmToken } from './npmOidc.mjs'; + +describe('escapePackageName', () => { + it('escapes a scope the way npm does', () => { + expect(escapePackageName('@ailoud/core')).toBe('@ailoud%2fcore'); + }); + + it('leaves an unscoped name alone', () => { + expect(escapePackageName('ailoud')).toBe('ailoud'); + }); +}); + +describe('canExchange', () => { + it('is false outside a job with id-token: write', () => { + // Both variables are needed; GitHub sets them only for `id-token: write`, + // and the tests' own harness scrubs every GITHUB_ variable. + const saved = [ + process.env.ACTIONS_ID_TOKEN_REQUEST_URL, + process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN, + ]; + try { + delete process.env.ACTIONS_ID_TOKEN_REQUEST_URL; + delete process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN; + expect(canExchange()).toBe(false); + process.env.ACTIONS_ID_TOKEN_REQUEST_URL = 'https://example.invalid/token'; + expect(canExchange()).toBe(false); + process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN = 'x'; + expect(canExchange()).toBe(true); + } finally { + for (const [i, key] of [ + 'ACTIONS_ID_TOKEN_REQUEST_URL', + 'ACTIONS_ID_TOKEN_REQUEST_TOKEN', + ].entries()) { + if (saved[i] === undefined) delete process.env[key]; + else process.env[key] = saved[i]; + } + } + }); +}); + +describe('withNpmToken', () => { + it('writes the token to a private npmrc and points npm at it', () => { + let seen = null; + const result = withNpmToken('secret-token', (env) => { + seen = env.NPM_CONFIG_USERCONFIG; + expect(readFileSync(seen, 'utf8')).toBe('//registry.npmjs.org/:_authToken=secret-token\n'); + return 'body ran'; + }); + expect(result).toBe('body ran'); + // The point of the temporary file: a token on a command line is visible to + // every process on the machine, and one left on disk outlives the job. + expect(existsSync(seen)).toBe(false); + }); + + it('removes the npmrc even when the body throws', () => { + let seen = null; + expect(() => + withNpmToken('t', (env) => { + seen = env.NPM_CONFIG_USERCONFIG; + throw new Error('boom'); + }), + ).toThrow('boom'); + expect(existsSync(seen)).toBe(false); + }); +}); diff --git a/scripts/make-fixtures.mjs b/scripts/make-fixtures.mjs index 87331c2..fb6d779 100644 --- a/scripts/make-fixtures.mjs +++ b/scripts/make-fixtures.mjs @@ -85,6 +85,52 @@ const FIXTURES = [ { name: 'two-speakers-mixed', voices: ['Daniel', 'Milena'] }, ]; +/** + * The video containers to wrap an audio fixture in. + * + * `ailoud audio import` accepts video because a meeting recording usually is + * one, and only the audio track matters -- so these need no picture worth + * looking at. 32x32 of black at 5 fps keeps each file under 15 kB while still + * being a real, decodable video stream rather than a container with a stub in + * it. Each gets the codec pair it actually carries in the wild: H.264 with AAC + * in mp4 and mov, VP9 with Opus in webm, H.264 with Opus in mkv. + * @type {{container: string, args: string[]}[]} + */ +const VIDEO_CONTAINERS = [ + { + container: 'mp4', + args: ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '51', '-c:a', 'aac', '-b:a', '32k'], + }, + { + container: 'mov', + args: ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '51', '-c:a', 'aac', '-b:a', '32k'], + }, + { + container: 'mkv', + args: [ + '-c:v', + 'libx264', + '-preset', + 'veryfast', + '-crf', + '51', + '-c:a', + 'libopus', + '-b:a', + '24k', + ], + }, + { + // No -preset: libvpx-vp9 does not take one, and -b:v 0 is what makes + // -crf the only thing deciding the size. + container: 'webm', + args: ['-c:v', 'libvpx-vp9', '-b:v', '0', '-crf', '63', '-c:a', 'libopus', '-b:a', '24k'], + }, +]; + +/** Which audio fixture the video fixtures wrap. Its .txt is their reference too. */ +const VIDEO_SOURCE = 'en-short'; + // Generous, not tight: these clips are a few seconds of speech each, but a // loaded machine (or a cold-start speech-synthesis voice download) can take // a while, and a hang here should still end the script rather than run @@ -173,6 +219,56 @@ function concatenate(clauseWavs, outputWav) { ]); } +/** The duration of a media file in seconds, as ffprobe reports it. */ +function durationOf(path) { + const seconds = execFileSync( + 'ffprobe', + ['-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', path], + { encoding: 'utf8', timeout: COMMAND_TIMEOUT_MS }, + ).trim(); + if (!/^[0-9]+(\.[0-9]+)?$/.test(seconds)) { + throw new Error(`ffprobe gave no duration for ${path}: ${seconds}`); + } + return seconds; +} + +/** + * Wraps a WAV fixture in each video container. + * + * The video length is taken from the audio rather than left to `-shortest`, + * which produced an 18-second mp4 from a 2.5-second source: the muxer wrote + * its own idea of the duration and the fixture no longer matched the clip it + * came from. + */ +function makeVideos(wav) { + const seconds = durationOf(wav); + for (const { container, args } of VIDEO_CONTAINERS) { + const output = join(fixturesDir, `${VIDEO_SOURCE}.${container}`); + console.log(`generating ${VIDEO_SOURCE}.${container} (${seconds}s)`); + run('ffmpeg', [ + '-v', + 'error', + '-y', + '-f', + 'lavfi', + '-i', + `color=c=black:s=32x32:r=5:d=${seconds}`, + '-i', + wav, + '-t', + seconds, + '-map', + '0:v', + '-map', + '1:a', + '-pix_fmt', + 'yuv420p', + ...args, + output, + ]); + } +} + function main() { const scratch = mkdtempSync(join(tmpdir(), 'ailoud-fixtures-')); try { @@ -207,10 +303,11 @@ function main() { const wav = join(fixturesDir, `${fixture.name}.wav`); concatenate(clauseWavs, wav); } + makeVideos(join(fixturesDir, `${VIDEO_SOURCE}.wav`)); } finally { rmSync(scratch, { recursive: true, force: true }); } - console.log('done. Review fixtures/*.wav and commit them (they go through Git LFS).'); + console.log('done. Review the fixtures and commit them (they go through Git LFS).'); } main(); diff --git a/scripts/preflight-npm-auth.mjs b/scripts/preflight-npm-auth.mjs new file mode 100644 index 0000000..37aea8c --- /dev/null +++ b/scripts/preflight-npm-auth.mjs @@ -0,0 +1,46 @@ +#!/usr/bin/env node +// Establish that npm will accept us for EVERY package before publishing any. +// +// Usage: node scripts/preflight-npm-auth.mjs +// +// The publish loop goes library, library, CLI. Without this, a credential that +// works for two of the three publishes two of the three -- and a version +// number npm has seen can never be reused, so the release cannot simply be +// retried at the same version. Trusted publishing is configured per package on +// npmjs.com, one page at a time, which is exactly the sort of thing that is +// complete for two packages and not the third. +// +// A token in NPM_TOKEN covers whatever it was granted, and asking the registry +// to confirm that would mean a write; the bootstrap path is checked by using +// it. This exists for the OIDC path, where the answer is knowable up front. +import { PACKAGES, fail } from './lib/changelog.mjs'; +import { canExchange, tokenForPackage } from './lib/npmOidc.mjs'; + +const SCOPE = 'preflight-npm-auth'; + +if (process.env.NPM_TOKEN !== undefined && process.env.NPM_TOKEN !== '') { + console.log(`${SCOPE}: NPM_TOKEN is set; nothing to exchange.`); + process.exit(0); +} + +if (!canExchange()) { + fail(SCOPE, 'no NPM_TOKEN and no OIDC identity -- npm would refuse every publish.'); +} + +const missing = []; +for (const pkg of PACKAGES) { + const token = await tokenForPackage(pkg); + console.log(` ${pkg}: ${token === null ? 'NO CREDENTIAL' : 'ok'}`); + if (token === null) missing.push(pkg); +} + +if (missing.length > 0) { + fail( + SCOPE, + `npm would refuse to publish ${missing.join(', ')}. Attach the trusted publisher on ` + + 'each package page (organization lorem-dev, repository ailoud, workflow publish.yml, ' + + 'environment empty). Nothing has been published, so no version number is spent.', + ); +} + +console.log(`${SCOPE}: every package will accept this run.`); diff --git a/scripts/release-notes.mjs b/scripts/release-notes.mjs index 6502df3..82f883d 100755 --- a/scripts/release-notes.mjs +++ b/scripts/release-notes.mjs @@ -8,61 +8,51 @@ // The heading format is the contract between three things: `bump-version.mjs` // writes `## Version `, this reads it, and CHANGES.md documents it. Change // one and the release stops producing notes. -import { readFileSync, writeFileSync } from 'node:fs'; -import { dirname, join } from 'node:path'; -import { fileURLToPath } from 'node:url'; - -const root = join(dirname(fileURLToPath(import.meta.url)), '..'); - -/** Fails loudly rather than writing empty notes, which nobody would notice. */ -function fail(message) { - console.error(`release-notes: ${message}`); - process.exit(1); -} +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { + HARD_LIMIT, + ROOT, + SOFT_LIMIT, + countBullets, + fail, + readChanges, + splitSections, + versionFromTag, + versionHeading, + warn, +} from './lib/changelog.mjs'; + +const SCOPE = 'release-notes'; const rawTag = process.argv[2] ?? process.env.GITHUB_REF_NAME; -if (!rawTag) fail('no tag given (pass one, or set $GITHUB_REF_NAME)'); -const version = rawTag.replace(/^v/, ''); - -const lines = readFileSync(join(root, 'CHANGES.md'), 'utf8').split('\n'); +if (!rawTag) fail(SCOPE, 'no tag given (pass one, or set $GITHUB_REF_NAME)'); +const version = versionFromTag(rawTag); -// Tolerates a trailing ` -- ` after the version. -const escaped = version.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); -const heading = new RegExp(`^## Version ${escaped}(\\s|$)`); - -const start = lines.findIndex((line) => heading.test(line)); -if (start === -1) fail(`no "## Version ${version}" section in CHANGES.md`); - -let end = lines.length; -for (let at = start + 1; at < lines.length; at += 1) { - if (lines[at].startsWith('## ')) { - end = at; - break; - } -} +const { sections } = splitSections(readChanges()); +const own = sections.find((section) => versionHeading(version).test(section.heading)); +if (own === undefined) fail(SCOPE, `no "## Version ${version}" section in CHANGES.md`); -const body = lines - .slice(start + 1, end) - .join('\n') - .trim(); -if (body === '') fail(`the section for ${version} is empty`); +const body = own.body.trim(); +if (body === '') fail(SCOPE, `the section for ${version} is empty`); -// The limits CHANGES.md and AGENTS.md state, enforced here because this is the -// last point before the notes reach anyone. A release that quietly shipped 90 -// entries would have been reviewed by nobody. -const bullets = body.split('\n').filter((line) => /^\s*- /.test(line)).length; -if (bullets > 50) { +// The limits CHANGES.md and AGENTS.md state, enforced here because this is +// the last point before the notes reach anyone. A release that quietly +// shipped 90 entries would have been reviewed by nobody. +const bullets = countBullets(body); +if (bullets > HARD_LIMIT) { fail( - `the section for ${version} has ${bullets} entries; the hard limit is 50. ` + + SCOPE, + `the section for ${version} has ${bullets} entries; the hard limit is ${HARD_LIMIT}. ` + 'Merge related entries, or cut what does not affect a user.', ); } -if (bullets > 10) { - console.error( - `release-notes: warning: ${bullets} entries, over the soft limit of 10. ` + +if (bullets > SOFT_LIMIT) { + warn( + `${SCOPE}: ${bullets} entries, over the soft limit of ${SOFT_LIMIT}. ` + 'Worth a look for entries to merge before tagging.', ); } -writeFileSync(join(root, 'RELEASE_NOTES.md'), `${body}\n`); +writeFileSync(join(ROOT, 'RELEASE_NOTES.md'), `${body}\n`); console.log(body); diff --git a/scripts/release-notes.test.mjs b/scripts/release-notes.test.mjs new file mode 100644 index 0000000..3a60206 --- /dev/null +++ b/scripts/release-notes.test.mjs @@ -0,0 +1,33 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { changes, entries, makeSandbox, run, useSandboxes } from './testing/harness.mjs'; + +useSandboxes(); + +describe('release-notes', () => { + it('writes the section body to RELEASE_NOTES.md and prints it', () => { + const dir = makeSandbox(changes('## Development\n\n## Version 1.0.0\n\n### Added\n\n- One.\n')); + const result = run(dir, 'release-notes.mjs', ['v1.0.0']); + expect(result.code).toBe(0); + expect(result.stdout).toContain('- One.'); + expect(readFileSync(join(dir, 'RELEASE_NOTES.md'), 'utf8')).toContain('- One.'); + }); + + it('refuses an unknown version rather than writing empty notes', () => { + const dir = makeSandbox(changes('## Development\n\n## Version 1.0.0\n\n- One.\n')); + const result = run(dir, 'release-notes.mjs', ['v2.0.0']); + expect(result.code).toBe(1); + expect(existsSync(join(dir, 'RELEASE_NOTES.md'))).toBe(false); + }); + + it('refuses an empty section', () => { + const dir = makeSandbox(changes('## Development\n\n## Version 1.0.0\n')); + expect(run(dir, 'release-notes.mjs', ['1.0.0']).stderr).toMatch(/is empty/); + }); + + it('refuses past the hard limit', () => { + const dir = makeSandbox(changes(`## Development\n\n## Version 1.0.0\n\n${entries(51)}\n`)); + expect(run(dir, 'release-notes.mjs', ['1.0.0']).stderr).toMatch(/hard limit is 50/); + }); +}); diff --git a/scripts/retire-prereleases.mjs b/scripts/retire-prereleases.mjs new file mode 100644 index 0000000..8ccc519 --- /dev/null +++ b/scripts/retire-prereleases.mjs @@ -0,0 +1,142 @@ +#!/usr/bin/env node +// Retire the pre-releases of a version once its final release is out. +// +// Usage: node scripts/retire-prereleases.mjs [--yes] +// +// Run from a laptop, under `npm login`. NOT from CI, and not because nobody +// wired it up: trusted publishing authenticates `npm publish` and nothing +// else. Exchanging the OIDC identity for a token does work -- npm's own client +// does it -- but the token it returns cannot deprecate. Measured on the 1.0.0 +// release, where the first call answered +// E404 ... or you do not have permission +// and every call after it +// E401 ... token is invalid +// so the token is publish-scoped and spent. This file said so before the +// automation was attempted; the release settled it. +// +// Prints the plan and changes nothing without --yes. Two of the three actions +// cannot be undone, so consent is explicit here for the same reason it is in +// `setup` and `rm`. +// +// WHY DEPRECATE AND NOT UNPUBLISH +// +// npm allows unpublish only within 72 hours, a version number can never be +// reused afterwards, and anyone who pinned the version has their install +// broken. Deprecating leaves every existing install working and prints a +// notice on the next one, which is what "this is superseded" should mean. +// +// WHY ONLY SOME GIT TAGS ARE DELETED +// +// A published package's provenance names both the commit and the tag it was +// built from. Deleting a tag whose commit is reachable from main costs only the +// name: verification needs the commit, and main keeps it alive. Deleting a tag +// that holds the only reference to its commit lets the commit be collected, +// which costs the attestation its subject -- so those tags are reported and +// left alone. +import { spawnSync } from 'node:child_process'; +import { PACKAGES, fail, planRetirement, versionFromTag, warn } from './lib/changelog.mjs'; + +const SCOPE = 'retire-prereleases'; + +const version = versionFromTag(process.argv[2] ?? ''); +if (!/^\d+\.\d+\.\d+$/.test(version)) { + fail(SCOPE, `expected a released version like 1.0.0, got "${process.argv[2] ?? ''}"`); +} +const confirmed = process.argv.includes('--yes'); + +function git(args) { + const result = spawnSync('git', args, { encoding: 'utf8' }); + if (result.status !== 0) fail(SCOPE, `git ${args.join(' ')} failed: ${result.stderr?.trim()}`); + return result.stdout ?? ''; +} + +const tags = git(['tag', '--list', `v${version}-*`]) + .split('\n') + .filter(Boolean); +const onMain = (tag) => + spawnSync('git', ['merge-base', '--is-ancestor', tag, 'origin/main'], { encoding: 'utf8' }) + .status === 0; + +const { versions, deletable, kept } = planRetirement(version, tags, onMain); + +if (versions.length === 0) { + console.log(`${SCOPE}: no pre-release tags for ${version}; nothing to retire.`); + process.exit(0); +} + +console.log(`${SCOPE}: retiring ${versions.length} pre-release(s) of ${version}`); +for (const prerelease of versions) { + for (const pkg of PACKAGES) { + console.log(` deprecate ${pkg}@${prerelease}`); + } +} +console.log(` drop the "dev" dist-tag from ${PACKAGES.at(-1)}`); +for (const tag of deletable) console.log(` delete tag ${tag} (local and origin)`); +for (const tag of kept) { + warn( + `${SCOPE}: keeping ${tag} -- its commit is not reachable from origin/main, and deleting ` + + 'the tag could orphan the commit the published provenance attests.', + ); +} + +/** Runs npm under whatever credentials the machine already has. */ +function npm(args) { + return spawnSync('npm', args, { encoding: 'utf8', stdio: 'inherit' }); +} + +if (!confirmed) { + console.log(`${SCOPE}: nothing was changed. Re-run with --yes to carry this out.`); + process.exit(0); +} + +// Everything that did not happen. Collected rather than warned about and +// forgotten, because the tag deletion below is the irreversible half and only +// worth doing if the npm half actually took. +const problems = []; + +for (const pkg of PACKAGES) { + for (const prerelease of versions) { + const result = npm(['deprecate', `${pkg}@${prerelease}`, `superseded by ${version}`]); + if (result.status !== 0) problems.push(`could not deprecate ${pkg}@${prerelease}`); + } +} + +// The `dev` dist-tag still points at the last snapshot, so `npm install +// ailoud@dev` would hand out something older than `latest`. +const cli = PACKAGES.at(-1); +if (npm(['dist-tag', 'rm', cli, 'dev']).status !== 0) { + problems.push('could not drop the "dev" dist-tag'); +} + +if (problems.length > 0) { + for (const problem of problems) warn(`${SCOPE}: ${problem}`); + // Refusing here is the whole point. Deleting the tags anyway would leave the + // versions installable and undeprecated, `dev` pointing at a snapshot, and + // nothing left to name what was missed -- on a green release run, because + // warnings do not fail anything. + fail( + SCOPE, + `${problems.length} thing(s) above did not happen on npm, so the tags are left in place. ` + + 'Fix the cause and re-run; nothing here has to be undone first.', + ); +} + +// origin first, then locally. The remote is the copy others fetch, and the +// local one is what names the tag on a re-run: deleting locally first and +// failing to push left the tag on origin with nothing here to retry it by. +let undeleted = 0; +for (const tag of deletable) { + const pushed = spawnSync('git', ['push', 'origin', `:refs/tags/${tag}`], { encoding: 'utf8' }); + if (pushed.status !== 0) { + warn(`${SCOPE}: could not delete ${tag} on origin: ${pushed.stderr?.trim()}`); + undeleted += 1; + continue; + } + git(['tag', '-d', tag]); +} + +if (undeleted > 0) { + fail(SCOPE, `${undeleted} tag(s) are still on origin. The npm side is done; re-run to finish.`); +} + +console.log(`${SCOPE}: done.`); diff --git a/scripts/retire-prereleases.test.mjs b/scripts/retire-prereleases.test.mjs new file mode 100644 index 0000000..b3f8080 --- /dev/null +++ b/scripts/retire-prereleases.test.mjs @@ -0,0 +1,136 @@ +import { spawnSync } from 'node:child_process'; +import { chmodSync, mkdirSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { REPO, changes, makeSandbox, run, useSandboxes } from './testing/harness.mjs'; + +useSandboxes(); + +/** + * A throwaway repository with two pre-release tags: one reachable from + * origin/main, one only from a side branch. That is the distinction the + * script has to draw, and it cannot be drawn without a real repository. + */ +function makeTaggedSandbox() { + const dir = makeSandbox(changes('## Development\n')); + const git = (...args) => + spawnSync( + 'git', + [ + '-c', + 'user.email=t@example.com', + '-c', + 'user.name=Test', + '-c', + 'commit.gpgsign=false', + ...args, + ], + { cwd: dir, encoding: 'utf8' }, + ); + git('init', '-b', 'main'); + git('commit', '--allow-empty', '-m', 'released'); + git('tag', 'v1.0.0-dev.1'); + git('checkout', '-b', 'side'); + git('commit', '--allow-empty', '-m', 'abandoned'); + git('tag', 'v1.0.0-dev.2'); + git('checkout', 'main'); + // A real `origin`, because the script deletes tags there before deleting + // them locally -- a sandbox without one makes every push fail and says + // nothing about the logic being tested. + const remote = join(dir, 'origin.git'); + spawnSync('git', ['init', '--bare', remote], { encoding: 'utf8' }); + git('remote', 'add', 'origin', remote); + git('push', '--quiet', 'origin', 'main', '--tags'); + git('fetch', '--quiet', 'origin'); + return dir; +} + +/** + * A directory holding an `npm` that records its arguments and exits with + * `code`, for putting first on PATH. + * + * A test must never run the real `npm deprecate`: on a machine that happens to + * be logged in it would deprecate the project's actual published versions. + */ +function stubNpm(dir, code) { + const bin = join(dir, 'stub-bin'); + mkdirSync(bin, { recursive: true }); + const script = join(bin, 'npm'); + writeFileSync( + script, + `#!/bin/sh\necho "npm $@" >> "${join(dir, 'npm-calls.txt')}"\nexit ${code}\n`, + ); + chmodSync(script, 0o755); + return bin; +} + +describe('retire-prereleases', () => { + it('refuses anything that is not a released version', () => { + for (const arg of [[], ['1.0.0-dev.1'], ['nonsense']]) { + expect(run(REPO, 'retire-prereleases.mjs', arg).code).not.toBe(0); + } + }); + + it('does nothing for a version that never had a pre-release', () => { + const result = run(REPO, 'retire-prereleases.mjs', ['9.9.9']); + expect(result.code).toBe(0); + expect(result.stdout).toMatch(/nothing to retire/); + }); + + it('plans the deprecations and the deletions it can make safely', () => { + const dir = makeTaggedSandbox(); + const { code, stdout } = run(dir, 'retire-prereleases.mjs', ['1.0.0'], { cwd: dir }); + expect(code).toBe(0); + expect(stdout).toContain('deprecate ailoud@1.0.0-dev.1'); + expect(stdout).toContain('deprecate @ailoud/core@1.0.0-dev.2'); + expect(stdout).toContain('delete tag v1.0.0-dev.1'); + expect(stdout).toContain('drop the "dev" dist-tag'); + }); + + it('keeps a tag whose commit is not reachable from main', () => { + const dir = makeTaggedSandbox(); + const { stdout, stderr } = run(dir, 'retire-prereleases.mjs', ['1.0.0'], { cwd: dir }); + expect(stdout).not.toContain('delete tag v1.0.0-dev.2'); + expect(stderr).toMatch(/keeping v1\.0\.0-dev\.2/); + }); + + it('keeps the tags when the npm side fails', () => { + // The bug this covers: warnings do not fail anything, so a refused + // credential or a rejected deprecate used to leave the versions + // installable and undeprecated while the tags -- the irreversible half -- + // were deleted anyway, on a green run. + const dir = makeTaggedSandbox(); + const stubbedNpm = stubNpm(dir, 1); + const before = spawnSync('git', ['tag', '--list'], { cwd: dir, encoding: 'utf8' }).stdout; + const result = run(dir, 'retire-prereleases.mjs', ['1.0.0', '--yes'], { + cwd: dir, + env: { PATH: `${stubbedNpm}:${process.env.PATH ?? ''}` }, + }); + expect(result.code).not.toBe(0); + expect(`${result.stdout}${result.stderr}`).toMatch(/did not happen on npm/); + expect(spawnSync('git', ['tag', '--list'], { cwd: dir, encoding: 'utf8' }).stdout).toBe(before); + }); + + it('deletes the tags once every npm call has succeeded', () => { + const dir = makeTaggedSandbox(); + const stubbedNpm = stubNpm(dir, 0); + const result = run(dir, 'retire-prereleases.mjs', ['1.0.0', '--yes'], { + cwd: dir, + env: { PATH: `${stubbedNpm}:${process.env.PATH ?? ''}` }, + }); + expect(result.code).toBe(0); + // v1.0.0-dev.2 is the one whose commit is not on main, so it stays. + expect(spawnSync('git', ['tag', '--list'], { cwd: dir, encoding: 'utf8' }).stdout.trim()).toBe( + 'v1.0.0-dev.2', + ); + }); + + it('changes nothing at all without --yes', () => { + const dir = makeTaggedSandbox(); + const before = spawnSync('git', ['tag', '--list'], { cwd: dir, encoding: 'utf8' }).stdout; + const result = run(dir, 'retire-prereleases.mjs', ['1.0.0'], { cwd: dir }); + expect(result.stdout).toMatch(/Re-run with --yes/); + const after = spawnSync('git', ['tag', '--list'], { cwd: dir, encoding: 'utf8' }).stdout; + expect(after).toBe(before); + }); +}); diff --git a/scripts/testing/harness.mjs b/scripts/testing/harness.mjs new file mode 100644 index 0000000..29f32fc --- /dev/null +++ b/scripts/testing/harness.mjs @@ -0,0 +1,99 @@ +/** + * Shared harness for the release-script tests, one test module per script. + * + * Two of the scripts WRITE -- fold-prereleases rewrites CHANGES.md and + * release-notes creates RELEASE_NOTES.md -- so running them against this + * repository would damage the real changelog. Each script resolves the + * repository root from its own location, so copying `scripts/` into a + * throwaway directory beside a fixture CHANGES.md puts them somewhere they can + * do no harm. Every sandbox asserts it is under the temp directory before a + * script runs, and `useSandboxes` re-checks the real changelog after every + * single test rather than once at the end of one big file. + */ +import { spawnSync } from 'node:child_process'; +import { cpSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { afterEach, expect } from 'vitest'; + +export const REPO = join(dirname(fileURLToPath(import.meta.url)), '..', '..'); + +const made = new Set(); + +/** + * Register cleanup and the untouched-repository guard for a test module. + * + * Call once at the top of each test file. Tracking every sandbox in a set + * rather than one variable means a test that makes two of them still has both + * removed. + */ +export function useSandboxes() { + afterEach(() => { + for (const dir of made) rmSync(dir, { recursive: true, force: true }); + made.clear(); + expectRepoUntouched(); + }); +} + +/** A throwaway repository holding only the scripts and a CHANGES.md. */ +export function makeSandbox(changes) { + const sandbox = mkdtempSync(join(tmpdir(), 'ailoud-scripts-')); + // The guard that keeps a mistake here from touching the real file. + expect(sandbox.startsWith(tmpdir())).toBe(true); + expect(sandbox).not.toBe(REPO); + made.add(sandbox); + cpSync(join(REPO, 'scripts'), join(sandbox, 'scripts'), { recursive: true }); + writeFileSync(join(sandbox, 'CHANGES.md'), changes, 'utf8'); + return sandbox; +} + +/** + * spawnSync rather than execFileSync: the latter returns stdout only, and + * throws away stderr on success -- which is exactly where a warning goes. A + * soft-limit test could never have seen it. + * + * The environment is scrubbed of every GITHUB_ variable. Two of them change + * what the scripts do -- `GITHUB_REF_NAME` is the tag fallback, and + * `GITHUB_ACTIONS` moves warnings from stderr to a `::warning::` line on + * stdout -- so leaving them in place makes these tests pass on a laptop and + * fail on the runner, which is exactly how two of them first went red. + * Tests that want that behaviour ask for it through `env`. + */ +export function run(dir, script, args = [], { cwd = REPO, env = {} } = {}) { + const scrubbed = Object.fromEntries( + Object.entries(process.env).filter(([key]) => !key.startsWith('GITHUB_')), + ); + const result = spawnSync(process.execPath, [join(dir, 'scripts', script), ...args], { + encoding: 'utf8', + cwd, + env: { ...scrubbed, ...env }, + }); + return { + code: result.status ?? 1, + stdout: result.stdout ?? '', + stderr: result.stderr ?? '', + }; +} + +/** + * The bytes of the real changelog, read once when this module loads -- before + * any test has run a script. + */ +const REAL_CHANGES = readFileSync(join(REPO, 'CHANGES.md'), 'utf8'); + +/** + * The point of the sandbox. If either of these fails, a script resolved the + * wrong root and has been writing to the repository: `fold-prereleases` + * rewrites the changelog in place, and `release-notes` creates + * RELEASE_NOTES.md beside it. + */ +export function expectRepoUntouched() { + expect(readFileSync(join(REPO, 'CHANGES.md'), 'utf8')).toBe(REAL_CHANGES); + expect(existsSync(join(REPO, 'RELEASE_NOTES.md'))).toBe(false); +} + +export const changes = (body) => `# AILoud Changelog\n\n${body}`; + +export const entries = (count, prefix = 'Entry') => + Array.from({ length: count }, (_, i) => `- ${prefix} ${i + 1}.`).join('\n'); diff --git a/vitest.config.ts b/vitest.config.ts index fadbd32..0d0a799 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -16,7 +16,15 @@ export default defineConfig({ // Jest's e2e config only ever matches e2e/tests/**/*.spec.ts, a // different directory and suffix, so the two runners never collect // each other's files. - include: ['packages/*/src/**/*.test.ts', 'apps/*/src/**/*.test.ts', 'e2e/src/**/*.test.ts'], + include: [ + 'packages/*/src/**/*.test.ts', + 'apps/*/src/**/*.test.ts', + 'e2e/src/**/*.test.ts', + // The release scripts are plain .mjs, so their tests are too -- no TS + // project covers scripts/, and adding one for four files would be more + // configuration than the files are. + 'scripts/**/*.test.mjs', + ], environment: 'node', passWithNoTests: true, coverage: { From cc1c917b4738e0f63225bc2bec87b12df9b456e3 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 8 Sep 2026 18:01:00 +0200 Subject: [PATCH 2/6] chore: back-merge main into develop (#7) * ci: back-merge without waiting for a person to merge a PR The old workflow opened a pull request and left the merge to a human. The first one was merged with "squash", which copied main's content into develop as one ordinary commit and dropped its history. Git then had no way to know develop already held those changes, so every later back-merge tried to re-apply 236 commits whose content was already there: 67 conflicting files on a merge that should have been a formality. A back-merge carries no decision, so it no longer waits for one. The workflow fast-forwards develop when develop has nothing of its own, and otherwise makes the merge commit itself and pushes it. A pull request is still opened, but only when the work genuinely cannot be done unattended -- a real conflict, or a branch rule refusing the push -- and its body now says in capitals not to squash it. develop itself was repaired separately: main merged in with its content taken throughout, verified byte-identical to main's tree, which lost nothing because develop carried no work of its own. * ci: let the back-merge pull request merge itself The previous commit tried to have the workflow push the merge to develop. That cannot work: `develop` is protected and refuses pushes outright -- "Changes must be made through a pull request". The attempt is on record in the run it declined. So the pull request stays; the person goes. The workflow now turns on auto-merge with the MERGE method, and GitHub merges the PR once its required checks pass. Nobody picks a method, so the squash that broke this once cannot happen by accident. Where auto-merge is unavailable the run warns and the PR body says, in capitals, not to squash it. This needed `allow_auto_merge` on the repository, which was off. Permissions drop to `contents: read`, since nothing pushes any more. * ci: open dependabot pull requests against develop A dependency bump is ordinary work, and ordinary work enters through `develop` per Branching in CONTRIBUTING.md. Opening them against `main` put them straight onto the release branch. Both ecosystems move, npm and github-actions. The one already open against `main` stays there; Dependabot does not retarget an existing pull request. --------- Co-authored-by: Lorem Dev --- .github/dependabot.yml | 7 ++++ .github/workflows/backmerge.yml | 69 +++++++++++++++++++++------------ 2 files changed, 52 insertions(+), 24 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 480e7d6..e3b20e6 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -9,6 +9,11 @@ # want: a known advisory beats an unaudited release, which is the same # exception `scripts/dependency-age-exceptions.json` records for a human. # +# Both ecosystems target `develop`, not `main`. A dependency bump is ordinary +# work, and ordinary work enters through `develop` -- see Branching in +# CONTRIBUTING.md. Opening these against `main` put them on the release branch +# and left the back-merge to carry them the wrong way round. +# # See .agents/skills/check-dependencies/SKILL.md for the reasoning, and # AGENTS.md for how a dependency change reaches a release. @@ -18,6 +23,7 @@ updates: - package-ecosystem: npm # The workspace root: one pnpm-lock.yaml covers all four manifests. directory: / + target-branch: develop schedule: interval: weekly day: monday @@ -46,6 +52,7 @@ updates: - package-ecosystem: github-actions directory: / + target-branch: develop schedule: interval: weekly day: monday diff --git a/.github/workflows/backmerge.yml b/.github/workflows/backmerge.yml index e47c8f7..69f1980 100644 --- a/.github/workflows/backmerge.yml +++ b/.github/workflows/backmerge.yml @@ -1,8 +1,19 @@ -# After anything lands on main, keep develop in sync by opening (or refreshing) -# a pull request that merges main back into develop. +# After anything lands on main, bring develop up to it. # -# A PR rather than a direct push, so the merge is visible and CI runs on it. -# Conflicts are resolved on the PR. +# A pull request that merges ITSELF. `develop` is protected -- pushes to it are +# refused with "Changes must be made through a pull request" -- so the PR is not +# optional. What is optional is the person, and the person was the problem: the +# first back-merge PR was merged with "squash", which copied main's content into +# develop as one ordinary commit and dropped its history. Git then had no way to +# know develop already held those changes, so every later back-merge tried to +# re-apply 236 commits whose content was already there: 67 conflicting files on +# a merge that should have been a formality. +# +# So the workflow opens the PR and turns on auto-merge with the MERGE method. +# GitHub merges it once the required checks pass, nobody chooses a method, and +# a squash cannot happen by accident. Auto-merge has to be enabled on the +# repository for this (`allow_auto_merge`); if it is off, the run says so and +# leaves the PR for a person, who must merge it with a merge commit. # # While the project is pre-release the flow is inverted from the eventual one: # work happens on main, and develop follows it. Once releases start, features @@ -16,6 +27,8 @@ on: branches: [main] permissions: + # Read is enough: nothing here pushes. The merge is GitHub'''s, done on the + # pull request once its checks pass. contents: read pull-requests: write @@ -24,60 +37,68 @@ concurrency: cancel-in-progress: false jobs: - open-backmerge-pr: + back-merge: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 with: fetch-depth: 0 - - name: Skip if develop already contains main + - name: Decide what is needed id: check run: | set -uo pipefail # Said plainly rather than dying on `fatal: Not a valid object name`, # which is what the first run did before develop existed. if ! git rev-parse --verify --quiet origin/develop >/dev/null; then - echo "needs_pr=false" >> "$GITHUB_OUTPUT" + echo "action=none" >> "$GITHUB_OUTPUT" echo "::notice::there is no develop branch, so there is nothing to back-merge into." exit 0 fi if git merge-base --is-ancestor origin/main origin/develop; then - echo "needs_pr=false" >> "$GITHUB_OUTPUT" + echo "action=none" >> "$GITHUB_OUTPUT" echo "develop already contains main; nothing to do." else - echo "needs_pr=true" >> "$GITHUB_OUTPUT" + echo "action=pr" >> "$GITHUB_OUTPUT" fi - - name: Open or update back-merge PR - if: steps.check.outputs.needs_pr == 'true' + - name: Open the back-merge PR and let it merge itself + if: steps.check.outputs.action == 'pr' env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} WORKFLOW_NAME: ${{ github.workflow }} RUN_ID: ${{ github.run_id }} run: | + set -uo pipefail MAIN_SHA="$(git rev-parse --short origin/main)" MAIN_SUBJECT="$(git log -1 --format='%s' origin/main)" - EXISTING="$(gh pr list --head main --base develop --state open --json number --jq '.[0].number')" - if [ -n "$EXISTING" ]; then - echo "PR #$EXISTING already open for main -> develop; it will auto-update with the new push." - exit 0 - fi - + NUMBER="$(gh pr list --head main --base develop --state open --json number --jq '.[0].number')" BODY_FILE="$(mktemp)" cat > "$BODY_FILE" < Date: Tue, 8 Sep 2026 13:46:32 +0000 Subject: [PATCH 3/6] build(deps-dev): bump the dev-toolchain group across 1 directory with 3 updates Bumps the dev-toolchain group with 3 updates in the / directory: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node), [eslint](https://github.com/eslint/eslint) and [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint). Updates `@types/node` from 26.2.0 to 26.3.0 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Updates `eslint` from 10.9.0 to 10.9.1 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/compare/v10.9.0...v10.9.1) Updates `typescript-eslint` from 8.67.0 to 8.68.0 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.68.0/packages/typescript-eslint) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 26.3.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: dev-toolchain - dependency-name: eslint dependency-version: 10.9.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: dev-toolchain - dependency-name: typescript-eslint dependency-version: 8.68.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: dev-toolchain ... Signed-off-by: dependabot[bot] --- package.json | 6 +- pnpm-lock.yaml | 322 +++++++++++++++++++++++++------------------------ 2 files changed, 167 insertions(+), 161 deletions(-) diff --git a/package.json b/package.json index 6ffeace..5f84463 100644 --- a/package.json +++ b/package.json @@ -31,9 +31,9 @@ "devDependencies": { "@eslint/js": "10.0.1", "@types/jest": "30.0.0", - "@types/node": "26.2.0", + "@types/node": "26.3.0", "@vitest/coverage-v8": "4.1.11", - "eslint": "10.9.0", + "eslint": "10.9.1", "eslint-config-prettier": "10.1.8", "eslint-import-resolver-typescript": "4.4.5", "eslint-plugin-boundaries": "7.2.0", @@ -41,7 +41,7 @@ "prettier": "3.9.6", "ts-jest": "29.4.12", "typescript": "6.0.3", - "typescript-eslint": "8.67.0", + "typescript-eslint": "8.68.0", "vitest": "4.1.11" } } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 98e8222..6ebde28 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -10,58 +10,58 @@ importers: devDependencies: '@eslint/js': specifier: 10.0.1 - version: 10.0.1(eslint@10.9.0) + version: 10.0.1(eslint@10.9.1) '@types/jest': specifier: 30.0.0 version: 30.0.0 '@types/node': - specifier: 26.2.0 - version: 26.2.0 + specifier: 26.3.0 + version: 26.3.0 '@vitest/coverage-v8': specifier: 4.1.11 version: 4.1.11(vitest@4.1.11) eslint: - specifier: 10.9.0 - version: 10.9.0 + specifier: 10.9.1 + version: 10.9.1 eslint-config-prettier: specifier: 10.1.8 - version: 10.1.8(eslint@10.9.0) + version: 10.1.8(eslint@10.9.1) eslint-import-resolver-typescript: specifier: 4.4.5 - version: 4.4.5(eslint@10.9.0) + version: 4.4.5(eslint@10.9.1) eslint-plugin-boundaries: specifier: 7.2.0 - version: 7.2.0(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0) + version: 7.2.0(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1) jest: specifier: 30.4.2 - version: 30.4.2(@types/node@26.2.0) + version: 30.4.2(@types/node@26.3.0) prettier: specifier: 3.9.6 version: 3.9.6 ts-jest: specifier: 29.4.12 - version: 29.4.12(@babel/core@7.29.7)(@jest/transform@30.4.1)(@jest/types@30.4.1)(babel-jest@30.4.1(@babel/core@7.29.7))(jest-util@30.4.1)(jest@30.4.2(@types/node@26.2.0))(typescript@6.0.3) + version: 29.4.12(@babel/core@7.29.7)(@jest/transform@30.4.1)(@jest/types@30.4.1)(babel-jest@30.4.1(@babel/core@7.29.7))(jest-util@30.4.1)(jest@30.4.2(@types/node@26.3.0))(typescript@6.0.3) typescript: specifier: 6.0.3 version: 6.0.3 typescript-eslint: - specifier: 8.67.0 - version: 8.67.0(eslint@10.9.0)(typescript@6.0.3) + specifier: 8.68.0 + version: 8.68.0(eslint@10.9.1)(typescript@6.0.3) vitest: specifier: 4.1.11 - version: 4.1.11(@types/node@26.2.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0)) + version: 4.1.11(@types/node@26.3.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0)) apps/cli: dependencies: - '@clack/prompts': - specifier: 1.7.0 - version: 1.7.0 '@ailoud/core': specifier: workspace:* version: link:../../packages/core '@ailoud/providers': specifier: workspace:* version: link:../../packages/providers + '@clack/prompts': + specifier: 1.7.0 + version: 1.7.0 '@modelcontextprotocol/sdk': specifier: 1.30.0 version: 1.30.0(zod@4.4.3) @@ -644,8 +644,8 @@ packages: '@types/json-schema@7.0.15': resolution: {integrity: sha512-5+fP8P8MFNC+AyZCDxrB2pkZFPGzqQWUzpSeuuVLvm8VMcorNYavBqoFcxK8bQz4Qsbn4oUEEem4wDLfcysGHA==} - '@types/node@26.2.0': - resolution: {integrity: sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg==} + '@types/node@26.3.0': + resolution: {integrity: sha512-L3fgrnchriRC2ExBflb8j4uZZURHZfQsmQeyVzhjcHW4kkwVyo8/0h1B2MVzMTrYUJYu6G7EWs14hW/L9putqw==} '@types/stack-utils@2.0.3': resolution: {integrity: sha512-9aEbYZ3TbYMznPdcdr3SmIrLXwC/AKZXQeCf9Pgao5CKb8CyHuEX5jzWPTkvregvhRJHcpRO6BFoGW9ycaOkYw==} @@ -656,63 +656,63 @@ packages: '@types/yargs@17.0.35': resolution: {integrity: sha512-qUHkeCyQFxMXg79wQfTtfndEC+N9ZZg76HJftDJp+qH2tV7Gj4OJi7l+PiWwJ+pWtW8GwSmqsDj/oymhrTWXjg==} - '@typescript-eslint/eslint-plugin@8.67.0': - resolution: {integrity: sha512-Un7Heoyj65NREbKAyIrFxeM143NZpExWmy1Nep4DLeQOeLlTeumPjoNKnBrU5D5moWXbPJgRa5Uwcdu0faVNGQ==} + '@typescript-eslint/eslint-plugin@8.68.0': + resolution: {integrity: sha512-WASHDpCm6qO5jj9g1a+8NiW5+GCkAyLReR56/4VruYmNgfUmqpxOfZ2Yfb8xGfJPWv5Qi6LSD8sXdces3vbp/Q==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: - '@typescript-eslint/parser': ^8.67.0 + '@typescript-eslint/parser': ^8.68.0 eslint: ^8.57.0 || ^9.0.0 || ^10.0.0 typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/parser@8.67.0': - resolution: {integrity: sha512-fUBfTuuEulWqX6V8+O3PtScV01tzYYRUDTAirHFKoRAt7nOzoGiPt0M/bB47wWNy0coOOcgEwAMUtBpykMxl6w==} + '@typescript-eslint/parser@8.68.0': + resolution: {integrity: sha512-fHq2VC1kpyYfvEcbiMjOpySY4WS7voEp89yAThrHRX5sm9j2lzYppCb2umFMEed4fWcyeLjHxrz0mpjNBaBxMQ==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: eslint: ^8.57.0 || ^9.0.0 || ^10.0.0 typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/project-service@8.67.0': - resolution: {integrity: sha512-cvE8c7ulYeXN9fYuszhCeCsbzyVEXuhrRCybnBre7TUmqb5nRmBfQAwCj0O3WJFDeyAZt4VYv51vMCC9LHSdYw==} + '@typescript-eslint/project-service@8.68.0': + resolution: {integrity: sha512-5GQtWZCXFcFYux955pvoS02WLc49pXNlvIxocKjS0clvwo3in1RdlzVKyiqQH9vE5AKWFLTaUgeQkOrTS+0Qxw==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/scope-manager@8.67.0': - resolution: {integrity: sha512-EgvsleTwS4E+WzzSvem8fAUubLwatMNF1B5hHSLQxcvs7q2dtRhGyujHwLJSYlG41niJ7GP24Aha2+0mb1b2kg==} + '@typescript-eslint/scope-manager@8.68.0': + resolution: {integrity: sha512-T5eXpcaJNg8bhjHJ8Rjp68Vq/QBteYtTKY8TZqVNPaUbuz0f6jI9t6aDkylwvalpAB9XTTFeFOjrjXAZ3YvmVA==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} - '@typescript-eslint/tsconfig-utils@8.67.0': - resolution: {integrity: sha512-vV+LUSv5njUWsknE71fqKTlXUva+R76SaeORd6Zojcunk/6DvKFXONU3BrAs2H49mbygUXt6gbYunzwqNwlhdg==} + '@typescript-eslint/tsconfig-utils@8.68.0': + resolution: {integrity: sha512-F7zrGQfiJHojPwi8vhxZQC1tWtJzvL74cK/nqri2lk8YUXvYaYwl263xOJ69jDWPUk1hmcdoayFwk9lX09npVw==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/type-utils@8.67.0': - resolution: {integrity: sha512-aVWDXbRmdXO9siTfX4ditQI1T9+zVcNazT48EJCD0v40/9RIFoUgZ05CmGEq9H2gixRpjUn/iplwvlcvutJW/Q==} + '@typescript-eslint/type-utils@8.68.0': + resolution: {integrity: sha512-X77zqoY1EjeWGs/0JNxeaMfp5C5lIz4Tw8y66F1Ne8Faq6g424sBNYM6xBAqElfGZPLpWS+CZAp0DXyKDzWiHg==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: eslint: ^8.57.0 || ^9.0.0 || ^10.0.0 typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/types@8.67.0': - resolution: {integrity: sha512-sBtgslww8nsMYUjhdPBiSyUqSzT8uR6g93A2QXnQC8+cGdjz0CyaOdqHDRJb1AtORbZCNUJBBeFA/tNR2uQmww==} + '@typescript-eslint/types@8.68.0': + resolution: {integrity: sha512-9RnpsGJjrAllCMefGVVsImJM24YurhC0Q1h4UbvivtvOqXmR/vEJge2OoE++z9m6hyg8T1Q8t5SNT6tHSbrxcg==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} - '@typescript-eslint/typescript-estree@8.67.0': - resolution: {integrity: sha512-EKQBCE9yNlRJYm7jdTW5AhDacDUmSwQb0FAJAmK2EKYrNXIsa2vxcSZx6PvJ/dEdI6lS+Y9W+EXckLj0iPFGcw==} + '@typescript-eslint/typescript-estree@8.68.0': + resolution: {integrity: sha512-OKKsD0tYmoNiU5PW2zehO1yO56jYOm1ShYlxon/Z0SJNidAkdVg86eg9ruRuoXf8xfnuWZGbwDsStkoXbZtIIA==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/utils@8.67.0': - resolution: {integrity: sha512-U9D1FdwEWBwok3hxxSdhclMb0twvt9QnjIQ0VfQ1AiX2epnpSgv2ubVDsayOFyY8K6FX+AQ7E0FKWVG3iKsj1A==} + '@typescript-eslint/utils@8.68.0': + resolution: {integrity: sha512-PB5gJMMOg0Q5P1tsgWtEAqQacJXq0qEqRHDX/YJ4FaTMLfZPpHB3gjl2EJuiZyPABxmj4ZQYiY9m1bdAJ5y7tQ==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: eslint: ^8.57.0 || ^9.0.0 || ^10.0.0 typescript: '>=4.8.4 <6.1.0' - '@typescript-eslint/visitor-keys@8.67.0': - resolution: {integrity: sha512-fkv8dHRDqfGtTHuJeebdrQ7cX6Ad4WAS00rgHh9UGvMycF1mjBfsxry1XsLIFhWZ6Judlh6UdzK+TYlbpCXgnA==} + '@typescript-eslint/visitor-keys@8.68.0': + resolution: {integrity: sha512-YR65gGdGvTUAWLldC3xLOvOzamdGzB4A5/N8rehEaHs3Zvoe39BhgY+u0SPch1OvrVTfLcc55wsSgK2NcnTS/A==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} '@ungap/structured-clone@1.3.3': @@ -974,8 +974,8 @@ packages: resolution: {integrity: sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==} engines: {node: 18 || 20 || >=22} - baseline-browser-mapping@2.11.18: - resolution: {integrity: sha512-1iEmLEYSiE1SeBoAfPo/Mnx3PzfzHUkDK61ASkCpuk3YXugYLH5DYK1SzqV55F8FMI6s0F+/tCP7Polz1QRjxw==} + baseline-browser-mapping@2.11.19: + resolution: {integrity: sha512-Grytf1xOxOEMTGRwx6rLGKkTabd4vMg3VrKdj/7joCmV0qgh4QwMMO6xh34YEXQqirAuUdgQGa5orJQQ+69RBw==} engines: {node: '>=6.0.0'} hasBin: true @@ -1036,8 +1036,8 @@ packages: resolution: {integrity: sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==} engines: {node: '>=10'} - caniuse-lite@1.0.30001809: - resolution: {integrity: sha512-xxWVywk6a6Arlk+hymeycyn/VgqEfLDxupvhH/xiY5SJ/18kmi9o6MiO320DCUzypORHLtvh0I4i04tUhCNHNQ==} + caniuse-lite@1.0.30001810: + resolution: {integrity: sha512-TITQPUkaz+aVk5GL6NhOdwk1aEaNTSDPsGFWrTuhKGtjTF70jL/Oht2W4c6rXUe5fu7Ie19VIahAXHIIiWWNeg==} chai@6.2.2: resolution: {integrity: sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==} @@ -1172,8 +1172,8 @@ packages: ee-first@1.1.1: resolution: {integrity: sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==} - electron-to-chromium@1.5.412: - resolution: {integrity: sha512-z4rMe3esBzlzovKHj4gxJnsCGZRK5l4baUvm+gCGJBPE+gsyUMKsuU9tnEUtI1dOebXz1ytAPGjvXhmQ7rIPwA==} + electron-to-chromium@1.5.414: + resolution: {integrity: sha512-aYlviXiaXBbzvKgyALpcMmqa3Np3sDr0XnZbEG62n2UpZFbEcjQ4EEMOLGzVPhwVnwTz0lvKY+GcARbunuHekw==} emittery@0.13.1: resolution: {integrity: sha512-DeWwawk6r5yR9jFgnDKYt4sLS0LmHJJi3ZOnb5/JdbYwj3nW+FxQnHIjhBKz8YLC7oRNPVM9NQ47I3CVx34eqQ==} @@ -1292,8 +1292,8 @@ packages: resolution: {integrity: sha512-tD40eHxA35h0PEIZNeIjkHoDR4YjjJp34biM0mDvplBe//mB+IHCqHDGV7pxF+7MklTvighcCPPZC7ynWyjdTA==} engines: {node: ^20.19.0 || ^22.13.0 || >=24} - eslint@10.9.0: - resolution: {integrity: sha512-5KeEOJZBfEVA47boFiBsf+6MmmJpffM7qEBg4pLla2e4nlKgdKlqCW0oSLOGsT8Wl5uCGJptLV1bkaiShj90Gw==} + eslint@10.9.1: + resolution: {integrity: sha512-9VaAkDURekixUQJy0oJYl2DcN6oKMfxay7XzaGYAWQwsb6qfKf+x76R2k1L8kb1boc+FyCAaTA9GmiKaaiaF+A==} engines: {node: ^20.19.0 || ^22.13.0 || >=24} hasBin: true peerDependencies: @@ -2108,6 +2108,10 @@ packages: resolution: {integrity: sha512-RvwwcruNjI1ncT5xRakeyS9Lf8lcItv34KD+aif+VH9kduAyfYBipGh12274xtenIPZ119/R9BdTBa8gAwSh0A==} engines: {node: '>=12'} + picomatch@4.0.7: + resolution: {integrity: sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==} + engines: {node: '>=12'} + pirates@4.0.7: resolution: {integrity: sha512-TfySrs/5nm8fQJDcBDuUng3VOUKsd7S+zqvbOTiGXHfxX4wK31ard+hoNuvkicM/2YFzlpDgABOevKSsB4G/FA==} engines: {node: '>= 6'} @@ -2433,8 +2437,8 @@ packages: resolution: {integrity: sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==} engines: {node: '>= 18'} - typescript-eslint@8.67.0: - resolution: {integrity: sha512-S2udFs8tCKEKffuJ4TB1idGUZiXdCPGi3IPBGWXarbLQ5UPXORV8QEVzJ4gCRduURMb5EkpNCdjbk0eDIuI8Yg==} + typescript-eslint@8.68.0: + resolution: {integrity: sha512-MHy0Y0ynqeEbx/S45+i/bBssdy3X6KNBfmJAP35GrgtNxu2TQ5K5xsFDhAnmsq1jvpdoZOPG1LGtJo0HWqYCrQ==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} peerDependencies: eslint: ^8.57.0 || ^9.0.0 || ^10.0.0 @@ -2825,10 +2829,10 @@ snapshots: '@bcoe/v8-coverage@1.0.2': {} - '@boundaries/elements@3.1.1(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0)': + '@boundaries/elements@3.1.1(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1)': dependencies: eslint-import-resolver-node: 0.3.9 - eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0) + eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1) handlebars: 4.7.9 is-core-module: 2.16.1 micromatch: 4.0.8 @@ -2870,9 +2874,9 @@ snapshots: tslib: 2.8.1 optional: true - '@eslint-community/eslint-utils@4.10.1(eslint@10.9.0)': + '@eslint-community/eslint-utils@4.10.1(eslint@10.9.1)': dependencies: - eslint: 10.9.0 + eslint: 10.9.1 eslint-visitor-keys: 3.4.3 '@eslint-community/regexpp@4.12.2': {} @@ -2893,9 +2897,9 @@ snapshots: dependencies: '@types/json-schema': 7.0.15 - '@eslint/js@10.0.1(eslint@10.9.0)': + '@eslint/js@10.0.1(eslint@10.9.1)': optionalDependencies: - eslint: 10.9.0 + eslint: 10.9.1 '@eslint/object-schema@3.0.5': {} @@ -2946,7 +2950,7 @@ snapshots: '@jest/console@30.4.1': dependencies: '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 jest-message-util: 30.4.1 jest-util: 30.4.1 @@ -2960,7 +2964,7 @@ snapshots: '@jest/test-result': 30.4.1 '@jest/transform': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 ansi-escapes: 4.3.2 chalk: 4.1.2 ci-info: 4.4.0 @@ -2968,7 +2972,7 @@ snapshots: fast-json-stable-stringify: 2.1.0 graceful-fs: 4.2.11 jest-changed-files: 30.4.1 - jest-config: 30.4.2(@types/node@26.2.0) + jest-config: 30.4.2(@types/node@26.3.0) jest-haste-map: 30.4.1 jest-message-util: 30.4.1 jest-regex-util: 30.4.0 @@ -2994,7 +2998,7 @@ snapshots: dependencies: '@jest/fake-timers': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 jest-mock: 30.4.1 '@jest/expect-utils@30.4.1': @@ -3012,7 +3016,7 @@ snapshots: dependencies: '@jest/types': 30.4.1 '@sinonjs/fake-timers': 15.4.0 - '@types/node': 26.2.0 + '@types/node': 26.3.0 jest-message-util: 30.4.1 jest-mock: 30.4.1 jest-util: 30.4.1 @@ -3030,7 +3034,7 @@ snapshots: '@jest/pattern@30.4.0': dependencies: - '@types/node': 26.2.0 + '@types/node': 26.3.0 jest-regex-util: 30.4.0 '@jest/reporters@30.4.1': @@ -3041,7 +3045,7 @@ snapshots: '@jest/transform': 30.4.1 '@jest/types': 30.4.1 '@jridgewell/trace-mapping': 0.3.31 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 collect-v8-coverage: 1.0.3 exit-x: 0.2.2 @@ -3117,7 +3121,7 @@ snapshots: '@jest/schemas': 30.4.1 '@types/istanbul-lib-coverage': 2.0.6 '@types/istanbul-reports': 3.0.4 - '@types/node': 26.2.0 + '@types/node': 26.3.0 '@types/yargs': 17.0.35 chalk: 4.1.2 @@ -3289,7 +3293,7 @@ snapshots: '@types/json-schema@7.0.15': {} - '@types/node@26.2.0': + '@types/node@26.3.0': dependencies: undici-types: 8.3.0 @@ -3301,15 +3305,15 @@ snapshots: dependencies: '@types/yargs-parser': 21.0.3 - '@typescript-eslint/eslint-plugin@8.67.0(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint@10.9.0)(typescript@6.0.3)': + '@typescript-eslint/eslint-plugin@8.68.0(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint@10.9.1)(typescript@6.0.3)': dependencies: '@eslint-community/regexpp': 4.12.2 - '@typescript-eslint/parser': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - '@typescript-eslint/scope-manager': 8.67.0 - '@typescript-eslint/type-utils': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - '@typescript-eslint/utils': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - '@typescript-eslint/visitor-keys': 8.67.0 - eslint: 10.9.0 + '@typescript-eslint/parser': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + '@typescript-eslint/scope-manager': 8.68.0 + '@typescript-eslint/type-utils': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + '@typescript-eslint/utils': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + '@typescript-eslint/visitor-keys': 8.68.0 + eslint: 10.9.1 ignore: 7.0.6 natural-compare: 1.4.0 ts-api-utils: 2.5.0(typescript@6.0.3) @@ -3317,56 +3321,56 @@ snapshots: transitivePeerDependencies: - supports-color - '@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3)': + '@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3)': dependencies: - '@typescript-eslint/scope-manager': 8.67.0 - '@typescript-eslint/types': 8.67.0 - '@typescript-eslint/typescript-estree': 8.67.0(typescript@6.0.3) - '@typescript-eslint/visitor-keys': 8.67.0 + '@typescript-eslint/scope-manager': 8.68.0 + '@typescript-eslint/types': 8.68.0 + '@typescript-eslint/typescript-estree': 8.68.0(typescript@6.0.3) + '@typescript-eslint/visitor-keys': 8.68.0 debug: 4.4.3 - eslint: 10.9.0 + eslint: 10.9.1 typescript: 6.0.3 transitivePeerDependencies: - supports-color - '@typescript-eslint/project-service@8.67.0(typescript@6.0.3)': + '@typescript-eslint/project-service@8.68.0(typescript@6.0.3)': dependencies: - '@typescript-eslint/tsconfig-utils': 8.67.0(typescript@6.0.3) - '@typescript-eslint/types': 8.67.0 + '@typescript-eslint/tsconfig-utils': 8.68.0(typescript@6.0.3) + '@typescript-eslint/types': 8.68.0 debug: 4.4.3 typescript: 6.0.3 transitivePeerDependencies: - supports-color - '@typescript-eslint/scope-manager@8.67.0': + '@typescript-eslint/scope-manager@8.68.0': dependencies: - '@typescript-eslint/types': 8.67.0 - '@typescript-eslint/visitor-keys': 8.67.0 + '@typescript-eslint/types': 8.68.0 + '@typescript-eslint/visitor-keys': 8.68.0 - '@typescript-eslint/tsconfig-utils@8.67.0(typescript@6.0.3)': + '@typescript-eslint/tsconfig-utils@8.68.0(typescript@6.0.3)': dependencies: typescript: 6.0.3 - '@typescript-eslint/type-utils@8.67.0(eslint@10.9.0)(typescript@6.0.3)': + '@typescript-eslint/type-utils@8.68.0(eslint@10.9.1)(typescript@6.0.3)': dependencies: - '@typescript-eslint/types': 8.67.0 - '@typescript-eslint/typescript-estree': 8.67.0(typescript@6.0.3) - '@typescript-eslint/utils': 8.67.0(eslint@10.9.0)(typescript@6.0.3) + '@typescript-eslint/types': 8.68.0 + '@typescript-eslint/typescript-estree': 8.68.0(typescript@6.0.3) + '@typescript-eslint/utils': 8.68.0(eslint@10.9.1)(typescript@6.0.3) debug: 4.4.3 - eslint: 10.9.0 + eslint: 10.9.1 ts-api-utils: 2.5.0(typescript@6.0.3) typescript: 6.0.3 transitivePeerDependencies: - supports-color - '@typescript-eslint/types@8.67.0': {} + '@typescript-eslint/types@8.68.0': {} - '@typescript-eslint/typescript-estree@8.67.0(typescript@6.0.3)': + '@typescript-eslint/typescript-estree@8.68.0(typescript@6.0.3)': dependencies: - '@typescript-eslint/project-service': 8.67.0(typescript@6.0.3) - '@typescript-eslint/tsconfig-utils': 8.67.0(typescript@6.0.3) - '@typescript-eslint/types': 8.67.0 - '@typescript-eslint/visitor-keys': 8.67.0 + '@typescript-eslint/project-service': 8.68.0(typescript@6.0.3) + '@typescript-eslint/tsconfig-utils': 8.68.0(typescript@6.0.3) + '@typescript-eslint/types': 8.68.0 + '@typescript-eslint/visitor-keys': 8.68.0 debug: 4.4.3 minimatch: 10.2.6 semver: 7.8.5 @@ -3376,20 +3380,20 @@ snapshots: transitivePeerDependencies: - supports-color - '@typescript-eslint/utils@8.67.0(eslint@10.9.0)(typescript@6.0.3)': + '@typescript-eslint/utils@8.68.0(eslint@10.9.1)(typescript@6.0.3)': dependencies: - '@eslint-community/eslint-utils': 4.10.1(eslint@10.9.0) - '@typescript-eslint/scope-manager': 8.67.0 - '@typescript-eslint/types': 8.67.0 - '@typescript-eslint/typescript-estree': 8.67.0(typescript@6.0.3) - eslint: 10.9.0 + '@eslint-community/eslint-utils': 4.10.1(eslint@10.9.1) + '@typescript-eslint/scope-manager': 8.68.0 + '@typescript-eslint/types': 8.68.0 + '@typescript-eslint/typescript-estree': 8.68.0(typescript@6.0.3) + eslint: 10.9.1 typescript: 6.0.3 transitivePeerDependencies: - supports-color - '@typescript-eslint/visitor-keys@8.67.0': + '@typescript-eslint/visitor-keys@8.68.0': dependencies: - '@typescript-eslint/types': 8.67.0 + '@typescript-eslint/types': 8.68.0 eslint-visitor-keys: 5.0.1 '@ungap/structured-clone@1.3.3': {} @@ -3476,7 +3480,7 @@ snapshots: obug: 2.1.4 std-env: 4.2.0 tinyrainbow: 3.1.1 - vitest: 4.1.11(@types/node@26.2.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0)) + vitest: 4.1.11(@types/node@26.3.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0)) '@vitest/expect@4.1.11': dependencies: @@ -3487,13 +3491,13 @@ snapshots: chai: 6.2.2 tinyrainbow: 3.1.1 - '@vitest/mocker@4.1.11(vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0))': + '@vitest/mocker@4.1.11(vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0))': dependencies: '@vitest/spy': 4.1.11 estree-walker: 3.0.3 magic-string: 0.30.21 optionalDependencies: - vite: 8.2.2(@types/node@26.2.0)(yaml@2.9.0) + vite: 8.2.2(@types/node@26.3.0)(yaml@2.9.0) '@vitest/pretty-format@4.1.11': dependencies: @@ -3637,7 +3641,7 @@ snapshots: balanced-match@4.0.4: {} - baseline-browser-mapping@2.11.18: {} + baseline-browser-mapping@2.11.19: {} body-parser@2.3.0: dependencies: @@ -3672,9 +3676,9 @@ snapshots: browserslist@4.28.8: dependencies: - baseline-browser-mapping: 2.11.18 - caniuse-lite: 1.0.30001809 - electron-to-chromium: 1.5.412 + baseline-browser-mapping: 2.11.19 + caniuse-lite: 1.0.30001810 + electron-to-chromium: 1.5.414 node-releases: 2.0.53 update-browserslist-db: 1.3.1(browserslist@4.28.8) @@ -3706,7 +3710,7 @@ snapshots: camelcase@6.3.0: {} - caniuse-lite@1.0.30001809: {} + caniuse-lite@1.0.30001810: {} chai@6.2.2: {} @@ -3800,7 +3804,7 @@ snapshots: ee-first@1.1.1: {} - electron-to-chromium@1.5.412: {} + electron-to-chromium@1.5.414: {} emittery@0.13.1: {} @@ -3832,9 +3836,9 @@ snapshots: escape-string-regexp@4.0.0: {} - eslint-config-prettier@10.1.8(eslint@10.9.0): + eslint-config-prettier@10.1.8(eslint@10.9.1): dependencies: - eslint: 10.9.0 + eslint: 10.9.1 eslint-import-context@0.1.9(unrs-resolver@1.12.2): dependencies: @@ -3851,10 +3855,10 @@ snapshots: transitivePeerDependencies: - supports-color - eslint-import-resolver-typescript@4.4.5(eslint@10.9.0): + eslint-import-resolver-typescript@4.4.5(eslint@10.9.1): dependencies: debug: 4.4.3 - eslint: 10.9.0 + eslint: 10.9.1 eslint-import-context: 0.1.9(unrs-resolver@1.12.2) get-tsconfig: 4.14.3 is-bun-module: 2.0.0 @@ -3864,24 +3868,24 @@ snapshots: transitivePeerDependencies: - supports-color - eslint-module-utils@2.12.1(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0): + eslint-module-utils@2.12.1(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1): dependencies: debug: 3.2.7 optionalDependencies: - '@typescript-eslint/parser': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - eslint: 10.9.0 + '@typescript-eslint/parser': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + eslint: 10.9.1 eslint-import-resolver-node: 0.3.9 - eslint-import-resolver-typescript: 4.4.5(eslint@10.9.0) + eslint-import-resolver-typescript: 4.4.5(eslint@10.9.1) transitivePeerDependencies: - supports-color - eslint-plugin-boundaries@7.2.0(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0): + eslint-plugin-boundaries@7.2.0(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1): dependencies: - '@boundaries/elements': 3.1.1(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0) + '@boundaries/elements': 3.1.1(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1) chalk: 4.1.2 - eslint: 10.9.0 + eslint: 10.9.1 eslint-import-resolver-node: 0.3.9 - eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.0))(eslint@10.9.0) + eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@4.4.5(eslint@10.9.1))(eslint@10.9.1) handlebars: 4.7.9 micromatch: 4.0.8 transitivePeerDependencies: @@ -3901,9 +3905,9 @@ snapshots: eslint-visitor-keys@5.0.1: {} - eslint@10.9.0: + eslint@10.9.1: dependencies: - '@eslint-community/eslint-utils': 4.10.1(eslint@10.9.0) + '@eslint-community/eslint-utils': 4.10.1(eslint@10.9.1) '@eslint-community/regexpp': 4.12.2 '@eslint/config-array': 0.23.5 '@eslint/config-helpers': 0.7.0 @@ -4308,7 +4312,7 @@ snapshots: '@jest/expect': 30.4.1 '@jest/test-result': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 co: 4.6.0 dedent: 1.7.2 @@ -4328,7 +4332,7 @@ snapshots: - babel-plugin-macros - supports-color - jest-cli@30.4.2(@types/node@26.2.0): + jest-cli@30.4.2(@types/node@26.3.0): dependencies: '@jest/core': 30.4.2 '@jest/test-result': 30.4.1 @@ -4336,7 +4340,7 @@ snapshots: chalk: 4.1.2 exit-x: 0.2.2 import-local: 3.2.0 - jest-config: 30.4.2(@types/node@26.2.0) + jest-config: 30.4.2(@types/node@26.3.0) jest-util: 30.4.1 jest-validate: 30.4.1 yargs: 17.7.3 @@ -4347,7 +4351,7 @@ snapshots: - supports-color - ts-node - jest-config@30.4.2(@types/node@26.2.0): + jest-config@30.4.2(@types/node@26.3.0): dependencies: '@babel/core': 7.29.7 '@jest/get-type': 30.1.0 @@ -4373,7 +4377,7 @@ snapshots: slash: 3.0.0 strip-json-comments: 3.1.1 optionalDependencies: - '@types/node': 26.2.0 + '@types/node': 26.3.0 transitivePeerDependencies: - babel-plugin-macros - supports-color @@ -4402,7 +4406,7 @@ snapshots: '@jest/environment': 30.4.1 '@jest/fake-timers': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 jest-mock: 30.4.1 jest-util: 30.4.1 jest-validate: 30.4.1 @@ -4410,14 +4414,14 @@ snapshots: jest-haste-map@30.4.1: dependencies: '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 anymatch: 3.1.3 fb-watchman: 2.0.2 graceful-fs: 4.2.11 jest-regex-util: 30.4.0 jest-util: 30.4.1 jest-worker: 30.4.1 - picomatch: 4.0.5 + picomatch: 4.0.7 walker: 1.0.8 optionalDependencies: fsevents: 2.3.3 @@ -4450,7 +4454,7 @@ snapshots: jest-mock@30.4.1: dependencies: '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 jest-util: 30.4.1 jest-pnp-resolver@1.2.3(jest-resolve@30.4.1): @@ -4484,7 +4488,7 @@ snapshots: '@jest/test-result': 30.4.1 '@jest/transform': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 emittery: 0.13.1 exit-x: 0.2.2 @@ -4513,7 +4517,7 @@ snapshots: '@jest/test-result': 30.4.1 '@jest/transform': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 cjs-module-lexer: 2.2.1 collect-v8-coverage: 1.0.3 @@ -4560,11 +4564,11 @@ snapshots: jest-util@30.4.1: dependencies: '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 chalk: 4.1.2 ci-info: 4.4.0 graceful-fs: 4.2.11 - picomatch: 4.0.5 + picomatch: 4.0.7 jest-validate@30.4.1: dependencies: @@ -4579,7 +4583,7 @@ snapshots: dependencies: '@jest/test-result': 30.4.1 '@jest/types': 30.4.1 - '@types/node': 26.2.0 + '@types/node': 26.3.0 ansi-escapes: 4.3.2 chalk: 4.1.2 emittery: 0.13.1 @@ -4588,18 +4592,18 @@ snapshots: jest-worker@30.4.1: dependencies: - '@types/node': 26.2.0 + '@types/node': 26.3.0 '@ungap/structured-clone': 1.3.3 jest-util: 30.4.1 merge-stream: 2.0.0 supports-color: 8.1.1 - jest@30.4.2(@types/node@26.2.0): + jest@30.4.2(@types/node@26.3.0): dependencies: '@jest/core': 30.4.2 '@jest/types': 30.4.1 import-local: 3.2.0 - jest-cli: 30.4.2(@types/node@26.2.0) + jest-cli: 30.4.2(@types/node@26.3.0) transitivePeerDependencies: - '@types/node' - babel-plugin-macros @@ -4872,6 +4876,8 @@ snapshots: picomatch@4.0.5: {} + picomatch@4.0.7: {} + pirates@4.0.7: {} pkce-challenge@5.0.1: {} @@ -5153,12 +5159,12 @@ snapshots: dependencies: typescript: 6.0.3 - ts-jest@29.4.12(@babel/core@7.29.7)(@jest/transform@30.4.1)(@jest/types@30.4.1)(babel-jest@30.4.1(@babel/core@7.29.7))(jest-util@30.4.1)(jest@30.4.2(@types/node@26.2.0))(typescript@6.0.3): + ts-jest@29.4.12(@babel/core@7.29.7)(@jest/transform@30.4.1)(@jest/types@30.4.1)(babel-jest@30.4.1(@babel/core@7.29.7))(jest-util@30.4.1)(jest@30.4.2(@types/node@26.3.0))(typescript@6.0.3): dependencies: bs-logger: 0.2.6 fast-json-stable-stringify: 2.1.0 handlebars: 4.7.9 - jest: 30.4.2(@types/node@26.2.0) + jest: 30.4.2(@types/node@26.3.0) json5: 2.2.3 lodash.memoize: 4.1.2 make-error: 1.3.6 @@ -5192,13 +5198,13 @@ snapshots: media-typer: 1.1.1 mime-types: 3.0.2 - typescript-eslint@8.67.0(eslint@10.9.0)(typescript@6.0.3): + typescript-eslint@8.68.0(eslint@10.9.1)(typescript@6.0.3): dependencies: - '@typescript-eslint/eslint-plugin': 8.67.0(@typescript-eslint/parser@8.67.0(eslint@10.9.0)(typescript@6.0.3))(eslint@10.9.0)(typescript@6.0.3) - '@typescript-eslint/parser': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - '@typescript-eslint/typescript-estree': 8.67.0(typescript@6.0.3) - '@typescript-eslint/utils': 8.67.0(eslint@10.9.0)(typescript@6.0.3) - eslint: 10.9.0 + '@typescript-eslint/eslint-plugin': 8.68.0(@typescript-eslint/parser@8.68.0(eslint@10.9.1)(typescript@6.0.3))(eslint@10.9.1)(typescript@6.0.3) + '@typescript-eslint/parser': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + '@typescript-eslint/typescript-estree': 8.68.0(typescript@6.0.3) + '@typescript-eslint/utils': 8.68.0(eslint@10.9.1)(typescript@6.0.3) + eslint: 10.9.1 typescript: 6.0.3 transitivePeerDependencies: - supports-color @@ -5257,22 +5263,22 @@ snapshots: vary@1.1.2: {} - vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0): + vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0): dependencies: lightningcss: 1.33.0 - picomatch: 4.0.5 + picomatch: 4.0.7 postcss: 8.5.26 rolldown: 1.2.5 tinyglobby: 0.2.17 optionalDependencies: - '@types/node': 26.2.0 + '@types/node': 26.3.0 fsevents: 2.3.3 yaml: 2.9.0 - vitest@4.1.11(@types/node@26.2.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0)): + vitest@4.1.11(@types/node@26.3.0)(@vitest/coverage-v8@4.1.11)(vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.11 - '@vitest/mocker': 4.1.11(vite@8.2.2(@types/node@26.2.0)(yaml@2.9.0)) + '@vitest/mocker': 4.1.11(vite@8.2.2(@types/node@26.3.0)(yaml@2.9.0)) '@vitest/pretty-format': 4.1.11 '@vitest/runner': 4.1.11 '@vitest/snapshot': 4.1.11 @@ -5289,10 +5295,10 @@ snapshots: tinyexec: 1.3.0 tinyglobby: 0.2.17 tinyrainbow: 3.1.1 - vite: 8.2.2(@types/node@26.2.0)(yaml@2.9.0) + vite: 8.2.2(@types/node@26.3.0)(yaml@2.9.0) why-is-node-running: 2.3.0 optionalDependencies: - '@types/node': 26.2.0 + '@types/node': 26.3.0 '@vitest/coverage-v8': 4.1.11(vitest@4.1.11) transitivePeerDependencies: - msw From b231b4663ac85f749496c7c4615fe1f93a293f71 Mon Sep 17 00:00:00 2001 From: Lorem Dev Date: Tue, 8 Sep 2026 21:35:41 +0200 Subject: [PATCH 4/6] fix: stop the update notice announcing the running version Reported from a terminal: `self update` moved 1.1.0 to 1.2.0, and the line printed straight after it said "a newer version is available (1.2.0 -> 1.2.0)". The once-a-day cache is keyed on time alone, and a day is long enough to install the very version it names. The cache-hit path returned its stored target verbatim, so the answer written while 1.1.0 was running stayed authoritative for the rest of the day -- and nothing invalidates it, since an install can also happen through plain `npm i -g ailoud`, which leaves no trace here. A cached target is now put back through `chooseUpdateTarget` against the version running now. The same function as the fetch path on purpose: whether something counts as an upgrade is one policy, including what a pre-release may move to, and a second comparison here would be free to drift from it. It cannot throw outward either -- this runs outside the try that guards the fetch, and a notice must never be why a command fails. Verified against the built binary, not only in tests: with a cache naming 1.2.0 and 1.2.0 running, `ailoud --version` prints the version alone; with the cache naming 9.9.9 it still says so. Reverting the one-line change fails three of the four new cases. --- CHANGES.md | 6 +++ apps/cli/src/updateNotice.test.ts | 61 +++++++++++++++++++++++++++++++ apps/cli/src/updateNotice.ts | 31 +++++++++++++++- 3 files changed, 97 insertions(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index ea82669..e0f2f20 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -40,6 +40,12 @@ ## Development +### Fixed + +- The update notice no longer announces the version you are already running. + A cached answer is now re-checked against the running version, so `self +update` is not followed by "a newer version is available (1.2.0 -> 1.2.0)". + ## Version 1.2.0 ### Added diff --git a/apps/cli/src/updateNotice.test.ts b/apps/cli/src/updateNotice.test.ts index 165700a..8df8648 100644 --- a/apps/cli/src/updateNotice.test.ts +++ b/apps/cli/src/updateNotice.test.ts @@ -66,6 +66,67 @@ describe('startUpdateCheck', () => { expect(await notice.finish()).toBeNull(); }); + /** + * Reported from a real terminal: `self update` from 1.1.0 to 1.2.0, and the + * line right after it said "a newer version is available (1.2.0 -> 1.2.0)". + * The cache is keyed on nothing but time, so the answer written while 1.1.0 + * was running stayed authoritative for a day after the version it named + * became the version in use. + */ + describe('a cached answer this run has already caught up with', () => { + const cacheWith = (target: string): MemFs => + new MemFs({ + [updateCachePath(DATA_DIR)]: JSON.stringify({ + checkedAt: '2026-01-01T00:00:00.000Z', + target, + }), + }); + + it('says nothing when the cached target is the running version', async () => { + const deps = baseDeps({ + fs: cacheWith('1.2.0'), + currentVersion: '1.2.0', + published: hangingPublished(), + }); + + expect(await startUpdateCheck(deps).finish()).toBeNull(); + }); + + it('says nothing when the running version is newer than the cached target', async () => { + // A local build, or an install from a tarball ahead of the registry. + const deps = baseDeps({ + fs: cacheWith('1.2.0'), + currentVersion: '1.3.0', + published: hangingPublished(), + }); + + expect(await startUpdateCheck(deps).finish()).toBeNull(); + }); + + it('still speaks up when the cached target really is newer', async () => { + const deps = baseDeps({ + fs: cacheWith('1.3.0'), + currentVersion: '1.2.0', + published: hangingPublished(), + }); + + expect(await startUpdateCheck(deps).finish()).toBe('1.3.0'); + }); + + it('says nothing, rather than throwing, when it cannot read its own version', async () => { + // `chooseUpdateTarget` throws on a version it cannot parse, and this + // path runs outside the try that guards the fetch. A notice must never + // be the reason a command fails. + const deps = baseDeps({ + fs: cacheWith('1.3.0'), + currentVersion: 'not-a-version', + published: hangingPublished(), + }); + + expect(await startUpdateCheck(deps).finish()).toBeNull(); + }); + }); + it('prints from the cache when this run could not refresh it', async () => { const fs = new MemFs({ [updateCachePath(DATA_DIR)]: JSON.stringify({ diff --git a/apps/cli/src/updateNotice.ts b/apps/cli/src/updateNotice.ts index fcd4492..48e02ad 100644 --- a/apps/cli/src/updateNotice.ts +++ b/apps/cli/src/updateNotice.ts @@ -257,6 +257,35 @@ async function writeCache(deps: NoticeDeps, target: string | null): Promise 1.2.0)", because the entry written while 1.1.0 ran stayed + * authoritative afterwards. The same applies to any install that happens + * outside ailoud -- `npm i -g ailoud` leaves no trace here to invalidate. + * + * `chooseUpdateTarget` again rather than a fresh comparison: whether a + * version counts as an upgrade is one policy (it also decides what a + * pre-release may move to), and asking the same function twice is what keeps + * the answer from drifting from the one the fetch path gives. `deprecated` + * is false because a deprecated release was already excluded when this + * target was chosen; the cache stores only the string that survived. + * + * Throwing is not an option: this runs outside the try that guards the fetch, + * and a notice must never be why a command failed. + */ +function stillNewer(deps: NoticeDeps, target: string | null): string | null { + if (target === null) return null; + try { + return chooseUpdateTarget(deps.currentVersion, [{ version: target, deprecated: false }]); + } catch { + return null; + } +} + export function startUpdateCheck(deps: NoticeDeps): UpdateCheck { if (suppressed(deps)) return { finish: async () => null }; @@ -264,7 +293,7 @@ export function startUpdateCheck(deps: NoticeDeps): UpdateCheck { const inflight = (async (): Promise => { const cached = await readCache(deps); - if (cached !== null) return cached.target; + if (cached !== null) return stillNewer(deps, cached.target); try { const versions = await deps.published(controller.signal); const target = chooseUpdateTarget(deps.currentVersion, versions); From bcb487bf9283184a9afc5e2a4b06546a4e64a0ef Mon Sep 17 00:00:00 2001 From: Lorem Dev Date: Tue, 8 Sep 2026 21:52:50 +0200 Subject: [PATCH 5/6] docs: rewrap the update-notice changelog entry The continuation line had lost its indentation and split an inline code span across it. --- CHANGES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index e0f2f20..793c21c 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -42,9 +42,9 @@ ### Fixed -- The update notice no longer announces the version you are already running. - A cached answer is now re-checked against the running version, so `self -update` is not followed by "a newer version is available (1.2.0 -> 1.2.0)". +- The update notice no longer announces the version you are already running: + a cached answer is re-checked against the running version, so `self update` + is not followed by "a newer version is available (1.2.0 -> 1.2.0)". ## Version 1.2.0 From 49a3c36e1eaa362c303f46356ead6267a39d0700 Mon Sep 17 00:00:00 2001 From: Lorem Dev Date: Tue, 8 Sep 2026 21:53:23 +0200 Subject: [PATCH 6/6] chore: release 1.2.1 --- CHANGES.md | 2 ++ apps/cli/package.json | 2 +- package.json | 2 +- packages/core/package.json | 2 +- packages/providers/package.json | 2 +- 5 files changed, 6 insertions(+), 4 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 793c21c..de68e02 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -40,6 +40,8 @@ ## Development +## Version 1.2.1 + ### Fixed - The update notice no longer announces the version you are already running: diff --git a/apps/cli/package.json b/apps/cli/package.json index 2b1f12a..76f1920 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -1,6 +1,6 @@ { "name": "ailoud", - "version": "1.2.0", + "version": "1.2.1", "type": "module", "bin": { "ailoud": "./dist/bin/ailoud.js" diff --git a/package.json b/package.json index 5f84463..f1c14e9 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "ailoud-workspace", - "version": "1.2.0", + "version": "1.2.1", "private": true, "type": "module", "description": "Multilingual audio-to-text CLI with a recording library and LLM summaries", diff --git a/packages/core/package.json b/packages/core/package.json index 8f8319b..35b854f 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,6 +1,6 @@ { "name": "@ailoud/core", - "version": "1.2.0", + "version": "1.2.1", "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts", diff --git a/packages/providers/package.json b/packages/providers/package.json index 6d87362..4b9fff4 100644 --- a/packages/providers/package.json +++ b/packages/providers/package.json @@ -1,6 +1,6 @@ { "name": "@ailoud/providers", - "version": "1.2.0", + "version": "1.2.1", "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts",