chore: benchmark the downgraders with CodSpeed - #47
Conversation
Mirror middleapi/standard-server: `pnpm bench` runs Vitest benchmarks through the CodSpeed plugin, and a CodSpeed workflow runs them in simulation mode on pushes to main and on pull requests, so a performance change shows up on the pull request that causes it. The benches in packages/downgrader/benches cover both spec and schema converters of each step, plus the chained 3.2 to 3.0 path, on: - the official corpus the tests validate against - a generated CRUD API at 10 and 100 resources that uses every feature each step rewrites (removed parts to inline, $defs, $id, webhooks, query operations, item schemas, ...) - the shared and cyclic reference graphs that once made conversion exponential, so a regression there costs orders of magnitude READMEs gain the CodSpeed badge, the bench command, and a short performance section for the downgrader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjCihASFt61Eqjq9jfAoE5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjCihASFt61Eqjq9jfAoE5
|
your Pullfrog Router balance is empty, and this repo has no provider key to fall back on, so the agent never ran. To fix, any one of: add a payment method or top up your Router balance · add a provider API key (GitHub Actions secret or Pullfrog secret) · switch this repo to a free model. Top up Router → · Model settings → · Setup docs → · Ask in Discord → |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
The failing Everything else in CI passes on this head: Generated by Claude Code |
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|

Mirrors the setup in middleapi/standard-server:
pnpm benchruns Vitest benchmarks through@codspeed/vitest-plugin, and a CodSpeed workflow runs them in simulation mode on pushes tomainand on pull requests, so a performance change shows up on the pull request that causes it.Benches
packages/downgrader/benchescovers the spec and schema converters of each step, plus the chained 3.2 → 3.1 → 3.0 path, on:components.pathItems,$defs, an$idresource, webhooks, mutual TLS, multipart bodies, and in 3.2queryoperations,itemSchema,components.mediaTypes, Tag and Server fields, anddataValueexamples. It builds a fresh tree with no shared objects, like a document parsed from JSON.$refchain. These are the inputs that earlier fixes (fix(downgrader): convert shared schemas once in dereferenced documents #23, fix(downgrader): convert each Path Item hop once when merging $ref chains #33) made linear, so a regression there costs orders of magnitude.$defsand a recursive tree, and a 64-level dereferenced diamond.The generated API, the diamonds, and the chain pass the official schemas both before and after conversion. The callback graphs are checked on output only, as the existing tests do.
Other changes
vitest.config.ts: adds the CodSpeed plugin, which only switches on in benchmark mode, and the benchmark include/exclude globs.tsconfig.jsons exclude**/*.bench.*from the build, like their tests. The roottscstill type-checks the benches.pnpm benchin the root Development section, and a short Performance section in the downgrader README.pnpm-lock.yaml: the CodSpeed plugin only supports vite up to 7, so vitest now resolves vite 7.3.6 instead of 8.2.2, as in standard-server.After merging
CodSpeed already picked up all 19 benchmarks on this branch. Pull requests will show performance changes once the workflow has run on
main, which happens when this merges.🤖 Generated with Claude Code
https://claude.ai/code/session_01LjCihASFt61Eqjq9jfAoE5