Skip to content

perf(pr-merge): attribute enable-auto-merge's API calls per step - #11151

Merged
kyle-sexton merged 3 commits into
mainfrom
perf/eam-call-attribution
Oct 30, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
perf/eam-call-attribution

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: §9 attribution for enable-auto-merge GITHUB_TOKEN spend

Summary

Three enable-auto-merge jobs (runs 40983386272, 40986033944 and 40987581101) logged 337
github-token calls in their job-end api-calls notices. babysit_verify_evidence.py's
babysit-gh-calls lines account for 113 of them. Nothing logs where the other 224 (about 75 per
job) go. This PR makes each Node step in the job print its own share. It only measures: the job
sends the same requests, writes the same outputs and keeps the same holds.

Fix

  • .github/actions/log-api-budget/rate-guard.mjs: recordReading is the one place every Node
    client's reading lands (countRequests, every latch.guard). It now also keeps a per-process
    tally of the lines it appends, counted the way api_budget_emit_calls counts the file:

    • a non-304 line counts as a call for its token;
    • a 304 counts as not-modified;
    • a graphql resource counts as graphql.

    When API_BUDGET_SCRIPT names the step (lower-case kebab case), the process ends with one line,
    written at exit:
    ::notice title=api-calls::script=<name> github-token=N lanes-app=N not-modified=M graphql=K.
    Without that variable nothing prints, so no other workflow's output changes.

  • .github/workflows/pr-merge.yml: each of the enable-auto-merge job's eight steps that count
    calls (API_BUDGET_TOKEN) now sets API_BUDGET_SCRIPT. They are who, superseded,
    head-ref-map, listing-carry, carry-run-cache, verify-answers, lane-records and
    enable-auto-merge. The two carry-run-cache.mjs runs that make no GitHub call are left
    unnamed.

  • api-budget.bash, api_budget_emit_calls and the log-api-budget action are unchanged.
    pr-require-checks.yml embeds api-budget.bash verbatim, so it is left alone.

Metrics

The next measurement reads these lines from the enable-auto-merge job log:

  • ::notice title=api-calls::script=<name> github-token=N lanes-app=N not-modified=M graphql=K:
    one per named step. enable-auto-merge covers the arm step's own Node clients, meaning the PR
    listing, skip-state, detail and arm queries on the lane token, plus the verifier client on the
    job token.
  • babysit-gh-calls {...}: the Python child processes' calls. These are not in any script=
    line.
  • ::notice title=api-calls::github-token=N ... and ::notice title=api-calls::lanes-app=N ...:
    the job totals, unchanged.

Check: the script= lines' github-token values plus the babysit-gh-calls totals should add
up to the job-end github-token value. A job-total reader must match
api-calls::github-token=, not a bare github-token=, because the script= lines share the
api-calls title. No parser in this repository, or found by an org code search, reads these
notices.

Not counted anywhere, before or after this PR: rotation.mjs ghDownload (gh run download
through execFileSync). Its artifact downloads on the Actions token never reach the readings file.

Verification

  • node --test .github/actions/log-api-budget/*.test.mjs: 144 of 144 pass. Before: 142 tests on
    origin/main. The 2 new tests are in count-requests.test.mjs:
    • A child process sends two core reads, a 304 and a GraphQL query on the step's token, plus one
      read on a second token. It must print exactly
      script=who github-token=3 lanes-app=1 not-modified=1 graphql=1. The job-end lines over the
      same files must agree (github-token=3 not-modified=1 graphql=1, lanes-app=1).
    • The same calls with the variable unset, or set to Who, who::x or an empty string, print
      nothing.
  • node --test .github/scripts/pr-merge/*.test.mjs: 727 of 727 pass (726 before).
    • The existing superseded test runs the real who and superseded steps from the YAML. It now
      asserts script=who github-token=1 and script=superseded github-token=2 for
      enable-auto-merge, which add up to the job-end github-token=3 it already pins. It asserts no
      script= line for the merge job.
    • A new test checks that every counting step in enable-auto-merge sets a name, and that no two
      share one.
  • node --test .github/actions/check-trusted-trigger/*.test.mjs: 365 of 365 pass.
  • Mutation check: with scriptCalls.count commented out, both value tests fail. They pass again
    once it is restored.
  • actionlint .github/workflows/pr-merge.yml and zizmor --offline report no findings.
  • scripts/check-changelog-fragments.sh --check passes.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits October 29, 2026 23:29
The rate guard's recordReading also tallies, per process, the readings it
appends for the job-end api-calls notice, classified as
api_budget_emit_calls classifies the file. A process whose step sets
API_BUDGET_SCRIPT ends with
`::notice title=api-calls::script=<name> github-token=N lanes-app=N not-modified=M graphql=K`.
enable-auto-merge's eight counting steps name themselves. No request,
output file or hold changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 30, 2026 05:09
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T05:11:10.535521Z f12c428 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

claude-review needs no new review of 2ab68ed822151417392ff9a8381b4170f15b5260: no file of this pull request changed since the review of f12c428.

No findings. I read the diff, the PR context and the surrounding rate-guard.mjs, api-budget.bash and the enable-auto-merge job in pr-merge.yml. I did not run the tests. The passing counts in the PR description are the author's claims and I did not check them.

What I checked

  • Counting matches the job-end counter. createScriptCalls.count follows api_budget_emit_calls (.github/actions/log-api-budget/api-budget.bash:426-433): a non-304 line counts for its token, a 304 counts as not-modified, and a graphql resource counts as graphql whether or not the status was 304. The tally runs inside the try after the append, so it only counts lines that reached the file.
  • No duplicate or stray lines.
    • API_BUDGET_SCRIPT is validated against the same lower-case kebab pattern as the token, and the line is written once, at process exit.
    • The arm step's children are Python (enable-auto-merge.mjs:1129, :1196, :1217) and git. No child Node process inherits the variable and prints a second line under the same name.
    • Each named step runs a plain node ... command, so its stdout is not captured by $(...) or a redirect that the notice could pollute.
  • Override bucketing. For the arm step, budgetToken decides the bucket (lanes-app by default, github-token for the job-token reads), and recordReading receives that same token. The per-step tally therefore splits the same way as the job-end totals.
  • Tests. The two new tests in count-requests.test.mjs pin the exact line and its agreement with the job-end lines. They also pin that nothing prints for an unset, empty or malformed name. The workflow.test.mjs check requires every enable-auto-merge step with API_BUDGET_TOKEN to carry a unique name. I confirmed the YAML has that: all eight steps are named and the two unnamed carry-run-cache.mjs runs set no API_BUDGET_TOKEN.
  • Closing issues: none; the PR states "No related issue".
  • Manifests: no plugin.json or marketplace.json change, so no manifest lines apply.
  • Trusted paths: rate-guard.mjs and pr-merge.yml match the trusted-path list. I read both changes in full. They add counting and logging only. They change no request, output or hold, and no permission or token handling.

Spend: none. The change adds one log line per named step and makes no extra API calls.

Minor, not counted as a finding: the exit-time writeSync(1, ...) swallows an EAGAIN if stdout is a non-blocking pipe, so such a step would silently omit its line. That is consistent with the "never fail the job" intent.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review needs no new review of 2ab68ed822151417392ff9a8381b4170f15b5260: no file of this pull request changed since the review of f12c428.

No security issues found.

I read the full diff and the PR description, and checked the pr-merge.yml steps in context. The PR adds a per-process tally in rate-guard.mjs. When API_BUDGET_SCRIPT is set, the process prints one ::notice title=api-calls::script=<name> … line at exit. The workflow sets that variable as a constant on eight steps of the enable-auto-merge job.

  • Injection: the name is printed only if it matches ^[a-z][a-z0-9-]{0,39}$, so it can't carry :: or newlines into a workflow command. The values are workflow-YAML literals, and no PR, issue or comment text reaches them.
  • Stdout capture: the who, superseded, head-ref-map, listing-carry, carry-run-cache, verify-answers and lane-records steps are plain run: node … calls, with outputs written to files. The arm step's enable-auto-merge.mjs call isn't wrapped in a command substitution either. Nothing parses the new line as data or as GITHUB_OUTPUT.
  • Trust controls: the pr-merge.yml changes only add env entries. They leave the if: conditions, the lane-token request, the kill switch and every hold script's logic alone. The change reads no secrets and sends nothing out. rate-guard.mjs and pr-merge.yml match the trusted-path list. I read their whole changes and what they gate, and neither widens anything.
  • Manifests and instruction files: none changed.
  • Spend: none. This only adds log lines.

critical-findings: 0

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f12c42810b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 1ff2990 Oct 30, 2026
45 checks passed
@kyle-sexton
kyle-sexton deleted the perf/eam-call-attribution branch October 30, 2026 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant