fix(e2e-harnesses): loosen codex 401 assertion to drop third-party error body text - #329
Merged
Merged
Conversation
…ror body text
The outcomePattern for codex pinned OpenAI's full 401 body ("Missing
bearer or basic authentication in header") plus the cf-ray tail, both
of which we don't own and which OpenAI changed, breaking the assertion
on every CI run and blocking #324 (tracked in #327).
The regex now keeps only what's ours to assert: the "Reconnecting...
N/M" banner shape, "unexpected status 401 Unauthorized", and the
wss://api.openai.com/v1/responses URL.
Note the 401 path is unreachable on a machine with a real codex login
(an authenticated run gets a normal reply instead of the reconnect
banner), so this can only be proven end-to-end in CI, which runs
without credentials.
Fixes #328
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #328.
e2e/harnesses/tests/harnesses.spec.tsasserted OpenAI's full 401 error body. OpenAI reworded it, so the assertion broke on text we don't own — red one2e consumer apps (shard-3)every run, blocking release PR #324 (tracked in #327).Keeps what's ours to assert — the reconnect banner shape, the
401 Unauthorizedstatus, and the OpenAI responses host. Drops the body text and the trailingcf-ray. It has to match both wordings, since we don't control which one comes back:A finding worth recording
I ran
playwright test --project=codexlocally to get failing-first proof. It failed — but not on the 401:The dev machine has a real codex login (
codex login status→ "Logged in using ChatGPT", plusOPENAI_API_KEYin the env), so codex authenticates and returns a normal reply.This test can never pass locally on an authenticated machine, on either the old or the new pattern — it asserts an error banner, and a logged-in codex answers the prompt. It's implicitly CI-shaped, since CI has no codex credentials. Worth knowing before anyone else burns time trying to reproduce a harness failure locally.
Forcing an unauthenticated state with an empty
CODEX_HOMEproducedthread/resume failed: no rollout found for thread id …instead, never reaching the WebSocket handshake. That's the expected consequence of pointing the CLI at an empty home while a session references a rollout in the real one — classified as an artifact of the forced state, not filed as a bug.Gates
typecheck (
conciv-e2e-harnesses) pass;pnpm lint0 errors;fallow auditverdictpass, 0 introduced. The oneformat:checkfailure ispackages/ui-kit-chat/CHANGELOG.md, which fails onmaintoo — last touched by4fe834d0 chore: version packages (#317), unrelated to this diff.No changeset:
e2e/harnessesisprivate: true, not a published package.CI is the real gate here, which is fitting — CI is where it's currently red.
🤖 Generated with Claude Code