Skip to content

Fix deserialization of persisted byte maps - #264

Closed
kaiicheng wants to merge 1 commit into
cloudflare:mainfrom
kaiicheng:fix/restore-json-byte-maps
Closed

Fix deserialization of persisted byte maps#264
kaiicheng wants to merge 1 commit into
cloudflare:mainfrom
kaiicheng:fix/restore-json-byte-maps

Conversation

@kaiicheng

Copy link
Copy Markdown

Fixes a deserialization crash when a raw Uint8Array crosses a JSON persistence or transport boundary and is restored as a numeric-key object.

The decoder now accepts only ordinary or null-prototype objects whose keys are exactly contiguous "0" through "n" and whose values are integer bytes in 0..255. It continues to reject gaps, extra keys, fractions, strings, nulls, and out-of-range values. Existing Uint8Array, base64, and byte-container marker handling are unchanged.

This is a compatibility fix for consumers using jsonCompatibleWithBytes; it does not change the preferred transport behavior of preserving byte arrays or encoding them as base64.

Related: openai/codex#42760

Validation:

  • npx vitest run --project node __tests__/index.test.ts __tests__/limits.test.ts (222 passed)
  • npm exec --yes --package=node@24 -- npm run build

@changeset-bot

changeset-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 97b3615

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
capnweb Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kaiicheng

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Sep 8, 2026

@kentonv kentonv left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that you shouldn't be using jsonCompatibleWithBytes encoding level if your transport is not actually able to round-trip a Uint8Array. You should be using plain jsonCompatible. This is literally the only difference between them: jsonCompatible encodes bytes as base64 strings, which is really what you want in this case, as it will be much more efficient.

Can your use case switch to jsonCompatible mode instead? If not, can you elaborate on how you got into this situation, where you are using jsonCompatibleWithBytes but your transport doesn't actually support bytes?

@kaiicheng

Copy link
Copy Markdown
Author

Thanks for the clarification. I agree that reconstructing numeric-key objects in the decoder is not the right fix, so I'm withdrawing this PR.

The affected Codex extension bundle selects jsonCompatibleWithBytes on both sides of its VS Code RPC connection. However, further inspection shows that VS Code supports typed-array serialization, so postMessage alone does not establish where the type is lost. I have not yet isolated the failing boundary or validated an encoding change through a real restart/reconnect.

I'll continue the downstream investigation in openai/codex#42760. Thank you for pointing me toward the correct transport contract.

@kaiicheng kaiicheng closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants