Skip to content

buffer: add mask() and unmask() - #66335

Open
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/buffer-mask
Open

jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/buffer-mask

Conversation

@jasnell

@jasnell jasnell commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Add buffer.mask(source, mask, output[, offset[, length]]) and buffer.unmask(buffer, mask), which XOR data with a repeating 4-byte key. This is the masking that WebSocket clients apply to every frame they send (RFC 6455, Section 5.3), and that servers undo on every frame they receive.

undici and ws (without optional dependencies) both need mask and unmask. The bufferutil native addon makes it faster but it's an optional dependency that appears to only be installed with ws about 3% of the time. So almost all users run the JS loop.

Since we're shipping undici and WebSocket now, it's worth making this faster.

I put this on buffer because it's an operation on a buffer but it's likely only useful for WebSocket, so we could just as easily expose it via http. I have no particular preference.

For inputs < 24, the JS loop is still faster because we hit the floor of the C++ binding, even with fast apis. There's not much we can do about that. At larger sizes the perf boost is significant.

@jasnell
jasnell requested review from mcollina and ronag September 27, 2026 03:32
@jasnell jasnell added buffer Issues and PRs related to the buffer subsystem. http Issues and PRs related to the http subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Sep 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. typings Issues and PRs related to internal TypeScript declarations. labels Sep 27, 2026
@jasnell

jasnell commented Sep 27, 2026

Copy link
Copy Markdown
Member Author

@lpinca

@codecov

codecov Bot commented Sep 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.86992% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.37%. Comparing base (cd908df) to head (03fe080).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/node_buffer.cc 85.71% 0 Missing and 10 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66335      +/-   ##
==========================================
- Coverage   90.37%   90.37%   -0.01%     
==========================================
  Files         792      792              
  Lines      275324   275447     +123     
  Branches    52764    52797      +33     
==========================================
+ Hits       248828   248933     +105     
- Misses      16918    16920       +2     
- Partials     9578     9594      +16     
Files with missing lines Coverage Δ
lib/buffer.js 99.75% <100.00%> (+<0.01%) ⬆️
src/node_buffer.cc 71.17% <85.71%> (+0.67%) ⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pipobscure

pipobscure commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

First off: I like it! (LGTM)

Question: unmask is just mask(buffer,mask,buffer) to mask in place and as you said likely only useful to people that understand that well-enough to actually implement websocket. (And masking is the easy part. Ask me how I know). Is that really worth a separate function? (And yes this is an extremely nitpicking thing that can well be ignored)

@jasnell

jasnell commented Sep 27, 2026

Copy link
Copy Markdown
Member Author

I kept it as a separate function to keep the API shape the same as bufferutil. Happy to coalesce but would like to see what @lpinca thinks

Comment thread src/node_buffer.cc Outdated
@lpinca

lpinca commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

The discussion about whether to add this feature to the Node.js core dates back to 2015 (#1010 (comment), #1010 (comment), #1202) and I think it would be a valuable addition. Every WebSocket implementation would benefit from this just like buffer.isUtf8() which was originally added for the same reason.

I don't have a strong option about unmask(). I think it is nice sugar and the client only masks while the server only unmasks even though it is basically the same thing. Shipping only mask() is also ok.

Some tests are hard to follow.

@jasnell

jasnell commented Sep 27, 2026

Copy link
Copy Markdown
Member Author

I think there's value in matching the current API so that there's just less for someone migrating from bufferutil to think about.

@jasnell
jasnell force-pushed the jasnell/buffer-mask branch from 03fe080 to ae2229e Compare September 27, 2026 11:03
@jasnell
jasnell requested a review from panva September 27, 2026 11:04
Add buffer.mask(source, mask, output[, offset[, length]]) and
buffer.unmask(buffer, mask), which XOR data with a repeating 4-byte
key. This is the masking that WebSocket clients apply to every frame
they send (RFC 6455, Section 5.3), and that servers undo on every frame
they receive.

Userland WebSocket implementations do this either with a byte-by-byte
JS loop (undici, and ws without optional dependencies) or with the
bufferutil native addon. bufferutil is installed for only about 3% of
ws downloads and has no linux-arm64 prebuild, so almost all users run
the JS loop. The signature matches bufferutil, so existing users can
switch with a feature check, as ws did for buffer.isUtf8().

The implementation uses a V8 fast API call and processes 8-byte words,
which the compiler vectorizes. It handles overlapping source and output
views, and treats every view type as raw bytes. It is about 20x faster
than the JS loop at 1 KiB and 40x faster at 64 KiB, and faster than
bufferutil at every size from 32 bytes up.

Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
jasnell force-pushed the jasnell/buffer-mask branch from ae2229e to dc5a1d7 Compare September 27, 2026 11:10
@jasnell jasnell added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@lpinca lpinca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. http Issues and PRs related to the http subsystem. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version. typings Issues and PRs related to internal TypeScript declarations.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants