Skip to content

fix(test): handle big-endian encoding in EncodeUCS2 test - #1023

Open
abhayagarwal-dev wants to merge 2 commits into
nodejs:mainfrom
abhayagarwal-dev:ucs2-encoding-fix
Open

abhayagarwal-dev wants to merge 2 commits into
nodejs:mainfrom
abhayagarwal-dev:ucs2-encoding-fix

Conversation

@abhayagarwal-dev

@abhayagarwal-dev abhayagarwal-dev commented Sep 16, 2026 •

Copy link
Copy Markdown

Problem
The EncodeUCS2 test in test/cpp/strings.cpp fails on big-endian architectures (s390x). char16_t string literals (like u"hello") are stored in host byte order (BE on s390x), but node::Encode(..., UCS2) expects LE input and applies an internal endianness swap on BE systems.

            process.processTicksAndRejections (node:internal/process/task_queues:77:11)
        found:  栀攀氀氀漀
        wanted: hello
        diff:   |
          FOUND:  栀攀氀氀漀
          WANTED: hello
                  ^ (at position = 0)
      ...

Solution
Convert each 16-bit code unit from BE to LE on big-endian platforms (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__) before passing to node::Encode.
Compute buffer size dynamically (kNumChars * sizeof(uint16_t)).
Little-endian platforms (x86_64, arm64) are unaffected.

Test results
s390x (Big-Endian):

$ node ./node_modules/.bin/tap test/js/strings-test.js
ok test/js/strings-test.js .............................. 7/7
total ................................................... 7/7

ok

arm64(Little-Endian)

$ node ./node_modules/.bin/tap test/js/strings-test.js
ok test/js/strings-test.js .............................. 7/7
total ................................................... 7/7

ok

Full test suite:

$ npm test

> nan@2.28.0 test
> tap --gc --stderr test/js/*-test.js

ok test/js/accessors2-test.js ........................... 5/5
ok test/js/accessors-test.js ............................ 8/8
ok test/js/asyncprogressqueueworkerstream-test.js ....... 2/2
ok test/js/asyncprogressqueueworker-test.js ............. 7/7
ok test/js/asyncprogressworkersignal-test.js ............ 7/7
ok test/js/asyncprogressworkerstream-test.js ............ 2/2
ok test/js/asyncprogressworker-test.js .................. 7/7
ok test/js/asyncresource-test.js ........................ 8/8
ok test/js/asyncworkererror-test.js ..................... 4/4
ok test/js/asyncworker-test.js ........................ 10/10
ok test/js/buffer-test.js ............................... 9/9
ok test/js/bufferworkerpersistent-test.js ............... 8/8
ok test/js/callbackcontext-test.js ...................... 8/8
ok test/js/converters-test.js ......................... 33/33
ok test/js/error-test.js .............................. 61/61
ok test/js/gc-test.js ................................... 4/4
ok test/js/indexedinterceptors-test.js .................. 6/6
ok test/js/isolatedata-test.js .......................... 3/3
ok test/js/json-parse-test.js ........................... 9/9
ok test/js/json-stringify-test.js ..................... 23/23
ok test/js/makecallback-test.js ......................... 2/2
ok test/js/maybe-test.js ................................ 2/2
ok test/js/methodswithdata-test.js ...................... 9/9
ok test/js/morenews-test.js ........................... 17/17
ok test/js/multifile-test.js ............................ 3/3
ok test/js/namedinterceptors-test.js .................... 6/6
ok test/js/nancallback-test.js ........................ 20/20
ok test/js/nannew-test.js ............................. 95/95
ok test/js/news-test.js ............................... 53/53
ok test/js/objectwraphandle-test.js ................... 10/10
ok test/js/persistent-test.js ......................... 16/16
ok test/js/private-test.js .............................. 9/9
ok test/js/returnemptystring-test.js .................... 4/4
ok test/js/returnnull-test.js ........................... 4/4
ok test/js/returnundefined-test.js ...................... 4/4
ok test/js/returnvalue-test.js ........................ 10/10
ok test/js/setcallhandler-test.js ....................... 5/5
ok test/js/settemplate-test.js ........................ 23/23
ok test/js/strings-test.js .............................. 7/7
ok test/js/symbols-test.js .............................. 3/3
ok test/js/threadlocal-test.js .......................... 8/8
ok test/js/trycatch-test.js ............................. 3/3
ok test/js/typedarrays-test.js ........................ 29/29
ok test/js/weak2-test.js ................................ 4/4
ok test/js/weak-test.js ................................. 6/6
ok test/js/wrappedobjectfactory-test.js ................. 5/5
total ............................................... 581/581

ok

@abhayagarwal-dev

Copy link
Copy Markdown
Author

@kkoopa Could you approve the github workflow action when you get a chance? Thank you!

@abhayagarwal-dev

Copy link
Copy Markdown
Author

@kkoopa could you take a look at this PR and merge it if everything looks ok? Thank you.

@kkoopa

kkoopa commented Sep 27, 2026 via email

Copy link
Copy Markdown
Collaborator

@abhayagarwal-dev

Copy link
Copy Markdown
Author

Hi @kkoopa, thanks for the review!

__BYTE_ORDER__ and__ORDER_BIG_ENDIAN__are GCC/Clang predefined macros available since GCC 4.6 , well within any Node version NAN has ever supported. On LE platforms the #if branch is never taken so behaviour is unchanged. On MSVC (Windows, always LE), the macro isn't defined so the #else path is taken. No new compiler requirements beyond what C++11 already mandates for the existing char16_t/u"" literals in this file.

@kkoopa

kkoopa commented Sep 27, 2026 via email

Copy link
Copy Markdown
Collaborator

@abhayagarwal-dev

Copy link
Copy Markdown
Author

Thanks for the context!
Looking at nan.h, for NODE_MODULE_VERSION <= NODE_0_10_MODULE_VERSION the call falls through to imp::Encode in nan_string_bytes.h which does no endianness swap and expects host byte order ,so the swap I added would be wrong there.
I can add a NODE_MODULE_VERSION > NODE_0_10_MODULE_VERSION guard to the #if condition to be safe.

Would that address your concern?

@kkoopa

kkoopa commented Sep 27, 2026 via email

Copy link
Copy Markdown
Collaborator

@abhayagarwal-dev

Copy link
Copy Markdown
Author

That makes complete sense. I have moved the endianness handling directly into nan.h (Nan::Encode / Nan::TryEncode) across all supported Node.js version branches for Encoding == UCS2 on big-endian platforms.

This ensures consistent behavior across all Node versions without requiring any #ifdef conditionals in tests or consumer/user code. The test in test/cpp/strings.cpp remains clean and standard.

I've updated the PR branch with this change.

@abhayagarwal-dev

Copy link
Copy Markdown
Author

Could you approve the workflow for the changes so it can be tested? Thank you.

@abhayagarwal-dev

Copy link
Copy Markdown
Author

@kkoopa, all 23 checks are passing
Let me know if you have any further feedback on this updated approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants