fs: allocate FSReqPromise stat arrays lazily#63886
Open
MarshallOfSound wants to merge 1 commit into
Open
Conversation
Every promise-based fs operation eagerly allocated two AliasedBuffers (a stats array and a statfs array) at request creation, although only stat-family resolutions ever read the first and only statfs() reads the second. Each allocation is an ArrayBuffer, a TypedArray and a strong v8::Global. The callback path has no equivalent cost since it resolves through a shared global array. Construct the arrays lazily in ResolveStat()/ResolveStatFs() instead. Once created the lifetime is unchanged, so deferred continuations still read from request-owned memory. Improves fs/promises throughput under concurrency: writeFile +53%, stat +26%, readFile +22% at 64 in-flight operations on tmpfs, with callback paths unchanged. Signed-off-by: Sam Attard <sattard@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Change
Every promise-based fs operation allocates two
AliasedBuffers at request construction β an 18-field stats array and an 8-field statfs array β although only stat-family resolutions ever read the first and onlystatfs()reads the second. Each one costs anArrayBuffer, aTypedArrayand a strongv8::Global, so a plainfsp.writeFile()pays for two backing stores it can never use. The callback path has no equivalent cost:FSReqCallbackresolves stats through a shared per-process array.Under concurrency this allocation churn is the dominant overhead of the promises path β profiling 64 in-flight 1KB reads shows ~21% of CPU ticks in the malloc family for promises vs ~5% for callbacks issuing identical syscalls.
This PR constructs the arrays lazily in
ResolveStat()/ResolveStatFs():statfsallocates the statfs array.MemoryInforeports the arrays only when they exist.benchmark/compare.jsshows significant improvements on the concurrent configurations βwritefile-promises+12.6β¦20.7%,readfile-promises+19.1β¦21.5% (unencoded),bench-stat-promise+2.2β¦3.5% β with sequential large-payload configs neutral (those are threadpool-latency-bound, as expected).Benchmark results
benchmark/compare.js, 10 samples per configuration, Welch t-test (***p<0.001,**p<0.01,*p<0.05):Additional ad-hoc microbenchmarks (tmpfs, 64 ops in flight):
fsp.writeFile1KB +53%,fsp.stat+26%,fsp.readFile1KB +22% β with all callback variants unchanged (Β±3%), confirming attribution since callbacks never constructFSReqPromise. Scripts: https://gist.github.com/MarshallOfSound/7d22707e3e550a782e96f9da0081e9f1Checklist
test/parallel/test-fs-*pass (342 tests; 3 pre-existing environment failures fail identically onmain)fsp.stat,fsp.statfsandfsp.readFileresolve correctly through the lazily-created-array paths