Repository navigation
Conversation
meta-cla
Bot
added
the
CLA Signed
Contributor
robhogan
added a commit
that referenced
this pull request
Sep 24, 2026
## Summary A file map plugin with a worker has it run on every file its `filter` accepts, as files are crawled and as they change. That's right for something like dependency extraction, where nearly every file is needed, but it rules out plugins where only a small, unpredictable subset of files is ever used. Parsing `package.json` is the motivating case - on our benchmark app a build reads about 9% of the `package.json` files in the file map (93 of 1,045). This adds `lazy` to a plugin's worker description. A lazy plugin's worker isn't run at crawl time or on change, and its `filter` isn't consulted. Instead the plugin is given `processFile(mixedPath)` when it's initialised, which runs its worker on that one file - synchronously, in-band - stores the result as the file's plugin data, and returns it. It also emits `metadata`, as lazy SHA-1 does, so the cache knows there's something to save and processed files stay processed across restarts. The plugin decides when to call it. Plugin data is `undefined` for a file until a worker has run on it, and is `undefined` again once the file changes. `null` is a result like any other - it's a non-Haste file's Haste name, for example - so what a worker returns is stored with `undefined` replaced by `null`. Making that hold needed one fix. The crawlers and the watcher created file metadata with the first plugin slot already set to `null`, a relic of when that slot was the Haste name. So an unprocessed file was `null` to the first plugin and `undefined` to any other. They now leave plugin slots unset. On-demand processing bypasses two rules that apply to batches - the plugin's filter, and the exclusion of plugin workers from files in `node_modules` - because the plugin asked for this file. It writes only that plugin's slot, and doesn't mark the file as visited, since the other workers haven't seen it. `FileDataPlugin` takes `lazy`, gains `processFile`, and is now exported, so that a plugin can be defined outside `metro-file-map` without reaching into private paths. This revives the lazy mode from #1612, on top of the plugin machinery that has landed since. ## Test plan ``` yarn jest packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` - `FileProcessor-test.js`: a lazy plugin is left out of a batch and its filter isn't called, a file matched only by a lazy plugin isn't processed at all, and `processFileForPlugin` runs just that plugin in-band for a `node_modules` file, writes only its slot, and stores `null` for `undefined`. - `index-test.js`: end to end with a real worker - plugin data is `undefined` after the crawl, `processFile` stores it and emits `metadata`, and a change to the file leaves it `undefined` again. - `FileDataPlugin-test.js`: `lazy` reaches the worker description, and `processFile` throws before initialisation and delegates after. - The crawler tests now expect fresh metadata without a pre-filled plugin slot. Metadata for files already visited keeps its `null`, which is a real result there. Changelog: Internal
robhogan
added a commit
that referenced
this pull request
Sep 24, 2026
## Summary A file map plugin with a worker has it run on every file its `filter` accepts, as files are crawled and as they change. That's right for something like dependency extraction, where nearly every file is needed, but it rules out plugins where only a small, unpredictable subset of files is ever used. Parsing `package.json` is the motivating case - on our benchmark app a build reads about 9% of the `package.json` files in the file map (93 of 1,045). This adds `lazy` to a plugin's worker description. A lazy plugin's worker isn't run at crawl time or on change, and its `filter` isn't consulted. Instead the plugin is given `processFile(mixedPath)` when it's initialised, which runs its worker on that one file - synchronously, in-band - stores the result as the file's plugin data, and returns it. It also emits `metadata`, as lazy SHA-1 does, so the cache knows there's something to save and processed files stay processed across restarts. The plugin decides when to call it. Plugin data is `undefined` for a file until a worker has run on it, and is `undefined` again once the file changes. `null` is a result like any other - it's a non-Haste file's Haste name, for example - so what a worker returns is stored with `undefined` replaced by `null`. Making that hold needed one fix. The crawlers and the watcher created file metadata with the first plugin slot already set to `null`, a relic of when that slot was the Haste name. So an unprocessed file was `null` to the first plugin and `undefined` to any other. They now leave plugin slots unset. On-demand processing bypasses two rules that apply to batches - the plugin's filter, and the exclusion of plugin workers from files in `node_modules` - because the plugin asked for this file. It writes only that plugin's slot, and doesn't mark the file as visited, since the other workers haven't seen it. `FileDataPlugin` takes `lazy`, gains `processFile`, and is now exported, so that a plugin can be defined outside `metro-file-map` without reaching into private paths. This revives the lazy mode from #1612, on top of the plugin machinery that has landed since. ## Test plan ``` yarn jest packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` - `FileProcessor-test.js`: a lazy plugin is left out of a batch and its filter isn't called, a file matched only by a lazy plugin isn't processed at all, and `processFileForPlugin` runs just that plugin in-band for a `node_modules` file, writes only its slot, and stores `null` for `undefined`. - `index-test.js`: end to end with a real worker - plugin data is `undefined` after the crawl, `processFile` stores it and emits `metadata`, and a change to the file leaves it `undefined` again. - `FileDataPlugin-test.js`: `lazy` reaches the worker description, and `processFile` throws before initialisation and delegates after. - The crawler tests now expect fresh metadata without a pre-filled plugin slot. Metadata for files already visited keeps its `null`, which is a real result there. Changelog: Internal
robhogan
added a commit
that referenced
this pull request
Sep 24, 2026
## Summary A file map plugin with a worker has it run on every file its `filter` accepts, as files are crawled and as they change. That's right for something like dependency extraction, where nearly every file is needed, but it rules out plugins where only a small, unpredictable subset of files is ever used. Parsing `package.json` is the motivating case - on our benchmark app a build reads about 9% of the `package.json` files in the file map (93 of 1,045). This adds `lazy` to a plugin's worker description. A lazy plugin's worker isn't run at crawl time or on change, and its `filter` isn't consulted. Instead the plugin is given `processFile(mixedPath)` when it's initialised, which runs its worker on that one file - synchronously, in-band - stores the result as the file's plugin data, and returns it. It also emits `metadata`, as lazy SHA-1 does, so the cache knows there's something to save and processed files stay processed across restarts. The plugin decides when to call it. Plugin data is `undefined` for a file until a worker has run on it, and is `undefined` again once the file changes. `null` is a result like any other - it's a non-Haste file's Haste name, for example - so what a worker returns is stored with `undefined` replaced by `null`. Making that hold needed one fix. The crawlers and the watcher created file metadata with the first plugin slot already set to `null`, a relic of when that slot was the Haste name. So an unprocessed file was `null` to the first plugin and `undefined` to any other. They now leave plugin slots unset. On-demand processing bypasses two rules that apply to batches - the plugin's filter, and the exclusion of plugin workers from files in `node_modules` - because the plugin asked for this file. It writes only that plugin's slot, and doesn't mark the file as visited, since the other workers haven't seen it. `FileDataPlugin` takes `lazy`, gains `processFile`, and is now exported, so that a plugin can be defined outside `metro-file-map` without reaching into private paths. This revives the lazy mode from #1612, on top of the plugin machinery that has landed since. ## Test plan ``` yarn jest packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` - `FileProcessor-test.js`: a lazy plugin is left out of a batch and its filter isn't called, a file matched only by a lazy plugin isn't processed at all, and `processFileForPlugin` runs just that plugin in-band for a `node_modules` file, writes only its slot, and stores `null` for `undefined`. - `index-test.js`: end to end with a real worker - plugin data is `undefined` after the crawl, `processFile` stores it and emits `metadata`, and a change to the file leaves it `undefined` again. - `FileDataPlugin-test.js`: `lazy` reaches the worker description, and `processFile` throws before initialisation and delegates after. - The crawler tests now expect fresh metadata without a pre-filled plugin slot. Metadata for files already visited keeps its `null`, which is a real result there. Changelog: Internal
robhogan
force-pushed
the
export-D84417547
branch
from
October 15, 2026 05:51
752c8f5 to
c4c958a
Compare
Resolution reads every `package.json` it needs with `readFileSync` on the main thread, on every process start. This adds `PackageJsonPlugin`, a lazy file map plugin holding each manifest's parsed contents. A manifest is read and parsed the first time the resolver asks for it, and not again until it changes. Its contents are saved with the file map, so a warm start reads and parses none. `PackageCache` reads through the plugin via its existing `readPackageJson` option. The plugin reads through `getOrComputeFileData`, and `readPackageJson` takes `observations` to pass through, so every read can be recorded once resolutions collect observations. The stored object is returned as-is, so it's stable while the file is unchanged and shared by every reader, which `PackageJson` being read-only already requires. For a manifest that isn't valid JSON, the plugin stores the `SyntaxError` from parsing it and throws one with the same message, without reading the file again. This also exports the `Observations` type from `metro-file-map`, and adds `Error` to its `V8Serializable` type, since `v8.serialize` keeps an error's class, name and message. Changelog: [Performance] Cache `package.json` contents with the file map, so a warm start reads none from disk
robhogan
force-pushed
the
export-D84417547
branch
from
October 16, 2026 08:56
c4c958a to
ee18210
Compare
robhogan
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, Metro reads every
package.jsonthe resolver needs withreadFileSyncon the main thread, on every process start:metro/packages/metro/src/node-haste/PackageCache.js
Lines 24 to 25 in e69f849
This adds
PackageJsonPlugin, a lazy file map plugin holding each manifest's parsed contents. A manifest is parsed the first time the resolver asks for it, and saved with the file map, so a warm start reads none.PackageCachereads through it via its existingreadPackageJsonoption.Reads go through
getOrComputeFileData(#1951), so once resolutions collect observations every manifest read is recorded as content. The stored object is returned as-is - stable while the file is unchanged, andPackageJsonis already read-only. An invalid manifest stores itsSyntaxErrorand throws it on read, without reading the file again.Storing parsed contents rather than text also leaves room for a trimmed, resolver-shaped projection later, though that's a breaking change because
context.getPackagehands the whole manifest to custom resolvers.This also exports
Observationsfrommetro-file-map, and addsErrortoV8Serializable.Benchmark
On a warm start of an RN 0.87.1 app (6,935 resolutions), this takes
package.jsonreads from 93 to 0 and the first resolver pass down ~3%. Deserialising the stored manifests costs about the same at startup, so it's roughly even end to end today.AI-driven benchmark:
mainpackage.jsonreads from disk, first passChangelog
Test plan
New unit tests.
Resolver output is byte-identical to
mainacross all 6,935 resolutions of the benchmark app. In watch mode, retargeting, removing and restoring abrowserredirect in apackage.jsontakes effect on the next resolution.Benchmark app (AI-driven harness): RN 0.87.1 with a 400-module app and a 600-module workspace package outside the project root, replaying a recorded resolution workload through a watch-mode
DependencyGraphafter priming the file map cache. 15 interleaved rounds per variant on an M4 Pro, Node 26.11.1. The machine was loaded by other work, so timings are from runs at a load average below 10 (8-9 per variant), with medians and bootstrap 95% CIs on the difference.