Skip to content

metro-file-map: Give plugins getOrComputeFileData, record observed content - #1951

Merged
robhogan merged 3 commits into
mainfrom
robhogan/observe-content-lookup
Oct 16, 2026
Merged

robhogan merged 3 commits into
mainfrom
robhogan/observe-content-lookup

Conversation

@robhogan

@robhogan robhogan commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

#1944 lets FileSystem.lookup record what it observed, but a file found only goes in existence. A plugin's per-file data is derived from the file's contents, so reading it should be recorded in content, which is also invalidated by modification.

Plugins currently read their data from lookup()'s pluginData, and a lazy plugin calls processFile when that's undefined. Recording on either wouldn't be enough - on a warm start, or any read after the first, the data is already stored and processFile never runs.

This replaces both with getOrComputeFileData(path, observations?), on plugin init options and FileDataPlugin. It returns the stored data, or for a lazy plugin runs the worker in-band on a miss, and records the file in content either way. A missing path or directory is a result rather than an error, so a caller falling back to something else has still recorded it. Plugin lookup no longer returns data, so getOrComputeFileData is the only per-file read.

The motivation is package.json, which Metro currently reads with fs outside the file map, leaving nothing to record:

const readPackageJsonSync: ReadPackageJsonFn = absolutePackageJsonPath =>
JSON.parse(readFileSync(absolutePackageJsonPath, 'utf8'));

Changelog: Internal

Test plan:
New unit tests. Nothing passes observations to this yet, so it's only reachable from tests until the package.json plugin follows.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 18, 2026
@robhogan
robhogan force-pushed the robhogan/treefs-observations branch from 79d24cb to f249690 Compare September 29, 2026 14:30
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 71974ca to 94702e2 Compare September 29, 2026 15:26
@robhogan
robhogan changed the base branch from robhogan/treefs-observations to main September 29, 2026 15:26
@robhogan
robhogan changed the base branch from main to robhogan/resolve-package-once-per-directory September 29, 2026 15:31
@robhogan
robhogan changed the base branch from robhogan/resolve-package-once-per-directory to main September 29, 2026 15:31
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 94702e2 to 9e9232f Compare October 15, 2026 06:09
@robhogan robhogan changed the title metro-file-map: Let a lookup record a file it finds as content metro-file-map: Give plugins getFileData, which records the file it reads as content Oct 15, 2026
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 9e9232f to 2204ff8 Compare October 15, 2026 06:30
@robhogan robhogan changed the title metro-file-map: Give plugins getFileData, which records the file it reads as content metro-file-map: Give plugins getOrComputeFileData, which records the file as content Oct 15, 2026
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 2204ff8 to 1f33985 Compare October 15, 2026 10:05
@robhogan robhogan changed the title metro-file-map: Give plugins getOrComputeFileData, which records the file as content metro-file-map: Give plugins getOrComputeFileData, which records the file as observed content Oct 15, 2026
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 1f33985 to b559e11 Compare October 15, 2026 10:24
@robhogan robhogan changed the title metro-file-map: Give plugins getOrComputeFileData, which records the file as observed content metro-file-map: Give plugins getOrComputeFileData, which records as observed content Oct 15, 2026
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from b559e11 to 4f7c473 Compare October 15, 2026 10:38
@robhogan robhogan changed the title metro-file-map: Give plugins getOrComputeFileData, which records as observed content metro-file-map: Give plugins getOrComputeFileData, record observed content Oct 15, 2026
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch 2 times, most recently from a8afbec to 3837704 Compare October 15, 2026 10:41
@robhogan
robhogan marked this pull request as ready for review October 15, 2026 10:41
@robhogan
robhogan requested review from huntie and vzaidman October 15, 2026 10:41
@facebook-github-tools facebook-github-tools Bot added the Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team. label Oct 15, 2026
…ntent

#1944 lets `FileSystem.lookup` record what it observed, but a file found only goes in `existence`. A plugin's per-file data is derived from the file's contents, so reading it should be recorded in `content`, which is also invalidated by modification.

Plugins currently read their data from `lookup()`'s `pluginData`, and a lazy plugin calls `processFile` when that's `undefined`. Recording on either wouldn't be enough - on a warm start, or any read after the first, the data is already stored and `processFile` never runs.

This replaces both with `getOrComputeFileData(path, observations?)`, on plugin init options and `FileDataPlugin`. It returns the stored data, or for a lazy plugin runs the worker in-band on a miss, and records the file in `content` either way. A missing path or directory is a result rather than an error, so a caller falling back to something else has still recorded it. Plugin `lookup` no longer returns data, so `getOrComputeFileData` is the only per-file read.

The motivation is `package.json`, which Metro currently reads with `fs` outside the file map, leaving nothing to record:

https://github.com/react/metro/blob/32ff3c9993384580ca40310f3c4024e5f250837b/packages/metro/src/node-haste/PackageCache.js#L24-L25

Changelog: Internal

Test plan:
New unit tests. Nothing passes observations to this yet, so it's only reachable from tests until the `package.json` plugin follows.
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch from 3837704 to e8ab103 Compare October 15, 2026 10:56
@vzaidman

Copy link
Copy Markdown
Contributor

What if the observations created via both lookup and lookupContent, i.e. run the following for a file:

lookup(filePath, observations)
lookupContent(filePath, observations)

shouldn't we then move the file path from "existence" to "content"?

Comment thread packages/metro-file-map/src/index.js Outdated
@robhogan
robhogan force-pushed the robhogan/observe-content-lookup branch 2 times, most recently from 7d694df to e8ab103 Compare October 15, 2026 14:03
Compute whether a plugin is lazy once per plugin at initialization, rather than on every miss in getOrComputeFileData. A non-lazy plugin misses on every read of a file its filter excludes.

Correct the TreeFS comment that said a path belongs in only one of existence and content, and document on Observations.content that a path may be in both without changing what invalidates the result.
@robhogan

Copy link
Copy Markdown
Collaborator Author

shouldn't we then move the file path from "existence" to "content"?

Yeah good question - there's an argument for that, but I think it's easiest + most performant if we treat all of these sets as write-only during file map access. content is just a stronger check in any case - deletions invalidate it anyway, so being in both existence and content is harmless.

I've change a comment that suggested they were mutually exclusive.

TreeFS.lookup now returns the canonicalPath it found, and getOrComputeFileData adds it to observations.content after a plain lookup. This removes lookupContent, leaving TreeFS with one lookup method. A file read is now recorded in existence as well as content, which is redundant but doesn't change what invalidates the result.
@robhogan

Copy link
Copy Markdown
Collaborator Author

Modified this a bit further - lookupContent (which I wasn't keen on) is now gone and the observations.content.add is now handled by the caller, which is still self-contained within the file map.

@robhogan
robhogan merged commit f188883 into main Oct 16, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants