Skip to content

feat(core): add Node-style fs and path to Code Mode behind full_codemode - #61186

Closed
rekram1-node wants to merge 9 commits into
devfrom
codemode-fs
Closed

rekram1-node wants to merge 9 commits into
devfrom
codemode-fs

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

None.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Builds on #64363 (sync globals block on an Effect, which *Sync needs) and #64365 (Buffer), both merged.

With full_codemode on, Code Mode programs get Node's fs (sync and fs.promises) and path, also importable as node:fs, fs/promises and node:path. Every call checks the same permission rules the file tools check. With the flag off nothing changes: no globals, same execute description.

const fs = require("fs")
const files = fs.readdirSync("src")
const notes = fs.readFileSync("notes.txt", "utf8")
fs.writeFileSync("out.txt", notes.toUpperCase())

How permissions apply

readFile, readdir, stat, lstat, access, existsSync,
glob (on the pattern's fixed prefix)               → FileAccess.authorizeRead   read rule (+ external_directory)
writeFile, appendFile, copyFile (destination)      → FileWrite.write            edit rule, diff, format, BOM
mkdir, mkdtemp, unlink, truncate, binary writes    → FileWrite.authorize        edit rule
rm, rmdir, rename, cp                              → one edit request covering every path touched
permission errors reach the program as core reports them for a tool ("Permission denied: read", or the user's feedback)
missing → { code: "ENOENT" }
outside the project → external_directory is approved before any disk check, so a denied program can't tell
                      whether an outside path exists (rm, rename, copyFile EXCL, appendFile)

Same actions and resource strings as the tools, so existing permissions rules and the plugin permission hook cover fs without changes. Each call gets a progress row (tool fs, method and path).

Why permissions, not tool calls

Node's fs doesn't map onto the file tools: read returns line-numbered pages, write formats, and there's no stat/readdir/rename tool. Routing every call through a tool would change the semantics and pay a tool call per stat in a scan. So fs calls the services the tools use instead.

The trade-off: tool.execute.before/after hooks don't fire for fs calls, the same as fetch today. A plugin that blocks files through tool hooks rather than the permission system won't see them, which is one reason this stays behind the opt-in-only full_codemode flag.

What it covers

  • readFile (a Buffer without an encoding), writeFile (with a/wx flags), appendFile, readdir (withFileTypes, recursive, encoding: "buffer"), stat/lstat (with throwIfNoEntry), access, mkdir (recursive returns the first directory created), mkdtemp, rm, rmdir, unlink, truncate, rename, copyFile, cp (recursive) and glob (cwd, exclude, withFileTypes), each as *Sync and on fs.promises, plus existsSync (never throws) and constants. Dirents carry name, parentPath, path and every is*().
  • Writes keep exactly the bytes written: no formatter and no BOM handling, unlike the write tool. A missing parent directory is ENOENT, as in Node.
  • appendFile and truncate are single host operations: Environment.Files gains append and truncate (fs.appendFile/fs.truncate locally, cat >>/truncate -s in the exec driver, plus the memory driver and the workspace passthrough), so concurrent appends to one file keep every chunk instead of read-modify-write racing.
  • path is the host's own module, with path.posix and path.win32. resolve and relative start from the session's directory, where fs resolves relative paths too.

Known differences from Node

  • stat has size, mtimeMs, mtime and the is*() checks only.
  • Not here: callback forms, chmod/chown/utimes/symlink/readlink/realpath/link/statfs and the rest of stat, file descriptors and FileHandle, streams, and watch.

How did you verify your code works?

  • packages/core: test/tool-execute.test.ts (43 pass) covers reads and writes with the right permission requests, denied writes never touching disk (sync throws, promise rejects), ENOENT shape, listings, copy/rename/rm, and no fs/path without the flag, UTF-8 vs binary readFile, every outside-path call failing with the same permission error whether or not the path exists, mkdir/mkdtemp/rmdir/truncate, cp and glob (dotfiles skipped like Node), writeFile flags, throwIfNoEntry, existsSync returning false when it may not check, writes keeping exact bytes even with a formatter that rewrites files (fails on the old write path), path.resolve against the session directory, and 20 concurrent appends all landing (fails with read-then-write).
  • test/environment.test.ts: a new conformance check for append and truncate runs against the local, memory and GNU exec drivers; all 35 pass on Linux. Codemode, tool registry, tool search and file access suites pass (134).
  • Live, with the dev CLI and OPENCODE_FLAGS=full_codemode on Claude Haiku 5.5: a program listed src, read notes.txt and wrote out.txt through fs. With permissions denying read notes.txt and edit src/*, both calls failed with Permission denied and an allowed write succeeded. With the flag off, typeof fs was "undefined".
  • bun typecheck in core and codemode; bun run check from the root.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Added by the opencode bot (AI-generated).

Public API: adds optional experimental.codemode_fs to config, the OpenAPI spec and the generated client types (backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as described. I ran programs through the execute tool against a temporary directory and recorded every permission request. Reads, writes, binary writes, append, stat, readdir (with withFileTypes), recursive rm, recursive rename, paths outside the project and .. paths all asked the expected read, edit and external_directory permissions and gave Node-shaped results. In packages/codemode, all 1501 tests pass. In packages/core, the changed test files pass except one MCP test that also fails on the base. tsc is clean in both packages.

Public API: unchanged. This adds Code Mode globals and updates the execute tool description; the FileWrite.authorize and Code Mode extension changes are internal.

One gap worth fixing (inline): readFile/readFileSync with no encoding returns a plain Uint8Array, so the common Node patterns fs.readFileSync(p).toString() and JSON.parse(fs.readFileSync(p)) silently produce "123,34,…" or throw.

Comment thread packages/core/src/codemode/fs.ts Outdated
const target = yield* host.access.authorizeRead(file, host.execution)
const bytes = (yield* host.files.read(target.absolute)).bytes
const encoding = encodingOf(options)
return encoding ? Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength).toString(encoding) : bytes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With no encoding this returns a plain Uint8Array. Inside Code Mode, fs.readFileSync("package.json").toString() returned "123,34,110,…", and JSON.parse(fs.readFileSync("package.json")) threw a SyntaxError. In Node, a Buffer turns into text in both cases, and models write these patterns all the time. Could the no-encoding result behave like a Buffer for toString() and string conversion? If not, the execute description should at least tell models to pass "utf8".

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit's flag and simplification work. Without experimental.codemode_fs, execute has no fs. With it, writes, binary writes, append, recursive rm and rename asked for the expected permissions in a probe through execute, and ENOENT/EEXIST errors keep Node's shape. The changed core tests plus test/session pass (802), as do all 1501 packages/codemode tests, and tsc is clean in core, codemode and schema.

Public API: adds the optional config key experimental.codemode_fs (boolean, defaults to false). Backwards compatible.

The earlier point about readFileSync without an encoding returning bytes is still open. One new concern, inline: rm and rename check the path on disk before external-directory approval, so a program can tell whether files outside the project exist without being approved.

// Every path inside a removed directory is authorized, so an edit rule for one of them still applies.
function* remove(host: Host, file: string, options?: { recursive?: boolean; force?: boolean }) {
const absolute = (yield* host.access.resolve({ path: file })).absolute
const type = yield* typeOf(host, absolute)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typeOf runs before edit() calls authorizeExternal, so paths outside the project are checked on disk before approval. fs.rmSync("../zz-none", { force: true }) returned without asking anything. Without force, a missing outside path fails with ENOENT before any prompt, while an existing one prompts, so a program can learn which files exist outside the project without approval. rename's destination typeOf has the same order. Running the external-directory check on the resolved targets before any stat or list would close this.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit only regenerates the client types for experimental.codemode_fs, and the change matches the schema.

Public API: the client's ConfigEntry type gains the optional experimental.codemode_fs?: boolean. Backwards compatible.

The two earlier points, bytes returned by readFile with no encoding and the disk check before external-directory approval in rm and rename, are still open.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit only regenerates the OpenAPI spec for experimental.codemode_fs, and it matches the schema and the client types.

Public API: the OpenAPI config schema and the client's ConfigEntry gain the optional experimental.codemode_fs boolean. Backwards compatible.

The two earlier points, bytes returned by readFile with no encoding and the disk check before external-directory approval in rm and rename, are still open.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new Node behaviours work as described. The added test passes and tsc is clean in core. Public API: unchanged beyond the already-noted optional experimental.codemode_fs.

Neither earlier point is addressed: readFileSync(p).toString() still returns "97", and the new rename and copyFile checks widen the pre-approval disk access from thread 2. With external-directory access denied, renameSync onto an outside non-empty directory returned ENOTEMPTY, and copyFileSync(…, COPYFILE_EXCL) returned EEXIST or a permission error depending on whether the outside file exists. Inline comments on both.

// As in Node: a directory may replace an empty directory; anything else onto a directory fails.
if ((yield* typeOf(host, target)) === "directory") {
if ((yield* typeOf(host, source)) !== "directory") return yield* Effect.fail("EISDIR" as const)
if ((yield* tree(host, target)).length > 0) return yield* Effect.fail("ENOTEMPTY" as const)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tree(host, target) lists the destination before edit() asks for external-directory approval. With external_directory denied, fs.renameSync("d", "/tmp/…/full") returned ENOTEMPTY without asking anything, so a program can tell whether an outside directory has anything in it. Please do these typeOf/tree checks after the external check on the source and target.

The COPYFILE_EXCL existence check runs before write() asks for external-directory approval. With external_directory denied, an existing outside destination gave EEXIST and a missing one gave Permission denied, which reveals whether the file exists. Please resolve the destination and run authorizeExternal on it before the typeOf.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit only makes the fs Node-behaviour test platform-independent. It now builds the file URL with pathToFileURL and the nested path with path.join. tool-execute.test.ts passes.

Public API: unchanged by this commit.

The earlier points are still open: bytes returned by readFile with no encoding, and the disk checks before external-directory approval in rm, rename and copyFile.

@rekram1-node rekram1-node changed the title feat(core): add Node-style fs and path to Code Mode feat(core): add Node-style fs and path to Code Mode behind full_codemode Nov 17, 2026

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commits move the feature behind the existing full_codemode flag and make denials fail with Node's EACCES. Both work: a denied outside read fails with EACCES and the expected message. tool-execute.test.ts (34) and all of packages/codemode (1647) pass, and tsc is clean in both packages.

Public API: the experimental.codemode_fs config key, OpenAPI entry and client type are no longer added. full_codemode is an existing flag, and turning it on now also exposes Code Mode fs and path, whose calls bypass the plugin tool hooks. Backwards compatible.

The three earlier points are still open, re-checked at this head:

  • readFileSync(p).toString() returns "97".
  • With external access denied, rmSync of an outside path succeeds (with force), fails with ENOENT or fails with EACCES depending on whether the path exists, with no approval asked. The new EACCES makes that difference explicit.
  • renameSync onto a non-empty outside directory gives ENOTEMPTY, and copyFileSync(…, COPYFILE_EXCL) onto an existing outside file gives EEXIST, both before the external-directory check.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: readFile without an encoding now returns text for UTF-8 files, so .toString() and JSON.parse(fs.readFileSync(p)) work. Outside paths are now approved before any disk check in rm, unlink, rename, copyFile (COPYFILE_EXCL) and appendFile. With external-directory access denied, every outside call I re-ran failed with EACCES, whether or not the path exists. tool-execute.test.ts passes (36) and tsc is clean in core.

Public API: unchanged by this commit. Nothing new to raise.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier points are addressed: readFile returns a Buffer, so .toString() and JSON.parse(fs.readFileSync(p)) work, and outside paths are still approved before any disk check, including in the new rmdir, cp and writeFile wx paths. cp (recursive), mkdtemp and truncate worked with the expected permission requests. tool-execute.test.ts passes (40), as does packages/codemode (1655), and tsc is clean in core.

Public API: no external surface changes. Code Mode programs gain a Buffer global, and Glob.match in packages/util gains an optional dot option (backwards compatible).

New: glob misses matches for patterns with a wildcard in a middle segment (src/*/x.ts returns []) and breaks on absolute patterns. Inline comments on both.

Comment thread packages/core/src/codemode/fs.ts Outdated
const segments = pattern.split("/")
const literal = segments.findIndex((segment) => /[*?[\]{}()!+@]/.test(segment))
// The fixed part of the pattern is where listing starts, and what read permission is asked for.
const base = path.join(cwd, (literal < 0 ? segments : segments.slice(0, literal)).join("/") || ".")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path.join(cwd, …) with the default cwd of "." drops the leading / of an absolute pattern. fs.globSync("/tmp/proj/src/**/*.ts") asked read for tmp/proj/src (relative) and then failed with ENOENT. Models often build patterns with path.join(absoluteDir, "**/*.ts"). Use the pattern's base as is when it's absolute, and path.resolve/path.join only for relative ones.

Without ** this lists only the base directory's direct children, so a pattern with a wildcard in a middle segment never matches anything deeper. With src/a/x.ts and src/b/x.ts on disk, fs.globSync("src/*/x.ts") returned []. The walk needs to go as deep as the pattern has segments after the base, or recurse and let Glob.match filter.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: glob now matches wildcards in middle segments (src/*/x.ts finds src/a/x.ts and src/b/x.ts) and keeps absolute patterns absolute (<abs>/src/**/*.ts returns absolute paths). Checked by running both patterns through execute. readdir with and without recursive is unchanged. tool-execute.test.ts passes (40) and tsc is clean in core.

Public API: unchanged by this commit. Nothing new to raise.

@rekram1-node
rekram1-node force-pushed the codemode-fs branch 2 times, most recently from 1c43f45 to bdefb3e Compare November 17, 2026 05:55

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This push only rebases the pull request now that #64363 and #64365 are merged. The core and util code is unchanged since my last review. tool-execute.test.ts passes (40) and tsc is clean in core. The fixes for the four open threads are still in place.

Public API: no external surface changes. Glob.match in packages/util gains an optional dot option (backwards compatible). Nothing new to raise.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New in this push: fs.symlink lets a program get around the external-directory check entirely. With external access denied, symlinkSync("/tmp/opencode/outside/secret.txt", "k") then readFileSync("k") returned the outside file's contents. symlinkSync("/tmp/opencode/outside", "o") then writeFileSync("o/pwned.txt", …) wrote outside the project. Both only asked edit/read for in-project paths. Inline comment.

The other new pieces worked when I checked them through execute: full Stats, and path.resolve/path.relative against the session's directory. tool-execute.test.ts and test/environment pass (91), and tsc is clean in core. Writes now skip formatting and FileWrite, so the description's permission table is out of date for writeFile.

Public API: no external surface changes. The new Environment.Files methods are internal.

Comment thread packages/core/src/codemode/fs.ts Outdated
yield* host.files.utimes(absolute!, timeOf(atime), timeOf(mtime), false)
}),
// `fs.symlink(target, path)`: the new link at `path` holds `target` as written.
symlink: op("symlink", function* (host, target, file: PathLike) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the link's own path is authorized, and target is written as is. That turns the known path-as-written limitation into a direct way around external_directory. With external access denied, fs.symlinkSync("/tmp/outside/secret.txt", "k"); fs.readFileSync("k", "utf8") returned the secret. fs.symlinkSync("/tmp/outside", "o"); fs.writeFileSync("o/pwned.txt", "x") created a file outside the project. Please resolve target against the link's directory and run authorizeExternal on it before creating the link. Otherwise drop symlink until reads and writes check the real path.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: fs.symlink and the other new link and metadata functions are removed, along with their file-layer additions, so a program can no longer create a link that escapes external_directory. Writes still keep exact bytes, and path.resolve/relative still start from the session's directory. tool-execute.test.ts passes (42) and tsc is clean in core.

Public API: no external surface changes. Nothing new to raise.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This push only lets permission errors through as core reports them. I checked through execute: with external access denied, rmSync gives the same Permission denied: external_directory for missing and existing outside paths. A corrected edit request reaches the program with the user's feedback as the message. tool-execute.test.ts passes (42) and tsc is clean in core.

Public API: unchanged. Nothing new to raise.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This push makes appendFile and truncate single host operations, using the new Environment.Files append/truncate in the local, exec, memory and workspace drivers. Permission is still asked first, with the resulting diff for appends. The tool-execute, environment and workspace tests pass (99), including the concurrent-append and conformance checks, and tsc is clean in core.

Public API: unchanged (Environment.Files is internal). Nothing new to raise.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This push makes an append's edit request show only the added lines, without reading the file. The outside-path approval still runs before any disk check, and a directory target now gives EISDIR before asking. tool-execute.test.ts passes (44) and tsc is clean in core.

Public API: unchanged. Nothing new to raise.

@rekram1-node

Copy link
Copy Markdown
Collaborator Author

Closing in favor of Bun-style file APIs: #64518 (path landed separately in #64476).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant