Repository navigation
feat(core): add Node-style fs and path to Code Mode behind full_codemode - #61186
rekram1-node wants to merge 9 commits into
Conversation
There was a problem hiding this comment.
Works as described. I ran programs through the execute tool against a temporary directory and recorded every permission request. Reads, writes, binary writes, append, stat, readdir (with withFileTypes), recursive rm, recursive rename, paths outside the project and .. paths all asked the expected read, edit and external_directory permissions and gave Node-shaped results. In packages/codemode, all 1501 tests pass. In packages/core, the changed test files pass except one MCP test that also fails on the base. tsc is clean in both packages.
Public API: unchanged. This adds Code Mode globals and updates the execute tool description; the FileWrite.authorize and Code Mode extension changes are internal.
One gap worth fixing (inline): readFile/readFileSync with no encoding returns a plain Uint8Array, so the common Node patterns fs.readFileSync(p).toString() and JSON.parse(fs.readFileSync(p)) silently produce "123,34,…" or throw.
| const target = yield* host.access.authorizeRead(file, host.execution) | ||
| const bytes = (yield* host.files.read(target.absolute)).bytes | ||
| const encoding = encodingOf(options) | ||
| return encoding ? Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength).toString(encoding) : bytes |
There was a problem hiding this comment.
With no encoding this returns a plain Uint8Array. Inside Code Mode, fs.readFileSync("package.json").toString() returned "123,34,110,…", and JSON.parse(fs.readFileSync("package.json")) threw a SyntaxError. In Node, a Buffer turns into text in both cases, and models write these patterns all the time. Could the no-encoding result behave like a Buffer for toString() and string conversion? If not, the execute description should at least tell models to pass "utf8".
There was a problem hiding this comment.
The new commit's flag and simplification work. Without experimental.codemode_fs, execute has no fs. With it, writes, binary writes, append, recursive rm and rename asked for the expected permissions in a probe through execute, and ENOENT/EEXIST errors keep Node's shape. The changed core tests plus test/session pass (802), as do all 1501 packages/codemode tests, and tsc is clean in core, codemode and schema.
Public API: adds the optional config key experimental.codemode_fs (boolean, defaults to false). Backwards compatible.
The earlier point about readFileSync without an encoding returning bytes is still open. One new concern, inline: rm and rename check the path on disk before external-directory approval, so a program can tell whether files outside the project exist without being approved.
| // Every path inside a removed directory is authorized, so an edit rule for one of them still applies. | ||
| function* remove(host: Host, file: string, options?: { recursive?: boolean; force?: boolean }) { | ||
| const absolute = (yield* host.access.resolve({ path: file })).absolute | ||
| const type = yield* typeOf(host, absolute) |
There was a problem hiding this comment.
typeOf runs before edit() calls authorizeExternal, so paths outside the project are checked on disk before approval. fs.rmSync("../zz-none", { force: true }) returned without asking anything. Without force, a missing outside path fails with ENOENT before any prompt, while an existing one prompts, so a program can learn which files exist outside the project without approval. rename's destination typeOf has the same order. Running the external-directory check on the resolved targets before any stat or list would close this.
There was a problem hiding this comment.
The new commit only regenerates the client types for experimental.codemode_fs, and the change matches the schema.
Public API: the client's ConfigEntry type gains the optional experimental.codemode_fs?: boolean. Backwards compatible.
The two earlier points, bytes returned by readFile with no encoding and the disk check before external-directory approval in rm and rename, are still open.
There was a problem hiding this comment.
The new commit only regenerates the OpenAPI spec for experimental.codemode_fs, and it matches the schema and the client types.
Public API: the OpenAPI config schema and the client's ConfigEntry gain the optional experimental.codemode_fs boolean. Backwards compatible.
The two earlier points, bytes returned by readFile with no encoding and the disk check before external-directory approval in rm and rename, are still open.
There was a problem hiding this comment.
The new Node behaviours work as described. The added test passes and tsc is clean in core. Public API: unchanged beyond the already-noted optional experimental.codemode_fs.
Neither earlier point is addressed: readFileSync(p).toString() still returns "97", and the new rename and copyFile checks widen the pre-approval disk access from thread 2. With external-directory access denied, renameSync onto an outside non-empty directory returned ENOTEMPTY, and copyFileSync(…, COPYFILE_EXCL) returned EEXIST or a permission error depending on whether the outside file exists. Inline comments on both.
| // As in Node: a directory may replace an empty directory; anything else onto a directory fails. | ||
| if ((yield* typeOf(host, target)) === "directory") { | ||
| if ((yield* typeOf(host, source)) !== "directory") return yield* Effect.fail("EISDIR" as const) | ||
| if ((yield* tree(host, target)).length > 0) return yield* Effect.fail("ENOTEMPTY" as const) |
There was a problem hiding this comment.
tree(host, target) lists the destination before edit() asks for external-directory approval. With external_directory denied, fs.renameSync("d", "/tmp/…/full") returned ENOTEMPTY without asking anything, so a program can tell whether an outside directory has anything in it. Please do these typeOf/tree checks after the external check on the source and target.
The COPYFILE_EXCL existence check runs before write() asks for external-directory approval. With external_directory denied, an existing outside destination gave EEXIST and a missing one gave Permission denied, which reveals whether the file exists. Please resolve the destination and run authorizeExternal on it before the typeOf.
There was a problem hiding this comment.
The new commit only makes the fs Node-behaviour test platform-independent. It now builds the file URL with pathToFileURL and the nested path with path.join. tool-execute.test.ts passes.
Public API: unchanged by this commit.
The earlier points are still open: bytes returned by readFile with no encoding, and the disk checks before external-directory approval in rm, rename and copyFile.
a44f670 to
41211dc
Compare
There was a problem hiding this comment.
The new commits move the feature behind the existing full_codemode flag and make denials fail with Node's EACCES. Both work: a denied outside read fails with EACCES and the expected message. tool-execute.test.ts (34) and all of packages/codemode (1647) pass, and tsc is clean in both packages.
Public API: the experimental.codemode_fs config key, OpenAPI entry and client type are no longer added. full_codemode is an existing flag, and turning it on now also exposes Code Mode fs and path, whose calls bypass the plugin tool hooks. Backwards compatible.
The three earlier points are still open, re-checked at this head:
readFileSync(p).toString()returns"97".- With external access denied,
rmSyncof an outside path succeeds (withforce), fails withENOENTor fails withEACCESdepending on whether the path exists, with no approval asked. The newEACCESmakes that difference explicit. renameSynconto a non-empty outside directory givesENOTEMPTY, andcopyFileSync(…, COPYFILE_EXCL)onto an existing outside file givesEEXIST, both before the external-directory check.
There was a problem hiding this comment.
Addressed: readFile without an encoding now returns text for UTF-8 files, so .toString() and JSON.parse(fs.readFileSync(p)) work. Outside paths are now approved before any disk check in rm, unlink, rename, copyFile (COPYFILE_EXCL) and appendFile. With external-directory access denied, every outside call I re-ran failed with EACCES, whether or not the path exists. tool-execute.test.ts passes (36) and tsc is clean in core.
Public API: unchanged by this commit. Nothing new to raise.
c61a8e9 to
9a6bda4
Compare
There was a problem hiding this comment.
Earlier points are addressed: readFile returns a Buffer, so .toString() and JSON.parse(fs.readFileSync(p)) work, and outside paths are still approved before any disk check, including in the new rmdir, cp and writeFile wx paths. cp (recursive), mkdtemp and truncate worked with the expected permission requests. tool-execute.test.ts passes (40), as does packages/codemode (1655), and tsc is clean in core.
Public API: no external surface changes. Code Mode programs gain a Buffer global, and Glob.match in packages/util gains an optional dot option (backwards compatible).
New: glob misses matches for patterns with a wildcard in a middle segment (src/*/x.ts returns []) and breaks on absolute patterns. Inline comments on both.
| const segments = pattern.split("/") | ||
| const literal = segments.findIndex((segment) => /[*?[\]{}()!+@]/.test(segment)) | ||
| // The fixed part of the pattern is where listing starts, and what read permission is asked for. | ||
| const base = path.join(cwd, (literal < 0 ? segments : segments.slice(0, literal)).join("/") || ".") |
There was a problem hiding this comment.
path.join(cwd, …) with the default cwd of "." drops the leading / of an absolute pattern. fs.globSync("/tmp/proj/src/**/*.ts") asked read for tmp/proj/src (relative) and then failed with ENOENT. Models often build patterns with path.join(absoluteDir, "**/*.ts"). Use the pattern's base as is when it's absolute, and path.resolve/path.join only for relative ones.
Without ** this lists only the base directory's direct children, so a pattern with a wildcard in a middle segment never matches anything deeper. With src/a/x.ts and src/b/x.ts on disk, fs.globSync("src/*/x.ts") returned []. The walk needs to go as deep as the pattern has segments after the base, or recurse and let Glob.match filter.
There was a problem hiding this comment.
Addressed: glob now matches wildcards in middle segments (src/*/x.ts finds src/a/x.ts and src/b/x.ts) and keeps absolute patterns absolute (<abs>/src/**/*.ts returns absolute paths). Checked by running both patterns through execute. readdir with and without recursive is unchanged. tool-execute.test.ts passes (40) and tsc is clean in core.
Public API: unchanged by this commit. Nothing new to raise.
1c43f45 to
bdefb3e
Compare
There was a problem hiding this comment.
This push only rebases the pull request now that #64363 and #64365 are merged. The core and util code is unchanged since my last review. tool-execute.test.ts passes (40) and tsc is clean in core. The fixes for the four open threads are still in place.
Public API: no external surface changes. Glob.match in packages/util gains an optional dot option (backwards compatible). Nothing new to raise.
bdefb3e to
79d23d3
Compare
There was a problem hiding this comment.
New in this push: fs.symlink lets a program get around the external-directory check entirely. With external access denied, symlinkSync("/tmp/opencode/outside/secret.txt", "k") then readFileSync("k") returned the outside file's contents. symlinkSync("/tmp/opencode/outside", "o") then writeFileSync("o/pwned.txt", …) wrote outside the project. Both only asked edit/read for in-project paths. Inline comment.
The other new pieces worked when I checked them through execute: full Stats, and path.resolve/path.relative against the session's directory. tool-execute.test.ts and test/environment pass (91), and tsc is clean in core. Writes now skip formatting and FileWrite, so the description's permission table is out of date for writeFile.
Public API: no external surface changes. The new Environment.Files methods are internal.
| yield* host.files.utimes(absolute!, timeOf(atime), timeOf(mtime), false) | ||
| }), | ||
| // `fs.symlink(target, path)`: the new link at `path` holds `target` as written. | ||
| symlink: op("symlink", function* (host, target, file: PathLike) { |
There was a problem hiding this comment.
Only the link's own path is authorized, and target is written as is. That turns the known path-as-written limitation into a direct way around external_directory. With external access denied, fs.symlinkSync("/tmp/outside/secret.txt", "k"); fs.readFileSync("k", "utf8") returned the secret. fs.symlinkSync("/tmp/outside", "o"); fs.writeFileSync("o/pwned.txt", "x") created a file outside the project. Please resolve target against the link's directory and run authorizeExternal on it before creating the link. Otherwise drop symlink until reads and writes check the real path.
… read UTF-8 files as text
…, and return Buffers from readFile
…lute patterns absolute
…nst the session directory
79d23d3 to
f9f7f6b
Compare
There was a problem hiding this comment.
Addressed: fs.symlink and the other new link and metadata functions are removed, along with their file-layer additions, so a program can no longer create a link that escapes external_directory. Writes still keep exact bytes, and path.resolve/relative still start from the session's directory. tool-execute.test.ts passes (42) and tsc is clean in core.
Public API: no external surface changes. Nothing new to raise.
There was a problem hiding this comment.
This push only lets permission errors through as core reports them. I checked through execute: with external access denied, rmSync gives the same Permission denied: external_directory for missing and existing outside paths. A corrected edit request reaches the program with the user's feedback as the message. tool-execute.test.ts passes (42) and tsc is clean in core.
Public API: unchanged. Nothing new to raise.
…rrent appends keep every chunk
…dit request, without reading the file
There was a problem hiding this comment.
This push makes appendFile and truncate single host operations, using the new Environment.Files append/truncate in the local, exec, memory and workspace drivers. Permission is still asked first, with the resulting diff for appends. The tool-execute, environment and workspace tests pass (99), including the concurrent-append and conformance checks, and tsc is clean in core.
Public API: unchanged (Environment.Files is internal). Nothing new to raise.
There was a problem hiding this comment.
This push makes an append's edit request show only the added lines, without reading the file. The outside-path approval still runs before any disk check, and a directory target now gives EISDIR before asking. tool-execute.test.ts passes (44) and tsc is clean in core.
Public API: unchanged. Nothing new to raise.
Issue for this PR
None.
Type of change
What does this PR do?
Builds on #64363 (sync globals block on an Effect, which
*Syncneeds) and #64365 (Buffer), both merged.With
full_codemodeon, Code Mode programs get Node'sfs(sync andfs.promises) andpath, also importable asnode:fs,fs/promisesandnode:path. Every call checks the same permission rules the file tools check. With the flag off nothing changes: no globals, sameexecutedescription.How permissions apply
Same actions and resource strings as the tools, so existing
permissionsrules and the plugin permission hook coverfswithout changes. Each call gets a progress row (toolfs, method and path).Why permissions, not tool calls
Node's
fsdoesn't map onto the file tools:readreturns line-numbered pages,writeformats, and there's no stat/readdir/rename tool. Routing every call through a tool would change the semantics and pay a tool call perstatin a scan. Sofscalls the services the tools use instead.The trade-off:
tool.execute.before/afterhooks don't fire forfscalls, the same asfetchtoday. A plugin that blocks files through tool hooks rather than the permission system won't see them, which is one reason this stays behind the opt-in-onlyfull_codemodeflag.What it covers
readFile(aBufferwithout an encoding),writeFile(witha/wxflags),appendFile,readdir(withFileTypes,recursive,encoding: "buffer"),stat/lstat(withthrowIfNoEntry),access,mkdir(recursive returns the first directory created),mkdtemp,rm,rmdir,unlink,truncate,rename,copyFile,cp(recursive) andglob(cwd,exclude,withFileTypes), each as*Syncand onfs.promises, plusexistsSync(never throws) andconstants. Dirents carryname,parentPath,pathand everyis*().ENOENT, as in Node.appendFileandtruncateare single host operations:Environment.Filesgainsappendandtruncate(fs.appendFile/fs.truncatelocally,cat >>/truncate -sin the exec driver, plus the memory driver and the workspace passthrough), so concurrent appends to one file keep every chunk instead of read-modify-write racing.pathis the host's own module, withpath.posixandpath.win32.resolveandrelativestart from the session's directory, wherefsresolves relative paths too.Known differences from Node
stathassize,mtimeMs,mtimeand theis*()checks only.chmod/chown/utimes/symlink/readlink/realpath/link/statfsand the rest ofstat, file descriptors andFileHandle, streams, andwatch.How did you verify your code works?
packages/core:test/tool-execute.test.ts(43 pass) covers reads and writes with the right permission requests, denied writes never touching disk (sync throws, promise rejects),ENOENTshape, listings, copy/rename/rm, and nofs/pathwithout the flag, UTF-8 vs binaryreadFile, every outside-path call failing with the same permission error whether or not the path exists,mkdir/mkdtemp/rmdir/truncate,cpandglob(dotfiles skipped like Node),writeFileflags,throwIfNoEntry,existsSyncreturning false when it may not check, writes keeping exact bytes even with a formatter that rewrites files (fails on the old write path),path.resolveagainst the session directory, and 20 concurrent appends all landing (fails with read-then-write).test/environment.test.ts: a new conformance check for append and truncate runs against the local, memory and GNU exec drivers; all 35 pass on Linux. Codemode, tool registry, tool search and file access suites pass (134).OPENCODE_FLAGS=full_codemodeon Claude Haiku 5.5: a program listedsrc, readnotes.txtand wroteout.txtthroughfs. Withpermissionsdenyingread notes.txtandedit src/*, both calls failed withPermission deniedand an allowed write succeeded. With the flag off,typeof fswas"undefined".bun typecheckin core and codemode;bun run checkfrom the root.Screenshots / recordings
Not a UI change.
Checklist
Added by the opencode bot (AI-generated).
Public API: adds optional experimental.codemode_fs to config, the OpenAPI spec and the generated client types (backwards compatible)