Skip to content

feat(codemode): let a host function write into byte arrays it is given - #64379

Draft
rekram1-node wants to merge 2 commits into
devfrom
codemode-byte-args
Draft

rekram1-node wants to merge 2 commits into
devfrom
codemode-byte-args

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

None.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

A host function marked Extension.writesBytes can fill a Uint8Array (or Buffer) a program passes it. Code Mode copies every argument into the host, so today a host can't write into the program's buffer. That's what fs.readSync(fd, buffer, offset, length, position) and filehandle.read(buffer) need, for the file descriptor part of #61186.

const buffer = Buffer.alloc(16)
const read = fs.readSync(fd, buffer, 0, 16, 0)   // before: buffer stayed all zeros
buffer.toString("utf8", 0, read)                 // now: the file's first bytes
call a writesBytes function
  each Uint8Array argument crosses as a copy, as before, and the bytes it held are kept
  when the call settles (returned, threw, or its promise settled)
    → only the bytes the host changed are written into the program's array
unmarked functions: unchanged, they still can't change what they were passed

Writing back only changed bytes keeps what the program, or another call, wrote while an async call was running, so two filehandle.read calls into different parts of one buffer don't overwrite each other.

It's opt-in so the existing "values are converted, never shared" rule still holds for every other host. Only fs will mark functions. The write-back happens in extensionWork, the one place every extension call settles, so sync, sync-Effect and async calls all get it.

How did you verify your code works?

  • New test in test/extensions.test.ts: a marked function's writes come back for a plain sync call, a sync Effect, an async call, and a call that throws after writing. An unmarked function's writes don't.
  • Concurrent async calls filling different parts of one buffer keep each other's bytes, and a byte the program writes while a call is pending survives. This test fails with a whole-copy write-back.
  • The existing test that a host can't change an argument still passes unchanged.
  • packages/codemode: bun test (1657 pass), CODEMODE_ROUNDTRIP=1 for the extensions tests, bun typecheck.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Added by the opencode bot (AI-generated).

Public API: adds Extension.writesBytes / Extension.isWritingBytes to codemode's extension API (opt-in, backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write-back works for the sync, sync-Effect, async and throwing calls in the new test, and also for Buffer arguments and subarray views. But copying the host's whole copy back means async calls into the same buffer overwrite each other's results (see the inline comment). I checked with a throwaway test against packages/codemode. The full bun test passes, the extensions tests pass with CODEMODE_ROUNDTRIP=1, and tsc --noEmit passes.

Public API: adds Extension.writesBytes and Extension.isWritingBytes to the @opencode/codemode extension API, and an optional writeBack field on the internal ExtensionRequest. Backwards compatible: functions that aren't marked behave as before.

before ?? Effect.void,
settle(Effect.suspend(() => outcome<R>(...callHost(request.host as Function, request.args)))),
// However the call ends, the bytes the host changed in its copies reach the program's arrays.
const written = (effect: Effect.Effect<Value, Throw, R>) =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a call settles, this copies the host's whole copy over the program's array. For an async call, that copy is a snapshot from when the call started, so anything written to the buffer in between is overwritten with old bytes. I checked this against this branch:

const a = new Uint8Array(4)
await Promise.all([io.readAt(a, 0, /*ms*/ 5), io.readAt(a, 2, 20)]) // each fills 2 bytes at its offset
// → [0,0,3,3], expected [1,1,3,3]
const b = new Uint8Array(4); const p = io.readAt(b, 0, 5); b[3] = 9; await p
// → b[3] === 0

Concurrent filehandle.read(buffer, offset, …) calls into one buffer are a normal pattern, and Node only writes the bytes it reads. Keeping an untouched copy of the bytes at call time and writing back only the indices whose bytes the host changed would keep everything else the program wrote. A concurrent case in the test would cover it.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit fixes the point from my last review: on settle, only the bytes the host changed are written back. Concurrent writesBytes calls into one buffer now keep each other's bytes, and a write the program makes while a call is pending survives. The new concurrent test covers it, and the extensions tests (normally and with CODEMODE_ROUNDTRIP=1), the full bun test and tsc --noEmit all pass. Nothing new to raise.

The public API impact is unchanged from the last review: Extension.writesBytes and Extension.isWritingBytes are new and opt-in, so this is backwards compatible.

@rekram1-node
rekram1-node marked this pull request as draft November 17, 2026 06:56

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant