Repository navigation
feat(codemode): let a host function write into byte arrays it is given - #64379
rekram1-node wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Write-back works for the sync, sync-Effect, async and throwing calls in the new test, and also for Buffer arguments and subarray views. But copying the host's whole copy back means async calls into the same buffer overwrite each other's results (see the inline comment). I checked with a throwaway test against packages/codemode. The full bun test passes, the extensions tests pass with CODEMODE_ROUNDTRIP=1, and tsc --noEmit passes.
Public API: adds Extension.writesBytes and Extension.isWritingBytes to the @opencode/codemode extension API, and an optional writeBack field on the internal ExtensionRequest. Backwards compatible: functions that aren't marked behave as before.
| before ?? Effect.void, | ||
| settle(Effect.suspend(() => outcome<R>(...callHost(request.host as Function, request.args)))), | ||
| // However the call ends, the bytes the host changed in its copies reach the program's arrays. | ||
| const written = (effect: Effect.Effect<Value, Throw, R>) => |
There was a problem hiding this comment.
When a call settles, this copies the host's whole copy over the program's array. For an async call, that copy is a snapshot from when the call started, so anything written to the buffer in between is overwritten with old bytes. I checked this against this branch:
const a = new Uint8Array(4)
await Promise.all([io.readAt(a, 0, /*ms*/ 5), io.readAt(a, 2, 20)]) // each fills 2 bytes at its offset
// → [0,0,3,3], expected [1,1,3,3]
const b = new Uint8Array(4); const p = io.readAt(b, 0, 5); b[3] = 9; await p
// → b[3] === 0Concurrent filehandle.read(buffer, offset, …) calls into one buffer are a normal pattern, and Node only writes the bytes it reads. Keeping an untouched copy of the bytes at call time and writing back only the indices whose bytes the host changed would keep everything else the program wrote. A concurrent case in the test would cover it.
…t calls and program writes survive
There was a problem hiding this comment.
The new commit fixes the point from my last review: on settle, only the bytes the host changed are written back. Concurrent writesBytes calls into one buffer now keep each other's bytes, and a write the program makes while a call is pending survives. The new concurrent test covers it, and the extensions tests (normally and with CODEMODE_ROUNDTRIP=1), the full bun test and tsc --noEmit all pass. Nothing new to raise.
The public API impact is unchanged from the last review: Extension.writesBytes and Extension.isWritingBytes are new and opt-in, so this is backwards compatible.
Issue for this PR
None.
Type of change
What does this PR do?
A host function marked
Extension.writesBytescan fill aUint8Array(orBuffer) a program passes it. Code Mode copies every argument into the host, so today a host can't write into the program's buffer. That's whatfs.readSync(fd, buffer, offset, length, position)andfilehandle.read(buffer)need, for the file descriptor part of #61186.Writing back only changed bytes keeps what the program, or another call, wrote while an async call was running, so two
filehandle.readcalls into different parts of one buffer don't overwrite each other.It's opt-in so the existing "values are converted, never shared" rule still holds for every other host. Only
fswill mark functions. The write-back happens inextensionWork, the one place every extension call settles, so sync, sync-Effect and async calls all get it.How did you verify your code works?
test/extensions.test.ts: a marked function's writes come back for a plain sync call, a sync Effect, an async call, and a call that throws after writing. An unmarked function's writes don't.packages/codemode:bun test(1657 pass),CODEMODE_ROUNDTRIP=1for the extensions tests,bun typecheck.Screenshots / recordings
Not a UI change.
Checklist
Added by the opencode bot (AI-generated).
Public API: adds Extension.writesBytes / Extension.isWritingBytes to codemode's extension API (opt-in, backwards compatible)