feat: Add Bounded Concurrent BYOM Workspace Leases

docker-compose.yaml

@@ -13,6 +13,7 @@ services:
             - CODEAPI_BRIDGE_TOKEN=${CODEAPI_BRIDGE_TOKEN:-}
             - CODEAPI_BRIDGE_AUTH_MODE=${CODEAPI_BRIDGE_AUTH_MODE:-paired}
             - CODEAPI_BRIDGE_DYNAMIC_WORKERS=${CODEAPI_BRIDGE_DYNAMIC_WORKERS:-true}
+            - CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS=${CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS:-1}
             - CODEAPI_BRIDGE_WORKER_ID=${CODEAPI_BRIDGE_WORKER_ID:-}
             - CODEAPI_AUTH_PROVIDER=${CODEAPI_AUTH_PROVIDER:-}
             - CODEAPI_ALLOW_AUTH_PROVIDER_NONE=${CODEAPI_ALLOW_AUTH_PROVIDER_NONE:-}
@@ -61,6 +62,7 @@ services:
             - CODEAPI_BRIDGE_TOKEN=${CODEAPI_BRIDGE_TOKEN:-}
             - CODEAPI_BRIDGE_AUTH_MODE=${CODEAPI_BRIDGE_AUTH_MODE:-paired}
             - CODEAPI_BRIDGE_DYNAMIC_WORKERS=${CODEAPI_BRIDGE_DYNAMIC_WORKERS:-true}
+            - CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS=${CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS:-1}
             - CODEAPI_BRIDGE_WORKER_ID=${CODEAPI_BRIDGE_WORKER_ID:-}
             - CODEAPI_AUTH_PROVIDER=${CODEAPI_AUTH_PROVIDER:-}
             - CODEAPI_JWT_SINGLE_TENANT_ID=${CODEAPI_JWT_SINGLE_TENANT_ID:-}

packages/code/README.md

@@ -520,3 +520,63 @@ with `librechat-code reset-workspace <runtime-session-id>`. The command uses the
 configured worker credentials, registers a fresh incarnation, and only clears
 the server fence when no assignment is active. Run it while the normal worker
 process is stopped, then restart the normal worker after the command exits.
+
+### Opt-in concurrent native workspaces
+
+Code API defaults to **one execution slot**. To allow independent native roots
+to execute concurrently, configure `CODEAPI_BRIDGE_MAX_WORKSPACE_LEASE_SLOTS=2`
+on every Code API replica and start an updated worker with:
+
+```sh
+librechat-code run \
+  --worker-dir /projects/first \
+  --workspace second=/projects/second \
+  --workspace-lease-slots 2 \
+  --allow-workspace-writes \
+  --allow-workspace-commands
+```
+
+Keep the existing URL, pairing/identity, and network policy configuration.
+The primary root keeps its configured workspace ID (default `primary`). Repeat
+`--workspace id=path` to add named roots, up to the protocol's 32-root limit.
+Roots must already exist and must not overlap or alias one another. Commands
+retain the selected root's sandbox boundary, not a shared parent-directory grant.
+The `LIBRECHAT_CODE_WORKSPACE_QUARANTINE_FILE` single-file override is rejected
+when multiple roots are configured; unset it to use separate root-derived markers.
+
+`LIBRECHAT_CODE_WORKSPACE_LEASE_SLOTS` is the equivalent worker setting. Both
+ceilings must be integers from 1 to 8; the lower ceiling wins. An older Code API
+without the negotiation receipt keeps the worker on the serial protocol. Deploy
+the updated API to all replicas before enabling slots on workers. A capacity
+change while work is active fails closed; stop and drain the worker before
+changing it.
+
+Different roots can run concurrently; requests targeting the **same root remain
+serialized**, even across chats or agents. This is root-level exclusion, not
+file-level locking. Assign separate project/worktree roots for independent work.
+The admission queue remains bounded at 32 requests per worker. An idle SRT process
+cache is bounded by the local slot setting and evicts only idle executors. Runtime
+sandbox assignments continue through the exclusive legacy lane; this does not
+enable concurrent Docker/NsJail sessions or bypass any approval/network policy.
+
+An uncertain mutation or executor failure leaves an assignment-owned local guard
+and a server-side fence for that root. Healthy roots can continue. The worker
+does not replay the failed command. A guard-cleanup failure after settlement fences
+the root independently without replacing the committed result. Expiring ownership
+receipts exclude command payloads; explicit reset invalidates old fence requests.
+The server releases a root only after result finalization **and** explicit local
+cleanup confirmation. Local guard cleanup has a five-second bound; an expired
+receipt never implies a clean root. Control receipt delivery retries three times.
+If delivery remains unavailable, the root remains fenced while every advertised
+lane keeps polling. Capacity becomes reusable when its owned reservation is
+released or expires; inspect/reset the affected root before using it again.
+Reset-only registration stays unready and cannot attract new assignments.
+To recover a quarantined native root:
+
+1. Stop the worker and inspect or restore the affected directory.
+2. Run `librechat-code clear-workspace-quarantine --worker-dir /projects/second --workspace-id second` using the same deployment/identity configuration.
+3. Run the normal worker command with all its root/slot options plus `--reset-workspace-quarantine second`. This verifies the local guard is cleared, resets the server fence, then exits.
+4. Restart the normal worker command without the reset option.
+
+The workspace selector in LibreChat must preserve these registered IDs. Adding
+roots here does not grant a principal access or change an agent's selected root.

packages/code/src/cli-slots.test.ts

@@ -0,0 +1,143 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { spawnSync } from 'node:child_process';
+import { mkdtemp, mkdir, rm, stat } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+test('CLI rejects aliased and overlapping workspace roots before connecting', async (t) => {
+  const root = await mkdtemp(join(tmpdir(), 'byom-cli-roots-'));
+  t.after(() => rm(root, { recursive: true, force: true }));
+  await mkdir(join(root, '..nested'));
+  for (const extra of [root, join(root, '..nested')]) {
+    const result = spawnSync(
+      process.execPath,
+      [
+        fileURLToPath(new URL('./cli.js', import.meta.url)),
+        'run',
+        '--worker-dir',
+        root,
+        '--workspace',
+        `second=${extra}`,
+      ],
+      {
+        encoding: 'utf8',
+        timeout: 3000,
+        env: {
+          ...process.env,
+          LIBRECHAT_CODE_URL: 'http://127.0.0.1:1',
+          LIBRECHAT_CODE_WORKER_TOKEN: 'fixture',
+          LIBRECHAT_CODE_WORKER_ID: 'fixture-worker',
+          LIBRECHAT_CODE_COMMAND_SANDBOX: 'native-srt',
+        },
+      },
+    );
+    assert.notEqual(result.status, 0);
+    assert.match(result.stderr, /must not overlap or alias/);
+  }
+});
+
+test('CLI bounds requested workspace slots before connecting', () => {
+  const result = spawnSync(
+    process.execPath,
+    [
+      fileURLToPath(new URL('./cli.js', import.meta.url)),
+      'run',
+      '--workspace-lease-slots',
+      '9',
+    ],
+    {
+      encoding: 'utf8',
+      timeout: 3000,
+      env: {
+        ...process.env,
+        LIBRECHAT_CODE_URL: 'http://127.0.0.1:1',
+        LIBRECHAT_CODE_WORKER_TOKEN: 'fixture',
+        LIBRECHAT_CODE_WORKER_ID: 'fixture-worker',
+      },
+    },
+  );
+  assert.notEqual(result.status, 0);
+  assert.match(result.stderr, /cannot exceed 8/);
+});
+
+test('CLI rejects one quarantine-file override for multiple roots', async (t) => {
+  const root = await mkdtemp(join(tmpdir(), 'byom-cli-markers-'));
+  t.after(() => rm(root, { recursive: true, force: true }));
+  await mkdir(join(root, 'a'));
+  await mkdir(join(root, 'b'));
+  const result = spawnSync(
+    process.execPath,
+    [
+      fileURLToPath(new URL('./cli.js', import.meta.url)),
+      'run',
+      '--worker-dir',
+      join(root, 'a'),
+      '--workspace',
+      `second=${join(root, 'b')}`,
+    ],
+    {
+      encoding: 'utf8',
+      timeout: 3000,
+      env: {
+        ...process.env,
+        LIBRECHAT_CODE_URL: 'http://127.0.0.1:1',
+        LIBRECHAT_CODE_WORKER_TOKEN: 'fixture',
+        LIBRECHAT_CODE_WORKER_ID: 'fixture-worker',
+        LIBRECHAT_CODE_COMMAND_SANDBOX: 'native-srt',
+        LIBRECHAT_CODE_WORKSPACE_QUARANTINE_FILE: join(root, 'shared.json'),
+      },
+    },
+  );
+  assert.notEqual(result.status, 0);
+  assert.match(result.stderr, /single-root override/);
+});
+
+test('CLI preserves distinct case-sensitive roots on a non-Linux platform', async (t) => {
+  const root = await mkdtemp(join(tmpdir(), 'byom-cli-case-'));
+  t.after(() => rm(root, { recursive: true, force: true }));
+  await mkdir(join(root, 'Foo'));
+  await mkdir(join(root, 'foo'), { recursive: true });
+  if (
+    (await stat(join(root, 'Foo'))).ino === (await stat(join(root, 'foo'))).ino
+  ) {
+    t.skip('requires a case-sensitive test filesystem');
+    return;
+  }
+  const argv = [
+    'fixture',
+    'run',
+    '--worker-dir',
+    join(root, 'Foo'),
+    '--workspace',
+    `second=${join(root, 'foo')}`,
+    '--workspace-lease-slots',
+    '2',
+  ];
+  const result = spawnSync(
+    process.execPath,
+    [
+      '--input-type=module',
+      '-e',
+      `Object.defineProperty(process, 'platform', {value:'darwin'}); process.argv=[process.execPath,...${JSON.stringify(argv)}]; await import(${JSON.stringify(new URL('./cli.js', import.meta.url).href)});`,
+    ],
+    {
+      encoding: 'utf8',
+      timeout: 3000,
+      env: {
+        ...process.env,
+        LIBRECHAT_CODE_URL: 'http://127.0.0.1:1',
+        LIBRECHAT_CODE_WORKER_TOKEN: 'fixture',
+        LIBRECHAT_CODE_WORKER_ID: 'fixture-worker',
+        LIBRECHAT_CODE_COMMAND_SANDBOX: 'native-srt',
+        LIBRECHAT_CODE_WORKSPACE_QUARANTINE_FILE: '',
+      },
+    },
+  );
+  assert.match(
+    result.stderr,
+    /Concurrent workspace leases require native-srt commands/,
+  );
+  assert.doesNotMatch(result.stderr, /overlap or alias/);
+});