Skip to content

fix(harness): fail sandbox filesystem read ops on non-successful execute responses - #2967

Open
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:fix/2961-sandbox-execute-failure-masking
Open

fix(harness): fail sandbox filesystem read ops on non-successful execute responses#2967
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:fix/2961-sandbox-execute-failure-masking

Conversation

@Buktal

@Buktal Buktal commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

AgentScope-Java Version

main @ ea511ec2 (defect observed in production on 2.0.1)

Description

Fixes #2961

BaseSandboxFilesystem's read paths (ls / read / grep / glob) parsed output without consulting the exit code, so any non-successful execute() response was masked as a successful result — a sandbox-backend timeout surfaced to the model as Empty directory: <path>, the raw error text as file content (read), or as fabricated file paths (glob).

Changes

  • All four read methods return fail(...) on !ExecuteResponse.isSuccess(), riding FilesystemTool's existing "Error: ..." branch (the same path fix(harness): make ls() report errors instead of silently returning empty results #2413 established for not-exists / not-a-directory). These methods' shell commands are constructed to exit 0 on every designed path, so any non-successful response means output is not the protocol output the parsers expect.
  • Failure messages carry operation context (Error listing '<path>': ...) and fall back to the exit code when the response carries no diagnostic output — a null-output failure can no longer collapse back into a "success" (error == null).
  • Binary read keeps its designed file_not_found signal for positive exit codes other than 124 (timeout(1) convention), where the command ran and base64 could not read the file; null/negative/124 surface the execution-layer message instead.
  • AbstractSandboxFilesystem.execute() Javadoc documents the non-success contract.
  • 11 regression tests covering the four failure shapes (−1 backend failure, null exit code, 124 timeout, positive command failure) across all read methods.

How to test

mvn -pl agentscope-harness test -Dtest=BaseSandboxFilesystemTest — 21 tests (16 run on non-Linux; 5 are the pre-existing Linux-only shell integration cases).

Checklist

Please check the following items before code is ready to be reviewed.

  • Code has been formatted with mvn spotless:apply
  • All tests are passing (mvn test)
  • Javadoc comments are complete and follow project conventions
  • Related documentation has been updated (e.g. links, examples, etc.)
  • Code is ready for review

…ute responses

BaseSandboxFilesystem's read paths (ls/read/grep/glob) parsed output
without consulting the exit code, so sandbox execution failures —
backend errors mapped to -1, timeouts mapped to 124, unknown codes —
were masked as successful results (e.g. a backend timeout reported to
the model as 'Empty directory').

All read methods now fail on any non-successful response, riding
FilesystemTool's existing error branch. Failure messages carry
operation context and fall back to the exit code when the response has
no diagnostic output, so a null-output failure cannot collapse back
into a success. The binary read keeps its designed file_not_found
signal for positive exit codes other than 124 (timeout(1) convention),
where the command ran and base64 could not read the file.

Fixes agentscope-ai#2961
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...gent/filesystem/sandbox/BaseSandboxFilesystem.java 72.72% 0 Missing and 6 partials ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Sandbox execute failures masked as valid filesystem results - BaseSandboxFilesystem ignores the negative exitCode sentinel

1 participant