Repository navigation
Conversation
The scenario loader parsed `tools` with the generic non-empty string helper, so `"tools": ""` failed with "string value must be a non-empty string" while `--tools ""` was accepted. Parse `tools` at both the top level and per case with a helper that accepts "" and names the field in its error. An explicit "" already flowed through effectiveTools, the runner, and the cache key unchanged, so it now means no tools from every source. Install delivery still rejects it. Fixes #37 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — approve
The change adds file-grader support, JSON path assertions, strict template rendering, and skill installation with careful edge-case handling (null-prototype vars, own-property checks, reserved exit codes), backed by dedicated test coverage. No correctness, security, or performance risks surfaced in the cited code. Approving.
0 inline comment(s).
Verdict was
approve, posted as a comment — this repo does not allow GitHub Actions to approve PRs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #37.
Cause
The scenario loader read
tools, at the top level and per case, withstringValue, which rejects"". Its error,string value must be a non-empty string, did not name the field.--tools ""never went through that check. After config load,""was already handled correctly:effectiveToolsuses??, the claude-p runner checks=== "", and the cache key uses the tools value in effect.Fix
A
toolsValuehelper for the twotoolsfields accepts any string, including"". A non-string value now gets an error that names the field:""means no tools at every level. The order is: per-casetools, then--toolsor top-leveltools, then the mode default. Install delivery still rejects"".README and SPEC now describe the
toolsfield and this order.Behavior kept as is
toolswins over--tools, as before. This is now documented.validateRunnerSupportlooks only at the tools value in effect, so"tools": ""on an artifact-mode scenario passes on a text-only runner.--tools ""already behaved this way.Testing
bun run check: 165 tests pass, typecheck clean."mode": "text", "tools": "") loads, and runs the same as--tools ""on a text-only runner""sends no tools""overrides a top-level list""from the file🤖 Generated with Claude Code