Skip to content

Fix LM Studio validator tool compatibility; record real worker fault controls - #195

Draft
halfaipg wants to merge 2 commits into
mainfrom
fix/validator-single-tool-compatibility
Draft

halfaipg wants to merge 2 commits into
mainfrom
fix/validator-single-tool-compatibility

Conversation

@halfaipg

Copy link
Copy Markdown

Confirmed defect

The real LM Studio backend rejects named tool_choice objects with HTTP 400. Generated validator tool probes expose one function per stage, so required preserves the intended task while supporting string-only backends. Existing persisted challenges, seals and verdicts are unchanged. This is not a worker-wide rewrite of customer requests.

Verification

  • Two compatibility regressions failed before the fix.
  • 115 focused challenge, calibration, fidelity, transport, hostile-worker and sealing tests pass.
  • 32 isolated released-v0.3.9 worker HTTP/WebSocket/Redis paths completed, including 12 real GPT-OSS-20B calls.
  • Six valid native controls accepted; eight paired native-output faults detected (malformed tool arguments, output loss, partial truncation, ignored stops).
  • Crucially, all six fresh 20B calls advertised as 120B passed: these functional canaries do not detect this substitution.
  • Independent capture/hash/answer audit repeated successfully; raw captures stay private, aggregate evidence and immutable hashes are documented.
  • Original HTTP400 runs and harness failures preserved separately; corrected requests are not passed off as original-contract success.
  • Staged gitleaks and diff check pass.

Limits

Two cases per family on one engine/quant are not population accuracy. No signed public assignment, operator quorum, production config, payout or routing changes. Two-stage native chain qualification and deployment remain separate.

@halfaipg
halfaipg marked this pull request as draft September 15, 2026 21:33
@halfaipg

Copy link
Copy Markdown
Author

HOLD / converted to draft: additional direct owned DeepSeek controls do not qualify a global tool_choice change. Of four paired required-choice variants, only single-call nonstream passed; both streamed requests returned no calls, and the nonstream chain also returned no call. The originally captured named-choice stream had passed the single-call case, so there is a compatibility/regression risk. Raw controls are retained privately (capture SHA256 c38fa65f7765c0257b93760a89629a03f3891cad01490a899b0461b537e7219a). LM Studio compatibility evidence remains valid, but this runtime patch must be narrowed or replaced before merge. PR194 shadow-only changes are independently green and can deploy without this draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant