fix(errors): sanitise and unify the Router detail[] validation summary - #167
Conversation
The models.run() 422 path built its message by joining the raw detail[] entries' bare `msg` values, bypassing the excerpt sanitiser: a server-, proxy- or provider-supplied `msg` carrying newlines, C0/C1 controls, ANSI escapes or bidi overrides reached str(exc), tracebacks and log lines verbatim and unbounded, and many long entries made the string arbitrarily large. It also disagreed with the queued path on `.detail` for one wire body — two `field required` entries collapsed to the unrecoverable `field required; field required` on run() while the queued surface named both fields. - Add comfy_low.summarise_detail(): renders each entry as `<loc>: <msg>` and runs the joined line through clean_body_excerpt (unprintable reduction, whitespace collapse, 256-char cap). Both error_from_envelope (awaited run path) and router_exceptions (queued path) now summarise the same array through this one function, so one wire body yields one string; the redundant _summarise is removed. - to_sdk_error: a detail[] array selects a RouterError even under a bucket the v2 envelope also spells (unauthorized/forbidden/insufficient_credits) or the status-derived invalid_workflow guess a stripped X-Comfy-Error-Type falls to — the array only Router sends, so it identifies the surface. The typed entries and summary are no longer dropped, including when the body also carried error.message. - Correct _clean's docstring: the list detail shape still reaches it through the message = _clean(raw_detail) fallback, so its isinstance guard is load-bearing (it keeps a Python list repr out of the caller's message). Regression tests per item; run/queued `.detail` parity asserted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning Review paused — included plan limit reachedKeep your review moving with free on-demand reviews.
On-demand reviews are free for the next 21 days.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Promotion and pricing detailsOn-demand reviews are free for the next 21 days. After that, they cost $0.25 per reviewed file. Review limit detailsOr wait 59 minutes for your next included review. Limit details: You’ve used the included review currently available. Your 138 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Review configuration: ⚙️ Run configurationConfiguration used: Repository: Comfy-Org/comfy-python-sdk/.coderabbit.yaml Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (7)
Comment |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
Found 7 finding(s).
| Severity | Count |
|---|---|
| 🟠 High | 1 |
| 🟡 Medium | 1 |
| 🟢 Low | 5 |
Panel: 5/6 reviewers contributed findings.
Reviewers that did not contribute: gpt-5.6-sol-max:edge-case (error)
robinjhuang
left a comment
There was a problem hiding this comment.
Auto-approved under the full-autonomy policy.
Gates verified at 2096320b057367f1cd038c38d3787ec4f707aa44:
full-autonomylabel present- assigned to, or review requested from, @robinjhuang
- not a draft
- 8 required check(s) green — none failing, none pending
Issued by full-autonomy-approve.yml (run). This approval attests
that the machine gates above passed at this commit. It does not attest that a
human read the diff.
`main` landed #157, which merged the two exception hierarchies: the three buckets both surfaces spell alike (`unauthorized`, `forbidden`, `insufficient_credits`) are one `RouterError` subclass each now, and `_class_for` replaced `_router_only_class`. That dissolves the collision this branch's `to_sdk_error` guard was written for, and it dissolves it *properly*: for any response carrying real Router provenance, `_class_for` already resolves to a `RouterError` subclass -- the only `_BY_CODE`/bucket overlaps are those three, every other bucket hits `_BY_ERROR_TYPE`, and an unknown bucket falls back to `RouterError` -- so the typed `detail[]` entries are forwarded without any help from here. What the guard still did beyond that was the regression 4 of 6 panel reviewers flagged: it rerouted on the ARRAY, a body shape any server, proxy or gateway can send (a FastAPI `RequestValidationError` is exactly it), so a bucket-less 422 stopped raising `InvalidWorkflow` and `except InvalidWorkflow` silently went dead -- and the elected `RouterError` dropped `exc.details` and stamped a v2-vocabulary code onto `.error_type`. It also contradicted the rule `error_from_envelope` already documents, learned three times on live traffic: only a response that carries a BUCKET gets retyped. So the guard is removed and its tests now pin the opposite -- a bucket-less array keeps its v2 class, with the per-field reasons still reaching the caller through the summary. The sanitise-and-unify core is unchanged, plus the panel's other fixes: - `summarise_detail` stops accumulating at the excerpt budget instead of rendering a server-controlled array in full and slicing 256 chars off it. - A `loc` member that is not a path segment is dropped rather than coerced, so a nested one can no longer put a Python repr into the user-visible message. `ValidationErrorDetail.loc`/`.location` get the same treatment and now delegate to the one renderer, so the two cannot drift. - CHANGELOG entry for the user-visible message change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
🤖 The reviews loop filed Linear follow-up ticket(s) for review thread(s) deferred as out of scope for this PR:
The following carry
|
robinjhuang
left a comment
There was a problem hiding this comment.
Auto-approved under the full-autonomy policy.
Gates verified at 9525758506a914a54e12416feafe5a8d7c096b3d:
full-autonomylabel present- assigned to, or review requested from, @robinjhuang
- not a draft
- 8 required check(s) green — none failing, none pending
Issued by full-autonomy-approve.yml (run). This approval attests
that the machine gates above passed at this commit. It does not attest that a
human read the diff.
ELI-5
When a model call is rejected for a bad field, the server sends back a little list of what went wrong. The SDK turns that list into the human-readable error message.
Two problems: (1) it pasted the server's text straight in without cleaning it, so a message containing a newline, a hidden control byte, or an ANSI escape could scribble on your terminal or flood a log line — and a big list could make the message enormous. (2) The message you got from
models.run()looked different from the one you got on the queued/submit()path for the exact same server response:run()dropped the field names, so twofield requirederrors collapsed to a uselessfield required; field required, while the queued path told you which two fields.This change cleans the text (same 256-char, single-line, no-control-chars treatment every other body-derived string already gets) and makes both paths build the message with one shared function, so one server response always reads the same way.
What changed
comfy_low.summarise_detail()renders eachdetail[]entry as<loc>: <msg>and runs the joined line throughclean_body_excerpt(unprintable-category reduction, whitespace collapse,_BODY_EXCERPT_LIMIT/256 cap). Botherror_from_envelope(awaitedrun()path) androuter_exceptions(queued path) now summarise the array through this one function, so a single wire body yields a single.detail. The now-redundant_summariseinrouter_exceptionsis removed._BODY_EXCERPT_LIMIT * _BODY_EXCERPT_WINDOWprefixclean_body_excerptactually reads, rather than rendering a server-controlled array in full and slicing 256 characters off the end — so describing a body costs the same whether the body is small or huge.locmembers that are not path segments are dropped, not coerced.locis typedtuple[str | int, ...]— field names and array indices — so a nested, server-controlled member is skipped instead of beingstr()-ed into a Python repr in the user-visible message. This applied in two places: the summariser, andValidationErrorDetail, whose_detail_fromwas storing the repr and whose.locationre-rendered it..locationnow delegates to the one shared renderer, so the summary and.errors[i].locationcannot disagree about what a field is called._cleandocstring corrected: the listdetailshape still reaches_cleanthrough themessage = _clean(raw_detail)fallback (a validation array that produced no summary), so itsisinstance(value, str)guard is load-bearing — it keeps a Python list repr out of the caller's message. The old docstring claimed the shape never reaches the function, which invited deleting the guard.Each item has a regression test; the run-vs-queued
.detailparity is asserted (assert excinfo.value.detail == queued.detail).Dropped after review: the
to_sdk_errorrerouteAn earlier revision made a
detail[]array select aRouterErroreven when the code collided with a v2 envelope bucket. That is removed, for two independent reasons:mainsolved it properly. fix(errors): makeexcept RouterErrorcatch every Router refusal #157 merged the two exception hierarchies, sounauthorized/forbidden/insufficient_creditsare oneRouterErrorsubclass each. For any response carrying real Router provenance,_class_foralready resolves to aRouterErrorsubclass — the only_BY_CODE/bucket overlaps are exactly those three, every other bucket hits_BY_ERROR_TYPE, and an unknown bucket falls back toRouterError— so the typed entries are forwarded with no help from this branch. Verified acrossinvalid_input, the three shared buckets and an unknown future bucket: all yield aRouterErrorwith.errorspopulated.detail[]is a body shape any server, proxy or gateway can send (a FastAPIRequestValidationErroris exactly it), so keying the class off it let an intermediary decide whichexcepta caller runs: a bucket-less 422 stopped raisingInvalidWorkflow, the electedRouterErrordroppedexc.details, and a v2-vocabulary code got stamped onto.error_type. It also contradicted the ruleerror_from_envelopealready documents, learned three times on live traffic — only a response that carries a bucket gets retyped.The tests that pinned the reroute now pin the opposite: a bucket-less array keeps its v2 class (
not isinstance(err, RouterError)), withdetailsintact and the per-field reasons still reaching the caller through the summary.Scope note / judgment call
The fix chose the "one summariser" option (over merely aligning the low-level format). A side effect is that the queued path's summary is now sanitised and bounded too — the same latent gap the awaited path had. This is deliberate and consistent, and it makes the run/queued
.detailparity exact for every body rather than only clean ones..errorsstill carries the raw typed entries; only the human-readable summary string is cleaned and field-qualified. This also changeserr.message/.detailon therun()path to include the field path (body.steps: too largerather than baretoo large) — the pre-existing per-field tests were updated to the corrected value, and there is a CHANGELOG entry for it.Testing
ruff check .— cleanruff format --check .— clean (57 files)mypy src— no issues (21 files)pytest— 987 passed, 9 skippedorigin/mainate4773c7(which landed fix(errors): makeexcept RouterErrorcatch every Router refusal #157) into the branch; the above was run on the merged tree.Provenance
9525758): ruff check + ruff format --check + mypy src all clean; pytest 987 passed, 9 skipped. The removal of theto_sdk_errorreroute was checked empirically — every Router-provenance bucket still yields aRouterErrorwith.errorspopulated, and a bucket-less body keepsInvalidWorkflowwith itsdetails.to_sdk_errorreroute described in the original revision is removed, not shipped — subsumed by fix(errors): makeexcept RouterErrorcatch every Router refusal #157 and flagged as a regression by 4 of 6 panel reviewers (see the section above). The sanitise-and-unify core is unchanged. One panel finding is deferred to a follow-up rather than fixed here (below).Residual
detail/error.messageforms are still passed through with only a.strip()(four sites acrosscomfy_low.errorsandrouter_exceptions), so a server-supplied string with control characters or ANSI escapes still reachesstr(exc)unbounded. Out of scope here — this PR is scoped to the array shape, and fixing the string forms changes.detailfor every non-validation error on both surfaces. Recorded for filing; the docstring no longer implies the string shape is covered.Post reviewstep failed twice and left them only in an expired GitHub Actions run artifact. That artifact is gone and cannot be fetched; it was not needed, as each finding was re-derived and verified against mergedmain.