Not expressiveness gaps — those are closed and enforced by docs/probe_builder_features.py. These are places where a GraphSpec claims something that nothing verifies. All four measured 2026-08-30 against cc59d83.
3. object as a declared type opts out of checking, silently — the only real bug here
verdict = VariableSpec("verdict", object) # examples/ladder/stage9_decision.py uses this
check() -> CLEAN
Declaring object disables check_variable_types for that variable and says nothing. That is 0 FOUND rendering as a pass, which .claude/rules/checks.md is most insistent about: NOT CHECKED and 0 FOUND must never render the same.
Fix: check_variable_types emits NOT CHECKED — 'verdict' is declared as \object`, which accepts anything` and folds it into the existing single summary line. The opt-out becomes visible instead of invisible.
Cost: rung 9's output gains a NOT CHECKED line. That is an improvement — it makes a choice legible.
1. The value can be the wrong one. Not fixable by declaring harder.
cite = NodeSpec("cite", inputs=(draft_graph,), outputs=(cited_graph,))
async def cite_that_cites_nothing(ctx) -> CaseGraph:
return ctx.inputs # returns the DRAFT, untouched
check() -> CLEAN
run -> pmids=[] # nothing was cited
draft_graph, cited_graph, verified_graph, case_graph are all one type — which is exactly why VariableSpec names them. But naming makes the wiring checkable, not the behaviour.
No action. This is the boundary between what a spec checks and what an eval measures, and it should be documented as such rather than chased. It is also why a battle metric like CitedEdges is the thing that reads: it counts what the step actually produced.
2. check_variable_types reads the annotation, and Python does not enforce it
async def annotated_int_returns_str(ctx) -> int:
return "not an int"
check() -> CLEAN run -> 'not an int'
No action beyond documenting it. The check complements a static type checker; it does not replace one. Worth one sentence in the check's docstring so nobody over-trusts it.
4. A streaming node's return type is never comparable
An async generator has no useful return annotation, so check_variable_types always reports NOT CHECKED for streams=True nodes.
No action. Already honest — it reports rather than passing silently.
Repro
All four in one script; paste into the repo root:
uv run python3 - <<'PY'
# see the issue body above for the four cases
PY
Measured with workflow_workbench at cc59d83, pydantic-graph 2.35.1.
Not expressiveness gaps — those are closed and enforced by
docs/probe_builder_features.py. These are places where aGraphSpecclaims something that nothing verifies. All four measured 2026-08-30 againstcc59d83.3.
objectas a declared type opts out of checking, silently — the only real bug hereDeclaring
objectdisablescheck_variable_typesfor that variable and says nothing. That is0 FOUNDrendering as a pass, which.claude/rules/checks.mdis most insistent about: NOT CHECKED and 0 FOUND must never render the same.Fix:
check_variable_typesemitsNOT CHECKED — 'verdict' is declared as \object`, which accepts anything` and folds it into the existing single summary line. The opt-out becomes visible instead of invisible.Cost: rung 9's output gains a NOT CHECKED line. That is an improvement — it makes a choice legible.
1. The value can be the wrong one. Not fixable by declaring harder.
draft_graph,cited_graph,verified_graph,case_graphare all one type — which is exactly whyVariableSpecnames them. But naming makes the wiring checkable, not the behaviour.No action. This is the boundary between what a spec checks and what an eval measures, and it should be documented as such rather than chased. It is also why a battle metric like
CitedEdgesis the thing that reads: it counts what the step actually produced.2.
check_variable_typesreads the annotation, and Python does not enforce itNo action beyond documenting it. The check complements a static type checker; it does not replace one. Worth one sentence in the check's docstring so nobody over-trusts it.
4. A streaming node's return type is never comparable
An async generator has no useful return annotation, so
check_variable_typesalways reports NOT CHECKED forstreams=Truenodes.No action. Already honest — it reports rather than passing silently.
Repro
All four in one script; paste into the repo root:
Measured with
workflow_workbenchatcc59d83, pydantic-graph 2.35.1.