Skip to content

Four places the declaration outruns the check #1

Description

@borisdev

Not expressiveness gaps — those are closed and enforced by docs/probe_builder_features.py. These are places where a GraphSpec claims something that nothing verifies. All four measured 2026-08-30 against cc59d83.


3. object as a declared type opts out of checking, silently — the only real bug here

verdict = VariableSpec("verdict", object)     # examples/ladder/stage9_decision.py uses this
check() -> CLEAN

Declaring object disables check_variable_types for that variable and says nothing. That is 0 FOUND rendering as a pass, which .claude/rules/checks.md is most insistent about: NOT CHECKED and 0 FOUND must never render the same.

Fix: check_variable_types emits NOT CHECKED — 'verdict' is declared as \object`, which accepts anything` and folds it into the existing single summary line. The opt-out becomes visible instead of invisible.

Cost: rung 9's output gains a NOT CHECKED line. That is an improvement — it makes a choice legible.


1. The value can be the wrong one. Not fixable by declaring harder.

cite = NodeSpec("cite", inputs=(draft_graph,), outputs=(cited_graph,))

async def cite_that_cites_nothing(ctx) -> CaseGraph:
    return ctx.inputs          # returns the DRAFT, untouched

check() -> CLEAN
run -> pmids=[]                # nothing was cited

draft_graph, cited_graph, verified_graph, case_graph are all one type — which is exactly why VariableSpec names them. But naming makes the wiring checkable, not the behaviour.

No action. This is the boundary between what a spec checks and what an eval measures, and it should be documented as such rather than chased. It is also why a battle metric like CitedEdges is the thing that reads: it counts what the step actually produced.


2. check_variable_types reads the annotation, and Python does not enforce it

async def annotated_int_returns_str(ctx) -> int:
    return "not an int"

check() -> CLEAN      run -> 'not an int'

No action beyond documenting it. The check complements a static type checker; it does not replace one. Worth one sentence in the check's docstring so nobody over-trusts it.


4. A streaming node's return type is never comparable

An async generator has no useful return annotation, so check_variable_types always reports NOT CHECKED for streams=True nodes.

No action. Already honest — it reports rather than passing silently.


Repro

All four in one script; paste into the repo root:

uv run python3 - <<'PY'
# see the issue body above for the four cases
PY

Measured with workflow_workbench at cc59d83, pydantic-graph 2.35.1.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions