Add complete bundled type stubs - #237
Open
heejaechang wants to merge 4 commits into
Open
heejaechang wants to merge 4 commits into
heejaechang wants to merge 4 commits into
Conversation
Ship typed interfaces for the public JSONPath AST, parsers, extensions, and CLI together with strict Pyright and mypy fixtures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Represent runtime inheritance conflicts conservatively and surface the five remaining stubtest mismatches rather than hiding them with checker directives or allowlists. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.pyicoverage for all 13 public authored modules: package reexports, JSONPath AST, lexer/parser, exceptions, extensions, and CLIAnyor checker suppressionspy.typedand every stub in wheel and sdist, with strict positive and negative Pyright/mypy consumption fixturesCloses #192.
Completeness and compatibility
490/490exported symbols known;23/23referenced non-exported symbols known)masterauthored surface: 13 modules, 34 classes, 6 public module functions, 237 class methods, and 709 signature slotspy.typed, bundled stubs, or typeshed package.pyifiles,jsonpath_ng/py.typed,setup.py, and 3 typing fixtures/config files_plyfiles changed; all 16 runtime.pyfiles are byte-identical tomasterMandatory suppression and uncertainty audit
Scanned all 17 added typing artifacts (13
.pyifiles,py.typed, and 3 typing fixtures/config files):type: ignore: 0pyright: ignore: 0noqa: 0typing.cast(...)/cast(...): 0Any: 0The runtime has exactly five pre-existing
# noqalines. No precision is inferred solely from them: the 20itertoolswildcard exports are verified by runtime object identity and standard-library contracts; root and extensionparsereexports are verified by object identity and concrete source signatures;Fields.fieldsis independently evidenced by constructor assignment, parser flow, runtime tuple values, and tests.Stubtest representability audit
Unflagged
python -m mypy.stubtest jsonpath_ngcannot reach comparison because private vendored_ply/lex.pyreferences removed Python-2 namestypes.StringTypeandtypes.UnicodeType. A cleanmastercheckout reproduces both errors and additionally reports the existing read-only override inAutoIdForDatum.valueand an existing parser assignment error. Making full-package stubtest compile would require modifying or comprehensively stubbing private vendored_ply, contrary to this contribution's public-authored-module boundary.Running stubtest over every authored leaf module reaches runtime comparison and reports exactly five conflicts:
JSONPath.find(data=...)JSONPath.find_or_create(data=...)AutoIdForDatum.valueAutoIdForDatum.pathAutoIdForDatum.contextEach was investigated independently:
JSONPath.find/find_or_createaccepts thedata=keyword, while concrete overrides such asThis.findrename it todatum=and rejectdata=. Runtime introspection confirmsdata=True, datum=Falsefor the base anddata=False, datum=TrueforThis,Child,Parent,Descendants,Fields,Index,Slice, and extension overrides. Declaring the base parameter keyword-capable is runtime-faithful for the base but violates static override compatibility; a general positional-plus-keyword overload produces 20 Pyright/mypy override errors across authored subclasses. Receiver-constrained overloads are not an escape hatch: mypy rejects a narrowedselfbecause its erased type is not a supertype ofJSONPath. Declaring the common base parameter positional-only is safe for every dynamic receiver but stubtest correctly notes that the base runtime accepts the keyword. Omitting concrete declarations is also not a workaround: omitting onlyThis.findincreases stubtest from five to seven conflicts because inheriteddatadisagrees with runtimedatum.DatumInContext.valueis a property with a setter andpath/contextare writable instance fields; direct assignment to all three succeeds.AutoIdForDatumis a realDatumInContextsubclass and replacesvalue,path, andcontextwith properties whosefsetisNone; direct assignment to each raisesAttributeError. Faithful read-only subclass properties are rejected by both Pyright and mypy as overrides of writable attributes (cleanmastermypy reports this directly). Replacing the base fields/property with a generic writable descriptor and declaring explicit read-only subclass properties still produces three Pyright and four mypy override errors. Inheriting the conservative writable union surface preserves static substitutability, while stubtest correctly reports the runtime descriptors as read-only.Final, explicit property, descriptor, and special-method alternatives either fail checker semantics or falsely promise runtime behavior.The stubs therefore keep the safest statically substitutable common contracts and expose the concrete runtime methods where representable. The five residuals are reported explicitly, never suppressed, and no unrelated API is weakened.
Annotation follow-up
A second source/runtime audit replaced the parser's
object-valued PLY production boundary with a finite invariant grammar-value union. The local protocol now matches vendoredYaccProductioninteger and slice indexing, plus mutation, while covering token scalars, JSONPath nodes, field/index lists, filter-expression lists, and sort tuples/lists. Base and extended grammar callbacks share this union so the mutable production parameter remains override-compatible. The positive consumer now models the sample document and items withTypedDictand proves concreteIndex.updateidentity without falsely claiming that destructive field filtering preserves required TypedDict keys.The proposed concrete transform narrowings were checked but intentionally not committed:
Len.findvalues areint,Path.findvalues arestr,Sub/Strvalues arestr, andSplitvalues arestr | list[str].DatumInContext[T]is necessarily invariant becausevalueis writable. ThereforeDatumInContext[int]andDatumInContext[str]are not subtypes ofDatumInContext[object]; narrowing these override returns creates eight Pyright/mypy override errors. Making the datum wrapper covariant would be unsound because callers can replace its value.Sub.findinput tostr | DatumInContext[str]also violates itsThis.find(object)override. Runtime raisesTypeErrorfor non-string values, but the inherited API contradiction cannot be represented by a narrower override without suppression.Split.findis not string-only: the*segment returns alist[str], as both source and runtime tests demonstrate.These contracts remain conservative rather than using
objectas a claim of precise transform output, fake covariant wrappers, or checker exceptions.Validation
python -m tox: 351 passed on CPython 3.13.12 and 3.14.3; CPython 3.12.13 repeatedly completed 350 passed with onetest_roundtrip_extendedHypothesis deadline failure, reproduced unchanged on cleanmaster(runtime exceeded 200 ms once and completed below the deadline on replay); configured 3.10/3.11 skipped because unavailable; aggregate coverage 85%make lintunder GNU Make in WSL on equivalent clean trees: branch and cleanmastereach exit 2 with 734 identical lines and identical SHA-256 output; zero attributable differencespyright --verifytypes jsonpath_ng --ignoreexternal: 100%twine checkpasses both, with only the pre-existing missinglong_description_content_typewarningpy.typed)GitHub and maintainer policy
The final head has no reviews, conversation comments, inline feedback, check runs, or commit statuses. CI and CodeQL workflow runs are
action_requiredwith zero jobs created, which is GitHub's maintainer-approval gate for fork workflows rather than an attributable test failure.Bundled stubs are the primary fit because the project owns the runtime and packaging, compatibility reexports can ship in lockstep, and users become typed from the normal distribution. A typeshed-only package would separate the contract from implementation and leave the existing distribution unmarked. The remaining maintainer decisions are whether bundled typing is preferred over typeshed and whether the runtime inheritance contradictions should eventually be normalized in a separate behavioral change.