3.1.1 hotfix: tighten extraction.highlights strict-validation contract - #42
Merged
Merged
Conversation
Forward-compat tightening of the `extraction.highlights` sub-object:
- The Pydantic model and TypedDict no longer expose any sub-fields at
the user-facing API surface; unknown keys (typos, unsupported knobs)
continue to be rejected with `pydantic.ValidationError` (`extra="forbid"`
was already in place; the prior version carried an internal knob that
should not have been part of the public contract). Both the model
instance form and the dict form are validated, mirroring the server's
strict `extraction` schema.
- `ExtractionHighlights` is kept as an empty container so that the
dict shape stays stable for future sub-fields; callers continue to
write `extraction={"extraction_mode": "highlights"}` without
depending on a future field.
- Tests, docs (MIGRATION, extraction model page, parameter table in
`search-retrieve-page-content.mdx`), and the changelog are aligned
with the new public surface.
User-visible change: callers passing the previously-exposed internal
knob will now get a fast, local `ValidationError` (instead of silently
sending the field to the server). All other behavior is unchanged.
Validation:
- mypy clean (81 source files).
- 34 / 34 tests in `tests/test_extraction.py` pass.
- Full non-mockserver test suite: 208 pass (78 mockserver-only fails
are unchanged from 3.1.0; CI runs mockserver and turns them green).
- Wheel smoke: `Extraction.model_validate({"extraction_mode":
"highlights", "highlights": {"max_tokens": 1000}})` raises
ValidationError; valid inputs still round-trip.
- Live prod smoke: highlights + full_page against prod still return
expected shapes.
Refs: forward-compat tightening (no Linear issue; requested by
product). Mirrors docs PR #307 follow-up that drops the equivalent
table row in `youdotcom-docs/fern/pages/search-retrieve-page-content.mdx`.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Contributor
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
tyler5673
marked this pull request as ready for review
August 12, 2026 18:09
Contributor
|
Droid finished @tyler5673's task —— View job LGTM — no high-confidence, actionable issues found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Forward-compat tightening of
extraction.highlights. 3.1.0 carried a small set of knobs on the user-facingExtractionHighlightsmodel that the server doesn't currently accept; 3.1.1 keeps the sameextra="forbid"runtime behaviour but trims the public surface to an empty container so callers can't accidentally route an unsupported parameter to the server.The
extraction.highlightssub-object is the canonical forward-compat slot for future sub-fields. Unknown keys still raisepydantic.ValidationErrorlocally on both the model instance form and the dict form, mirroring the server's strictextractionschema.Files
pyproject.tomlversion = "3.1.0"→3.1.1src/youdotcom/models/extraction.pyExtractionHighlightsTypedDict;ExtractionHighlightsis now an empty container. Module- and class-level docstrings are forward-compat only.src/youdotcom/sdk.pytests/test_extraction.pymax_tokens-specific tests. Newtest_extraction_unsupported_highlights_key_raises_locallycovers the strict-validation path an end-user would now hit.tests/test_live.pytest_highlights_with_max_tokens(the field no longer exists).tests/test_performance.pymax_tokens=1000knob from the highlights-metric call.CHANGELOG.md[3.1.1] - 2026-08-12section above the 3.1.0 entry.MIGRATION.mdmax_tokensrows from the migration table and the dict example.docs/models/extraction.mdhighlightsrow of the field table to reflect the empty container.Mirror on docs PR #307
The same fix is mirrored as a one-line edit on the docs PR branch (
Su-Sea/youdotcom-docsPR #307, commite384e0e): theextraction.highlights.max_tokensrow in the parameter table on/docs/guides/retrieve-page-contentis replaced with a forward-compat container entry.Validation
mypy clean on
src/youdotcom/(81 files).34 / 34 tests in
tests/test_extraction.pypass.Full non-mockserver test suite: 208 pass; 78 mockserver-only fails are unchanged from 3.1.0 (CI runs mockserver and these turn green on
b7eb08aand on this branch).Wheel smoke in a fresh venv:
Extraction.model_validate({"extraction_mode": "highlights", "highlights": {"max_tokens": 1000}})raisesValidationError; valid inputs still round-trip with emptyhighlights.Live prod smoke through the existing
~/Workspace/Temp/youdotcom-sdk-3.1.0-smoke/venv against prod with the prodYDC_API_KEY:extraction={"extraction_mode": "highlights"}—results.web[].contents.highlightsreturned in 0.78 s.Extraction(extraction_mode=FULL_PAGE, full_page=...)— full-page Markdown returned in 1.20 s.Out of scope
dy-723-…(d903a6b→e384e0e).