Conversation
This was referenced Sep 23, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of implementing #7073 (spec in #9485). This adds the internal model the semantic type contract needs and routes schema compatibility checks through one entry point, without changing the behavior of any existing table.
LogicalType::semantic()reads alogical_typestring as a semantic type (string,binary,list,decimal:<p>:<s>,json,blob, or an unchanged type) plus the output encoding a legacy alias implies:large_stringisstringwithlarge_utf8,dict:string:int16:falseisstringwithdictionary:int16:utf8, anddecimal:256:10:2isdecimal:10:2withdecimal256.OutputEncodingparses and prints the values of thelance-schema:output-encodingfield entry and checks them against a semantic type.Schema::to_canonical_types()produces the form a table under the contract records, keeping field IDs, nullability, metadata, and children; nothing calls it yet. The model stays internal to the Rust core; there is no plugin trait.Compatibility checks now decide field types through
Field::type_matcheswith aTypeComparison.SchemaCompareOptionscarries it, and every check against a dataset takes it fromDataset::type_comparison(), as does the Merge commit check for field bindings. Every table currently compares exactly, soField::data_type(), compatibility results, and error messages stay the same. The follow-up derivesTypeComparison::SemanticfromFLAG_SEMANTIC_TYPES.The only parsing change is that the canonical
decimal:<p>:<s>now parses, toDecimal128whenp≤ 38 andDecimal256otherwise. Nothing writes it yet.Checks that compare Arrow
DataTypes directly (project_by_field,do_intersection,Field::merge, the partial-schema check inmerge_insert, index copying and index segment merges) are left unchanged. They compare physical layouts on both sides, and the write path in the follow-up decides how they apply to tables with mixed layouts.