Check the ODCS array options minItems, maxItems and uniqueItems - #1531
Open
OGsiji wants to merge 1 commit into
Open
Check the ODCS array options minItems, maxItems and uniqueItems#1531OGsiji wants to merge 1 commit into
OGsiji wants to merge 1 commit into
Conversation
They measure the elements of one row's array, so they are invalid_count checks like the other logicalTypeOptions, one check per constraint so a report says which one failed. A column the contract calls an array but the server does not cannot be measured that way. Dropping the constraint there would report the check as passed, so it reports an error naming the type it found instead. uniqueItems: false is the default and asserts nothing, so it produces no check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1514, where you said the unimplemented
logicalTypeOptionswould be welcome.These measure the elements of one row's array rather than counting rows, so they are
invalid_countchecks like the existing options, and compile toARRAY_LENGTHandLIST_DISTINCTthrough ibis. One check per constraint, so a failure names which one broke.uniqueItems: falseis the default and asserts nothing, so it produces no check rather than one that always passes.The case I want to flag
A column the contract calls an array, but the server does not.
My first version guarded the predicate on the column being an array and left the constraint off otherwise. That turned out to be wrong in a way worth describing: when
_invalid_exprreturnsNonethe executor recordsinvalid_count = 0 (no validity constraints configured)and evaluates the check as passed. So a contract assertingminItems: 2against aVARCHARcolumn would have reported green.It now reports an error naming the type it found:
A constraint that cannot be measured is not a constraint that holds. Worth knowing that
Nonepath exists, since anything else added to_valid_exprthat can decline to build a predicate inherits the same behaviour.Contract-level mistakes are already caught upstream — the ODCS schema rejects
minItemson a string property, so this only arises when the column itself has drifted.On
minProperties/maxPropertiesI said on the issue I would do these too, and I have left them out deliberately.
They are only row-level on a
mapcolumn, where the key count varies per row and ibis gives a usable.length(). On astructthe fields are fixed by the schema, so the count is the same for every row and the check is either trivially true or trivially false for the whole table — closer to the existing physical type check than to a value check.So they are two different checks wearing one option name, and which you get depends on the physical type rather than anything the contract says. I would rather ask than guess: should
minPropertiesapply only to map-typed columns, become a schema-level assertion on structs, or be left alone? Happy to implement whichever you prefer.multipleOfanddefaultTimezoneare also still unimplemented; I skipped them here to keep this reviewable.12 tests added, full suite green (2120 passed).