Skip to content

Check the ODCS array options minItems, maxItems and uniqueItems - #1531

Open
OGsiji wants to merge 1 commit into
datacontract:mainfrom
OGsiji:logical-type-options
Open

Check the ODCS array options minItems, maxItems and uniqueItems#1531
OGsiji wants to merge 1 commit into
datacontract:mainfrom
OGsiji:logical-type-options

Conversation

@OGsiji

@OGsiji OGsiji commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Part of #1514, where you said the unimplemented logicalTypeOptions would be welcome.

- name: tags
  logicalType: array
  logicalTypeOptions:
    minItems: 1
    maxItems: 10
    uniqueItems: true

These measure the elements of one row's array rather than counting rows, so they are invalid_count checks like the existing options, and compile to ARRAY_LENGTH and LIST_DISTINCT through ibis. One check per constraint, so a failure names which one broke.

uniqueItems: false is the default and asserts nothing, so it produces no check rather than one that always passes.

The case I want to flag

A column the contract calls an array, but the server does not.

My first version guarded the predicate on the column being an array and left the constraint off otherwise. That turned out to be wrong in a way worth describing: when _invalid_expr returns None the executor records invalid_count = 0 (no validity constraints configured) and evaluates the check as passed. So a contract asserting minItems: 2 against a VARCHAR column would have reported green.

It now reports an error naming the type it found:

Column tags is string, not an array, so the constraint cannot be measured.

A constraint that cannot be measured is not a constraint that holds. Worth knowing that None path exists, since anything else added to _valid_expr that can decline to build a predicate inherits the same behaviour.

Contract-level mistakes are already caught upstream — the ODCS schema rejects minItems on a string property, so this only arises when the column itself has drifted.

On minProperties / maxProperties

I said on the issue I would do these too, and I have left them out deliberately.

They are only row-level on a map column, where the key count varies per row and ibis gives a usable .length(). On a struct the fields are fixed by the schema, so the count is the same for every row and the check is either trivially true or trivially false for the whole table — closer to the existing physical type check than to a value check.

So they are two different checks wearing one option name, and which you get depends on the physical type rather than anything the contract says. I would rather ask than guess: should minProperties apply only to map-typed columns, become a schema-level assertion on structs, or be left alone? Happy to implement whichever you prefer.

multipleOf and defaultTimezone are also still unimplemented; I skipped them here to keep this reviewable.

12 tests added, full suite green (2120 passed).

They measure the elements of one row's array, so they are invalid_count checks
like the other logicalTypeOptions, one check per constraint so a report says
which one failed.

A column the contract calls an array but the server does not cannot be measured
that way. Dropping the constraint there would report the check as passed, so it
reports an error naming the type it found instead.

uniqueItems: false is the default and asserts nothing, so it produces no check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant