Repository navigation
fix(sqlalchemy-spanner): reflect vector columns and unrecognized column types - #18844
rocksnow1942 wants to merge 2 commits into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
…mn types _designate_type() raised KeyError for vector columns, whose type ends with the vector length (ARRAY<FLOAT32>(vector_length=>128)), and for types it doesn't map, like PROTO and ENUM columns, which broke the reflection of the whole schema. Parse the element type of vector columns, and reflect unrecognized types as NullType with a warning. Fixes googleapis#18839
06d25f6 to
21375ae
Compare
There was a problem hiding this comment.
Code Review
This pull request enhances the Spanner dialect's type designation to support vector columns and gracefully handle unrecognized types like PROTO and ENUM by reflecting them as NullType with a warning. Unit and integration tests have been added to verify these changes. Feedback on the changes highlights a potential robustness issue where using str_repr.index('>') could raise a ValueError and crash schema reflection if the character is missing; using find with a fallback is recommended.
… NullType
str_repr.index(">") raised ValueError for them, which would break the
reflection of the whole schema.
_designate_type()raisedKeyErrorfor column types it doesn't map. Sinceget_multi_columns()reads all the tables of a schema in one query, one such column breaks the reflection of the whole schema. These column types fail on the emulator (1.5.58):SPANNER_TYPEARRAY<FLOAT32>(vector_length=>3)KeyError: 'FLOAT32>(vector_length=>3'`my.package.Message`KeyError: '`my.package.Message`'`my.package.Status`KeyError: '`my.package.Status`'(The issue also mentions
INTERVALandSTRUCT, but neither can be a column type.)This PR:
>, so vector columns are reflected asARRAY(REAL)orARRAY(Float);NullTypewith anSAWarning, as SQLAlchemy's built-in dialects do. Arrays of such types are reflected asARRAY(NullType()).The
NullTypefallback was first proposed by @waiho-gumloop in #16622, which was moved to draft after #17911 fixed itsTOKENLISTpart. This PR supersedes it and adds the vector, proto and enum cases.Tested with the unit and mockserver tests and the compliance suite against the emulator.
Fixes #18839