[Format][FlightSQL] Advertise supported Substrait dialect via SqlInfo / GetInfo #53872

Describe the enhancement requested

Flight SQL and ADBC can already say whether a backend accepts Substrait and which versions it accepts. They can’t say which part of Substrait it accepts. In practice no consumer implements the whole spec, so producers find out a plan is unsupported only when execution fails.

Substrait now has a standard format for this: dialect files. A dialect is a YAML document listing the types, relations, expressions and function signatures a system supports, plus optional mapping metadata (e.g. the engine’s own function names). For example, substrait-java already publishes one for its Spark integration (spark_dialect.yaml).

Flight SQL and ADBC should be able to advertise the exact dialect supported on a connection. The biggest issue here is the size, though. A dialect listing every function impl in the standard Substrait extensions (207 functions, 507 impls) plus all types, relations and expressions comes to about 34 KiB / ~1,900 lines of YAML, or about 3.6 KiB gzipped. Real dialects are subsets of this. That’s small enough to return inline once per connection, but can probably cause headaches for a particularly chatty client. So the proposal is to add two separate info codes Substrait Dialect (SQL_SUBSTRAIT_DIALECT) and it's hash (SQL_SUBSTRAIT_DIALECT_HASH). The hash is simply there to allow clients to implement some sort of a client-side cache if necessary.

I'm thinking of putting these in [500, 1000) syntax codes range because they are describing substrait syntax similar to othere sql syntax codes in the same range. If you think I should move them up with vendor info codes I can do that, though that would mean the change would apply only to Flight SQL here and we will need to add them to adbc separately.

Component(s)

Format