feat(admin): upgrade sql-metadata to 3.0.1 and bound the cluster table functions - #8298
feat(admin): upgrade sql-metadata to 3.0.1 and bound the cluster table functions#8298phacops wants to merge 2 commits into
Conversation
f409160 to
c1235a3
Compare
f0183a7 to
15cb471
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 15cb471. Configure here.
15cb471 to
4e984a3
Compare
|
On the sharding-key finding (Seer The documented signature is: cluster(['cluster_name', db.table, sharding_key][, SETTINGS name = value, ...])
cluster(['cluster_name', db, table, sharding_key][, SETTINGS name = value, ...])A sharding key only appears after a complete table spec. So That is what the code produces, verified against this branch: The two forms are genuinely ambiguous from syntax alone — Worth being explicit about the failure direction if I have this wrong: the extracted name would not be in Happy to change it for a counter-example from a real instance. Generated by Claude Code |
|
Seer One part of its suggestion was fair though — "adding a test case with a bare-identifier sharding key would help validate" — so that is now pinned rather than left to prose (07db84b): def test_cluster_function_bare_identifiers_are_db_and_table() -> None:
validate_ro_query("SELECT * FROM cluster('c', db, tbl)", allowed_tables={"db.tbl"})
# And the sharding key, when present, is not part of the table.
validate_ro_query("SELECT * FROM cluster('c', db, tbl, mykey)", allowed_tables={"db.tbl"})
with pytest.raises(InvalidCustomQuery):
validate_ro_query("SELECT * FROM cluster('c', db, tbl)", allowed_tables={"db"})The two readings are indistinguishable from syntax alone, so if someone later concludes ClickHouse resolves it the other way, that test is the single place to flip — and its docstring says which reading it encodes and why. Generated by Claude Code |
Bumps sql-metadata 2.11.0 -> 3.0.1 (pulling in sqlglot 30.16.0, which replaced sqlparse as its backend) now that getsentry/pypi#2457 has put both on the internal index. The upgrade and the check below belong together. On 2.11.0 the scoped admin tools were protected from table functions by accident: Parser.tables reported url, merge, clusterallreplicas and the rest as table names, so allowed_tables rejected them as unknown tables. 3.0.1 stops reporting them, so that protection disappears and the explicit rejection on the parent branch becomes the thing carrying it. cluster/clusterAllReplicas stay permitted -- fanning a read across replicas is normal in these tools -- but the table they fan out over now has to clear allowed_tables. The parser cannot see that argument, so it is extracted directly, covering the three forms ClickHouse accepts: ('name', db.table), ('name', db, table), and ('name', db.table, key). A call whose table argument cannot be read is rejected rather than waved through. Tracing, which passes no allowed_tables, is unaffected. Also drops the type: ignore[import-untyped] on the sql_metadata import: 3.0.1 ships py.typed, so the ignore is unused and strict mypy rejects it. The ARRAY JOIN massaging in validate_ro_query stays. 3.0.1 still puts `array` in Parser.tables_aliases for a plain ARRAY JOIN, so that block is still load-bearing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV
cluster('c', a, b) is db=a table=b, not table=a sharding_key=b: ClickHouse
documents cluster(cluster_name, db, table, sharding_key), so a sharding
key only follows a complete table spec. The two readings are
indistinguishable from syntax alone, and two review bots read it the other
way, so pin the documented one rather than leave it to prose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV
07db84b to
c314bed
Compare

Stacked on #8297 — review that first; this PR's diff is against it.
Bumps
sql-metadata2.11.0 → 3.0.1, pulling insqlglot30.16.0 (it replacedsqlparseas the backend), now that getsentry/pypi#2457 put both on the internal index.Why the upgrade and the check ship together
On 2.11.0 the scoped admin tools were protected from table functions by accident. That version reports
url,merge,clusterallreplicasand the rest inParser.tables, soallowed_tablesrejected them as unknown table names:FROM clusterAllReplicas('c', system.users), scoped toquerylog_distclusterallreplicasas a table3.0.1 stops reporting them, so that incidental protection disappears and the explicit rejection from #8297 becomes what carries it. Upgrading without the check below would open a live path for the querylog/cardinality/outcomes tools to read any table via
clusterAllReplicas.The check
cluster/clusterAllReplicasstay permitted — fanning a read across replicas is normal in these tools, and they are the two the system-queries validator already allows. But the table they fan out over now has to clearallowed_tables. The parser cannot see that argument, so it is extracted directly, covering all three call forms ClickHouse accepts:('name', db.table),('name', db, table),('name', db.table, key). A call whose table argument cannot be read is rejected rather than waved through. Tracing, which passes noallowed_tables, is unaffected.This is also what Warden flagged on #8289 (4VG-58Z).
Note that
cluster/clusterAllReplicasare notSOURCESmembers and carry noAccessType::REMOTEcheck — they are registered inTableFunctionRemote.cppwithallow_readonly = true, and the only check isSELECTon the target table. So revokingSOURCESfrom the admin users does not cover them, and this check is the layer that does, unless the ClickHouse grants are also narrowed per table.Two things the upgrade surfaced
py.typed, so the# type: ignore[import-untyped]on that import became unused and strict mypy rejected it. Removed.validate_ro_querystays. I expected 3.x to make it dead code and checked before deleting: 3.0.1 still putsarrayinParser.tables_aliasesfor a plainARRAY JOIN, so it is still load-bearing.The lockfile diff is 12 insertions / 4 deletions — only the two intended packages move. It was regenerated with uv 0.12.3 to match CI; an older uv rewrites markers across ~200 unrelated lines.
Testing
tests/adminpasses on 3.0.1. New tests cover the cluster-function argument in all three call forms, both rejected (system.users,other_table) and allowed (my_table, with a trailing sharding key, and repeated in one query).https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV
Generated by Claude Code