Skip to content

feat(admin): upgrade sql-metadata to 3.0.1 and bound the cluster table functions - #8298

Draft
phacops wants to merge 2 commits into
claude/pf-75-admin-sql-injectionfrom
claude/snuba-sql-metadata-3
Draft

feat(admin): upgrade sql-metadata to 3.0.1 and bound the cluster table functions#8298
phacops wants to merge 2 commits into
claude/pf-75-admin-sql-injectionfrom
claude/snuba-sql-metadata-3

Conversation

@phacops

@phacops phacops commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Stacked on #8297 — review that first; this PR's diff is against it.

Bumps sql-metadata 2.11.0 → 3.0.1, pulling in sqlglot 30.16.0 (it replaced sqlparse as the backend), now that getsentry/pypi#2457 put both on the internal index.

Why the upgrade and the check ship together

On 2.11.0 the scoped admin tools were protected from table functions by accident. That version reports url, merge, clusterallreplicas and the rest in Parser.tables, so allowed_tables rejected them as unknown table names:

FROM clusterAllReplicas('c', system.users), scoped to querylog_dist
sql-metadata 2.11.0 rejected — parser reports clusterallreplicas as a table
sql-metadata 3.0.1 allowed — parser reports nothing

3.0.1 stops reporting them, so that incidental protection disappears and the explicit rejection from #8297 becomes what carries it. Upgrading without the check below would open a live path for the querylog/cardinality/outcomes tools to read any table via clusterAllReplicas.

The check

cluster/clusterAllReplicas stay permitted — fanning a read across replicas is normal in these tools, and they are the two the system-queries validator already allows. But the table they fan out over now has to clear allowed_tables. The parser cannot see that argument, so it is extracted directly, covering all three call forms ClickHouse accepts: ('name', db.table), ('name', db, table), ('name', db.table, key). A call whose table argument cannot be read is rejected rather than waved through. Tracing, which passes no allowed_tables, is unaffected.

This is also what Warden flagged on #8289 (4VG-58Z).

Note that cluster/clusterAllReplicas are not SOURCES members and carry no AccessType::REMOTE check — they are registered in TableFunctionRemote.cpp with allow_readonly = true, and the only check is SELECT on the target table. So revoking SOURCES from the admin users does not cover them, and this check is the layer that does, unless the ClickHouse grants are also narrowed per table.

Two things the upgrade surfaced

  • 3.0.1 ships py.typed, so the # type: ignore[import-untyped] on that import became unused and strict mypy rejected it. Removed.
  • The ARRAY JOIN massaging in validate_ro_query stays. I expected 3.x to make it dead code and checked before deleting: 3.0.1 still puts array in Parser.tables_aliases for a plain ARRAY JOIN, so it is still load-bearing.

The lockfile diff is 12 insertions / 4 deletions — only the two intended packages move. It was regenerated with uv 0.12.3 to match CI; an older uv rewrites markers across ~200 unrelated lines.

Testing

tests/admin passes on 3.0.1. New tests cover the cluster-function argument in all three call forms, both rejected (system.users, other_table) and allowed (my_table, with a trailing sharding key, and repeated in one query).

https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV


Generated by Claude Code

Comment thread snuba/admin/clickhouse/common.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 15cb471. Configure here.

Comment thread snuba/admin/clickhouse/common.py
@phacops
phacops force-pushed the claude/snuba-sql-metadata-3 branch from 15cb471 to 4e984a3 Compare August 13, 2026 03:44

phacops commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

On the sharding-key finding (Seer 15907458/0 and Bugbot 9237ee6a — the same claim): I don't believe this is a bug, and I haven't changed the code. Seer has marked it resolved in 4e984a3, but that is only the rebase moving the commit; the extraction is identical.

The documented signature is:

cluster(['cluster_name', db.table, sharding_key][, SETTINGS name = value, ...])
cluster(['cluster_name', db, table, sharding_key][, SETTINGS name = value, ...])

A sharding key only appears after a complete table spec. So cluster('c', my_table, mykey) is db=my_table, table=mykey — not table plus sharding key. With bare identifiers the sharding key is the fourth argument: cluster('c', db, table, mykey).

That is what the code produces, verified against this branch:

cluster('c', my_table, mykey)            -> {'my_table.mykey'}   # db.table, per the signature
cluster('c', db, tbl, mykey)             -> {'db.tbl'}           # sharding key ignored
clusterAllReplicas('c', db.tbl, rand())  -> {'db.tbl'}           # sharding key ignored

The two forms are genuinely ambiguous from syntax alone — cluster('c', a, b) looks the same either way — and the docs resolve it as db, table.

Worth being explicit about the failure direction if I have this wrong: the extracted name would not be in allowed_tables, so the query is rejected. The cost of my being wrong is an operator hitting a confusing rejection, never an unauthorized read. Given the ambiguity, that is the direction I want to fail in.

Happy to change it for a counter-example from a real instance.


Generated by Claude Code

Comment thread snuba/admin/clickhouse/common.py

phacops commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Seer 15907613 is a re-file of 15907458 after the rebase — same claim, answered above. Still not changing the parse: the documented signature is cluster(cluster_name, db, table, sharding_key), so a sharding key follows a complete table spec and cluster('c', a, b) is db=a, table=b.

One part of its suggestion was fair though — "adding a test case with a bare-identifier sharding key would help validate" — so that is now pinned rather than left to prose (07db84b):

def test_cluster_function_bare_identifiers_are_db_and_table() -> None:
    validate_ro_query("SELECT * FROM cluster('c', db, tbl)", allowed_tables={"db.tbl"})
    # And the sharding key, when present, is not part of the table.
    validate_ro_query("SELECT * FROM cluster('c', db, tbl, mykey)", allowed_tables={"db.tbl"})
    with pytest.raises(InvalidCustomQuery):
        validate_ro_query("SELECT * FROM cluster('c', db, tbl)", allowed_tables={"db"})

The two readings are indistinguishable from syntax alone, so if someone later concludes ClickHouse resolves it the other way, that test is the single place to flip — and its docstring says which reading it encodes and why.


Generated by Claude Code

claude added 2 commits August 13, 2026 03:55
Bumps sql-metadata 2.11.0 -> 3.0.1 (pulling in sqlglot 30.16.0, which
replaced sqlparse as its backend) now that getsentry/pypi#2457 has put
both on the internal index.

The upgrade and the check below belong together. On 2.11.0 the scoped
admin tools were protected from table functions by accident: Parser.tables
reported url, merge, clusterallreplicas and the rest as table names, so
allowed_tables rejected them as unknown tables. 3.0.1 stops reporting
them, so that protection disappears and the explicit rejection on the
parent branch becomes the thing carrying it.

cluster/clusterAllReplicas stay permitted -- fanning a read across
replicas is normal in these tools -- but the table they fan out over now
has to clear allowed_tables. The parser cannot see that argument, so it
is extracted directly, covering the three forms ClickHouse accepts:
('name', db.table), ('name', db, table), and ('name', db.table, key). A
call whose table argument cannot be read is rejected rather than waved
through. Tracing, which passes no allowed_tables, is unaffected.

Also drops the type: ignore[import-untyped] on the sql_metadata import:
3.0.1 ships py.typed, so the ignore is unused and strict mypy rejects it.

The ARRAY JOIN massaging in validate_ro_query stays. 3.0.1 still puts
`array` in Parser.tables_aliases for a plain ARRAY JOIN, so that block is
still load-bearing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV
cluster('c', a, b) is db=a table=b, not table=a sharding_key=b: ClickHouse
documents cluster(cluster_name, db, table, sharding_key), so a sharding
key only follows a complete table spec. The two readings are
indistinguishable from syntax alone, and two review bots read it the other
way, so pin the documented one rather than leave it to prose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAWCwVv23LnmaGADQSuQgV
@phacops
phacops force-pushed the claude/snuba-sql-metadata-3 branch from 07db84b to c314bed Compare August 13, 2026 03:55
@phacops
phacops marked this pull request as draft August 13, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants