From 9681f62d914e179cba3ef5bafa343f1b29452d43 Mon Sep 17 00:00:00 2001 From: Andrey Zvonov <32552679+zvonand@users.noreply.github.com> Date: Sat, 24 Oct 2026 13:19:28 +0200 Subject: [PATCH 1/3] oauth-docs --- docs/concepts/features/interfaces/client.mdx | 44 +- .../external-authenticators/index.mdx | 2 +- .../external-authenticators/tokens.md | 477 ------------------ docs/en/antalya/oauth.md | 363 +++++++++++++ .../engines/database-engines/datalake.mdx | 2 +- 5 files changed, 367 insertions(+), 521 deletions(-) delete mode 100644 docs/concepts/features/security/external-authenticators/tokens.md create mode 100644 docs/en/antalya/oauth.md diff --git a/docs/concepts/features/interfaces/client.mdx b/docs/concepts/features/interfaces/client.mdx index 4a8a37698dd15..758f9a11e0078 100644 --- a/docs/concepts/features/interfaces/client.mdx +++ b/docs/concepts/features/interfaces/client.mdx @@ -832,10 +832,10 @@ All command-line options can be specified directly on the command line or as def | `-d [ --database ] ` | Select the database to default to for this connection. | The current database from the server settings (`default` by default) | | `-h [ --host ] ` | The hostname of the ClickHouse server to connect to. Can either be a hostname or an IPv4 or IPv6 address. Multiple hosts can be passed via multiple arguments. | `localhost` | | `--jwt ` | Use JSON Web Token (JWT) for authentication.

Server JWT authorization is only available in ClickHouse Cloud. | - | -| `--jwt-command ` | Shell command whose stdout is used as the JWT. Invoked on the first connect, before reconnects when the cached JWT is near expiry, and after the server rejects the cached token. See [`--jwt-command` details](#jwt-command-details) below. | - | +| `--jwt-command ` | Shell command whose stdout is used as the JWT. Invoked on the first connect, before reconnects when the cached JWT is near expiry, and after the server rejects the cached token. See [`--jwt-command` details](/antalya/oauth#jwt-command-details). | - | | `--jwt-command-timeout ` | Timeout for `--jwt-command`. Also settable as `` in the config file; CLI wins. | `30` | | `--login[=]` | Authenticate via OAuth2. Bare `--login` (no `=`) triggers ClickHouse Cloud automatic login — the provider is inferred from the server. To authenticate against a custom OpenID Connect provider, supply a `mode` and `--oauth-credentials`: `--login=browser` runs the Authorization Code + PKCE flow (opens a browser), `--login=device` runs the Device Authorization flow (prints a URL and short code — no browser needed). | - | -| `--oauth-credentials ` | Path to an OAuth2 credentials JSON file (Google Cloud Console format). Required when using `--login=browser` or `--login=device` with a custom OpenID Connect provider. See [OAuth credentials file format](#oauth-credentials-file) below. Refresh tokens are cached in `~/.clickhouse-client/oauth_cache.json` (mode `0600`). | `~/.clickhouse-client/oauth_client.json` | +| `--oauth-credentials ` | Path to an OAuth2 credentials JSON file (Google Cloud Console format). Required when using `--login=browser` or `--login=device` with a custom OpenID Connect provider. See [OAuth credentials file format](/antalya/oauth#oauth-credentials-file). Refresh tokens are cached in `~/.clickhouse-client/oauth_cache.json` (mode `0600`). | `~/.clickhouse-client/oauth_client.json` | | `--no-warnings` | Disable showing warnings from `system.warnings` when the client connects to the server. | - | | `--no-server-client-version-message` | Suppress server-client version mismatch message when the client connects to the server. | - | | `--password ` | The password of the database user. You can also specify the password for a connection in the configuration file. If you do not specify the password, the client will ask for it. | - | @@ -850,46 +850,6 @@ All command-line options can be specified directly on the command line or as def Instead of the `--host`, `--port`, `--user` and `--password` options, the client also supports [connection strings](#connection_string). -### OAuth credentials file {#oauth-credentials-file} - -When using `--login=browser` or `--login=device` with a custom OpenID Connect provider, the client reads a credentials JSON file. The file uses the same format produced by the Google Cloud Console ("OAuth 2.0 Client IDs" → "Download JSON"): - -```json -{ - "installed": { - "client_id": "YOUR_CLIENT_ID", - "client_secret": "YOUR_CLIENT_SECRET", - "auth_uri": "https://accounts.google.com/o/oauth2/auth", - "token_uri": "https://oauth2.googleapis.com/token", - "redirect_uris": ["http://127.0.0.1"] - } -} -``` - -The top-level key can be `installed` (desktop/CLI apps) or `web`. Required fields: `client_id`, `auth_uri`, `token_uri`. Optional fields: - -| Field | Description | -|---|---| -| `client_secret` | Confidential-client secret. Omit (or leave empty) for OIDC public clients — the auth-code flow is always protected by PKCE and the device flow by the device code, so a secret is not required by the protocol. When the field is absent the client never sends a `client_secret` form parameter, which is the form public-client registrations require (Auth0, Microsoft Entra ID, Keycloak, Okta and others reject empty secrets with `invalid_client`). | -| `device_authorization_uri` | Device authorization endpoint. Discovered automatically via OIDC Discovery if absent. | -| `issuer` | OIDC issuer URL (e.g. `https://accounts.google.com`). Used to locate the discovery document when `device_authorization_uri` is not set. | - -The default path is `~/.clickhouse-client/oauth_client.json`. Override it with `--oauth-credentials `. - -After a successful login the obtained refresh token is cached in `~/.clickhouse-client/oauth_cache.json` (file mode `0600`). Subsequent runs reuse the cached token silently and only open the browser or print a device code when the refresh token has expired. - -### `--jwt-command` details {#jwt-command-details} - -The command is executed via `/bin/sh -c`. Stdout is taken as the JWT (one trailing newline stripped); any human-facing output (prompts, URLs, device codes) must go to stderr — it is forwarded unbuffered to the client's stderr. Stdin is closed. - -The command runs on the first connect to obtain the initial token. On subsequent (re)connects the client reuses the cached token; it re-invokes the command only when (a) the cached token parses as a JWT whose `exp` claim is within 30 seconds, or (b) the server rejects the cached token with an authentication failure, in which case the client refetches the token and retries the handshake once. Opaque tokens (anything that does not parse as a JWT) and JWTs without a usable `exp` claim are reused until the server rejects them — caching/refresh in those cases is the script's responsibility. - -```bash -clickhouse-client --jwt-command "curl -sS https://idp.example/token | jq -r .access_token" -``` - -Cannot be combined with `--jwt`, `--login`, or a non-default `--user`. Non-zero exit, empty output, or exceeding `--jwt-command-timeout` (default `30`s, overridable via `` in `~/.clickhouse-client/config.xml`) fails authentication. On timeout the entire helper subprocess tree is terminated. - ### Query options {#command-line-options-query} | Option | Description | diff --git a/docs/concepts/features/security/external-authenticators/index.mdx b/docs/concepts/features/security/external-authenticators/index.mdx index d3f85d352f690..76da85252b827 100644 --- a/docs/concepts/features/security/external-authenticators/index.mdx +++ b/docs/concepts/features/security/external-authenticators/index.mdx @@ -15,4 +15,4 @@ The following external authenticators and directories are supported: - [SSL X.509 authentication](/concepts/features/security/external-authenticators/ssl-x509) - HTTP [Authenticator](/concepts/features/security/external-authenticators/http) - [JWT authentication](/concepts/features/security/external-authenticators/jwt) -- Token-based [Authenticator](/concepts/features/security/external-authenticators/tokens) \ No newline at end of file +- Token-based [Authenticator](/antalya/oauth) \ No newline at end of file diff --git a/docs/concepts/features/security/external-authenticators/tokens.md b/docs/concepts/features/security/external-authenticators/tokens.md deleted file mode 100644 index 3b61825b9f06e..0000000000000 --- a/docs/concepts/features/security/external-authenticators/tokens.md +++ /dev/null @@ -1,477 +0,0 @@ ---- -slug: /en/operations/external-authenticators/oauth -title: "Token-based authentication" ---- -import SelfManaged from '@site/docs/en/_snippets/_self_managed_only_no_roadmap.md'; - - - -ClickHouse users can be authenticated using tokens. This works in two ways: - -- An existing user (defined in `users.xml` or in local access control paths) can be authenticated with a token if this user can be `IDENTIFIED WITH jwt`. -- Use the information from the token or from an external Identity Provider (IdP) as a source of user definitions and allow locally undefined users to be authenticated with a valid token. - -Although not all tokens are JWTs, under the hood both ways are treated as the same authentication method to maintain better compatibility. - -# Token Processors - -## Configuration - -Token-based authentication is enabled by default. To disable it, set `enable_token_auth` to `0` in `config.xml`: - -```xml -0 -``` - -When disabled, token processors are not parsed, TokenAccessStorage is not available, and authentication via tokens (`--jwt` option or `Authorization: Bearer` header) is rejected. - -To use token-based authentication, add `token_processors` section to `config.xml` and define at least one token processor in it. -Its contents are different for different token processor types. - -**Common parameters** -- `type` -- type of token processor. Supported values: `jwt_static_key`, `jwt_static_jwks`, `jwt_dynamic_jwks`, `entra` (`azure` is accepted as a back-compat alias and resolves to the same `entra` processor — see the [Entra](#entra) section), `google`, `openid`. Mandatory. Case-insensitive. -- `token_cache_lifetime` -- maximum lifetime of cached token (in seconds). Optional, default: 3600. -- `username_claim` -- name of claim (field) that will be treated as ClickHouse username. Optional, default: "sub". -- `groups_claim` -- name of claim (field) that contains list of groups user belongs to. This claim will be looked up in the token itself (in case token is a valid JWT, e.g. in Keycloak) or in response from `/userinfo`. Optional, default: "groups". - -For each type, there are additional specific parameters (some of them are mandatory). -If some parameters that are not required for current processor type are specified, they are ignored. - -## JWT (JSON Web Token) - -JWT itself is a source of information about user. -It is decoded locally and its integrity is verified using either a local static key or JWKS (JSON Web Key Set), local or remote. - -### JWT with static key: -```xml - - - - jwt_static_key - HS256 - my_static_secret - - - -``` -**Parameters:** -- `algo` - Algorithm for signature validation. Mandatory. Supported values: - - | HMAC | RSA | ECDSA | PSS | EdDSA | - |-------| ----- | ------ | ----- | ------- | - | HS256 | RS256 | ES256 | PS256 | Ed25519 | - | HS384 | RS384 | ES384 | PS384 | Ed448 | - | HS512 | RS512 | ES512 | PS512 | | - | | | ES256K | | | - Also supports None (not recommended and must *NEVER* be used in production). -- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional. -- `static_key` - key for symmetric algorithms. Mandatory for `HS*` family algorithms. -- `static_key_in_base64` - indicates if the `static_key` key is base64-encoded. Optional, default: `False`. -- `public_key` - public key for asymmetric algorithms. Mandatory except for `HS*` family algorithms and `None`. -- `private_key` - private key for asymmetric algorithms. Optional. -- `public_key_password` - public key password. Optional. -- `private_key_password` - private key password. Optional. -- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional. -- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional. -- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`. - -### JWT with static JWKS -```xml - - - - jwt_static_jwks - {"keys": [{"kty": "RSA", "alg": "RS256", "kid": "mykid", "n": "_public_key_mod_", "e": "AQAB"}]} - - - -``` - -**Parameters:** - -- `static_jwks` - content of JWKS in JSON -- `static_jwks_file` - path to a file with JWKS -- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional. -- `verifier_leeway` - Clock skew tolerance (seconds). Useful for handling small differences in system clocks between ClickHouse and the token issuer. Optional. -- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional. -- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional. -- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`. - -:::note -Only one of `static_jwks` or `static_jwks_file` keys must be present in one verifier -::: - -:::note -For JWKS-based validators (`jwt_static_jwks` and `jwt_dynamic_jwks`), RS* and ES* family algorithms are supported. -::: - -### JWT with remote JWKS -```xml - - - - jwt_dynamic_jwks - http://localhost:8000/.well-known/jwks.json - 3600 - - - -``` - -**Parameters:** - -- `uri` - JWKS endpoint. Mandatory. -- `jwks_cache_lifetime` - Period for resend request for refreshing JWKS. Optional, default: 3600. -- `claims` - A string containing a JSON object that should be contained in the token payload. If this parameter is defined, token without corresponding payload will be considered invalid. Optional. -- `verifier_leeway` - Clock skew tolerance (seconds). Useful for handling small differences in system clocks between ClickHouse and the token issuer. Optional. -- `expected_issuer` - Expected value of the `iss` (issuer) claim in the JWT. If specified, tokens with a different issuer will be rejected. Optional. -- `expected_audience` - Expected value of the `aud` (audience) claim in the JWT. If specified, tokens with a different audience will be rejected. Optional. -- `allow_no_expiration` - If `true`, tokens without the `exp` (expiration) claim are accepted. Otherwise they are rejected. Optional, default: `false`. - - -## IdP-specific presets and generic external providers - -This section covers per-IdP processors (`entra`, built on top of the generic JWT processors, and `google`), and the generic `openid` processor that talks to an arbitrary OIDC-compliant identity provider. - -:::note -If the IdP issues access tokens that follow [RFC 9068](https://datatracker.ietf.org/doc/html/rfc9068) (the *JSON Web Token Profile for OAuth 2.0 Access Tokens*), the access token is itself a verifiable JWT and is best handled by one of the JWT processors above (typically `jwt_dynamic_jwks`) — no `/userinfo` or `/tokeninfo` round-trip is needed. The processors in this section exist for IdPs whose access tokens are opaque (e.g. Google), or whose JWT access tokens you prefer to validate by asking the IdP rather than locally. -::: - -### Entra (Microsoft Entra ID, pure OIDC) {#entra} - -`entra` is a preset for Microsoft Entra ID built on top of `jwt_dynamic_jwks`. Tokens are validated **locally** against Entra's per-tenant JWKS — no Microsoft Graph call, no userinfo round trip, no OIDC discovery fetch. `username_claim` and `groups_claim` are read directly from the JWT payload. Use this when the access token's `aud` is your own app (registered via Entra's *Expose an API* blade), not `https://graph.microsoft.com`. - -:::note Migrating from the legacy `azure` processor -`azure` is now an **alias** for `entra` — at config-parse time the type string is rewritten and the rest of the pipeline is identical. The previous `azure` implementation (which round-tripped every token through Microsoft Graph's `/oidc/userinfo` and `/v1.0/me/memberOf` endpoints) has been removed entirely. - -For operators upgrading: an `azure` block that previously had no other parameters will now fail to load with `'tenant_id' must be specified for 'entra' processor`. To migrate, add `` (and ideally ``) and make sure your application is configured to mint tokens whose `aud` is your own app, not Microsoft Graph. The setup recipe lives in `docs/entra-setup-draft.md`. -::: - -Minimum configuration — only `tenant_id` is required; all other parameters have sensible defaults: - -```xml - - - - entra - aaaabbbb-0000-cccc-1111-dddd2222eeee - - - -``` - -Example with common overrides (audience binding to a specific app, Entra-flavored username/groups claims): - -```xml - - entra - aaaabbbb-0000-cccc-1111-dddd2222eeee - api://clickhouse - preferred_username - roles - -``` - -**Parameters:** - -- `tenant_id` — Microsoft Entra tenant identifier (a GUID, or an `*.onmicrosoft.com` domain). **Mandatory.** Multi-tenant aliases (`common`, `organizations`, `consumers`) are rejected because `JwksJwtProcessor` does exact-match issuer validation. - -All remaining parameters are optional: - -- `jwks_uri` — Override for the JWKS endpoint. Default: `https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys`. Override only for sovereign clouds (`login.microsoftonline.us`, `login.partner.microsoftonline.cn`). -- `expected_issuer` — Expected value of the `iss` claim. Default: `https://login.microsoftonline.com/{tenant_id}/v2.0` (derived from `tenant_id`). Override for v1.0 tokens (`https://sts.windows.net/{tenant_id}/`) or sovereign clouds. -- `expected_audience` — Expected value of the `aud` claim, normally your app's Application ID URI (e.g. `api://clickhouse`) or client ID. If unset, no audience check is performed (any signature-valid token from the tenant will authenticate); a warning is logged at startup so the gap is visible. -- `username_claim` — JWT claim to use as the ClickHouse username. Default: `sub`. Common Entra alternatives: `preferred_username`, `upn`, `oid`. -- `groups_claim` — JWT claim that carries the array of group identifiers. Default: `groups`. Set to `roles` when using App Roles. See [Mapping groups to ClickHouse roles](#entra-group-mapping) for how to get human-readable values instead of GUIDs. -- `expected_typ`, `verifier_leeway`, `jwks_cache_lifetime`, `claims`, `allow_no_expiration`, `token_cache_lifetime` — Same as for `jwt_dynamic_jwks`. - -#### Mapping groups to ClickHouse roles {#entra-group-mapping} - -By default the `groups` claim contains group **object IDs (GUIDs)**, not names. Three ways to surface human-readable identifiers, in order of preference: - -**Option A — App Roles** (recommended) - -Operator-chosen role strings in a separate `roles` claim. Compact even for users in many groups (no `hasgroups` overage indicator), and immune to Entra-side group renames. - -1. App registration → **App roles** → **Create app role**. Set `Value` to the string ClickHouse should receive (e.g. `ch_admin`); `Allowed member types` = `Users/Groups`. -2. Enterprise application → **Properties** → `Assignment required` = **Yes**. -3. Enterprise application → **Users and groups** → assign each user or security group to a role. Group assignment requires Entra ID P1/P2; free-tier tenants can only assign individual users here. -4. On the processor: `roles`. - -**Option B — Format the `groups` claim** - -Names emitted in the existing `groups` claim. Works on free tier; useful when group membership is already maintained in Entra and a separate role-assignment surface is not wanted. - -Prerequisites in the app registration: - -- `"groupMembershipClaims": "ApplicationGroup"` (or `"SecurityGroup"` for tenant-wide). -- `optionalClaims.accessToken` entry for `groups` with `additionalProperties` set to one or more of: - -| Value | Effect | -|---|---| -| `sam_account_name` | On-prem-synced groups emit as `sAMAccountName`. | -| `dns_domain_and_sam_account_name` | On-prem-synced groups emit as `DOMAIN\sAMAccountName`. | -| `cloud_displayname` | Cloud-only groups emit their `displayName`. | - -Entra picks per group; groups not covered by a chosen format still emit as GUIDs. Display names are mutable — a rename in Entra silently breaks the mapping until config is updated. - -Leave `groups` (the default). - -**Option C — `roles_mapping`** - -Keep GUIDs in the token and translate them in the user-directory config (see [Identity Provider as an External User Directory](#idp-external-user-directory)). Always works, including on free tier. Tedious for many groups but immune to renames. - -:::note -When switching from GUIDs to names, retune any `roles_filter` regex — for example `\bclickhouse-[a-zA-Z0-9]+\b` will not match strings like `ch_admin`. -::: - -### Google {#google} - -`google` validates Google OAuth 2.0 access tokens. Google access tokens are opaque, so each token is checked against Google's `/tokeninfo` endpoint; `/userinfo` is additionally called only when `username_claim` is not returned by `/tokeninfo`. `expected_audience` and `username_claim` refer to fields of these endpoints' JSON responses, not of the token itself. Clients pass the access token via `--jwt` or the `Authorization: Bearer` header. - -Minimum configuration — no parameters are mandatory, but `expected_audience` should be set: - -```xml - - - - google - 1234567890-abcdef.apps.googleusercontent.com - - - -``` - -**Parameters:** - -- `expected_audience` — OAuth client ID the token must be issued to (the `aud` field returned by `/tokeninfo`). If unset, any valid Google access token is accepted regardless of the client it was issued to; a warning is logged at startup so the gap is visible. -- `username_claim` — Field to use as the ClickHouse username. Default: `sub` (the stable Google account ID). Common alternative: `email` (requires the `email` scope). Other fields (e.g. `hd`, or `name` with the `profile` scope) are read from `/userinfo`, which costs an extra request. -- `token_cache_lifetime` — Same as for other processors. - -#### Mapping groups to ClickHouse roles {#google-group-mapping} - -When the token carries the user's email, the processor looks up the user's direct Google Groups memberships via the Cloud Identity API and returns group email addresses as external groups. Requirements: - -- The token must include the `https://www.googleapis.com/auth/cloud-identity.groups.readonly` scope. -- The Cloud Identity API must be enabled in the Google Cloud project that owns the OAuth client. - -Only direct memberships are returned; groups inherited through nested groups are not. If the lookup fails, the user is still authenticated, without external groups. - -Since group identifiers are email addresses, use `roles_filter` and `roles_transform` to turn them into role names. For example, the following maps `ch-analysts@example.com` to the role `ch-analysts`: - -```xml - - - google_prod - ch-[a-z-]+@example\.com - s/@.*// - - -``` - -### OpenID - -The `openid` processor speaks the OIDC protocol surface — `/userinfo` for identity, plus (when discovered or configured) the local JWT fast-path against the IdP's JWKS and RFC 7662 token introspection. Two mutually-exclusive configuration shapes: - -- **Discovery** — point `configuration_endpoint` at `.well-known/openid-configuration`. Endpoints and the issuer are resolved from the doc. When it advertises `jwks_uri`, JWT access tokens (RFC 9068) are validated locally. When it advertises `introspection_endpoint` and you supply `introspection_client_id`/`introspection_client_secret`, RFC 7662 introspection runs on each authentication — alongside the JWT fast-path if both are available, since JWT validates signature/`exp` while introspection adds the revocation check. - -- **Manual** — `userinfo_endpoint` is mandatory. For RFC 9068 JWT access tokens prefer `jwt_dynamic_jwks`. Add `token_introspection_endpoint` + `introspection_client_id` + `introspection_client_secret` for RFC 7662 liveness, expiry, and `iss`/`aud` enforcement; without them, manual mode is `/userinfo` only. - -```xml - - - - openid - https://idp.example.com/.well-known/openid-configuration - my-clickhouse-client-id - clickhouse-rs - ... - - - - openid - https://idp.example.com/userinfo - https://idp.example.com/introspect - clickhouse-rs - ... - https://idp.example.com - clickhouse-rs - - - -``` - -:::note Parser rules -- `configuration_endpoint` and `userinfo_endpoint` are mutually exclusive. -- `jwks_uri` is rejected in both shapes — use `jwt_dynamic_jwks` for an explicit JWKS URL. -- `introspection_client_id` and `introspection_client_secret` must be set together; both honor `from_env=` / `from_zk=` for secrets handling. -- In manual mode, `expected_issuer` / `expected_audience` are accepted only when introspection is wired (`/userinfo` carries neither claim and so cannot enforce them). -::: - -#### Setting up the introspection client at your IdP - -Introspection needs an OAuth client representing ClickHouse-as-resource-server — separate from any user-facing client app, with no redirect URIs. - -| IdP | RFC 7662 introspection | How to create the introspection client | -|---|---|---| -| **Keycloak** | Yes | Realm → Clients → confidential client with *Service Accounts* enabled; copy `client_id` and the secret from the *Credentials* tab | -| **Okta** | Yes (Org AS + Custom AS) | Admin → Applications → Create App Integration → *API Services* | -| **Auth0** | Not for opaque user tokens | Auth0 does not provide `/introspect` for the opaque tokens issued at the `/userinfo` audience; for custom-API JWT access tokens use `jwt_dynamic_jwks` instead | -| **Google**, **GitHub**, **Microsoft Entra ID** (MS Graph) | No | No RFC 7662 endpoint — use the provider-specific processor (`google`) or JWT validation against your own API's tokens (`entra`, `jwt_dynamic_jwks`) | - -#### Parameters - -*Discovery mode:* -- `configuration_endpoint` — URI of the OIDC configuration document. Mandatory. -- `expected_issuer` — Expected `iss`. Enforced via the JWT fast-path or RFC 7662 introspection (whichever the discovery doc surfaces); also anchors the discovery doc's own `issuer` field. Optional. -- `expected_audience` — Expected `aud`. Same enforcement scope as `expected_issuer`. Optional. -- `introspection_client_id`, `introspection_client_secret` — `client_secret_basic` credentials for the introspection endpoint. Both must be set together. Optional; required only if you want introspection enabled. -- `allow_no_expiration` — Accept JWTs without `exp` on the JWT fast-path. Optional, default `false`. -- `verifier_leeway` — Clock-skew tolerance (seconds) for the JWT fast-path. Optional, default 60. -- `jwks_cache_lifetime` — JWKS refresh interval. Optional, default 3600. -- `allow_http_discovery_urls` — Allow non-HTTPS URLs returned by the discovery document. Optional, default `false`. - -*Manual mode:* -- `userinfo_endpoint` — URI of the OIDC userinfo endpoint. Mandatory. -- `token_introspection_endpoint` — URI of an RFC 7662 introspection endpoint. Optional; when set together with introspection credentials, enables liveness, `exp`, and `iss`/`aud` enforcement. -- `introspection_client_id`, `introspection_client_secret` — As above. Required iff `token_introspection_endpoint` is set. -- `expected_issuer`, `expected_audience` — Accepted only when introspection is wired; enforced against the introspection response. Optional. - -If the IdP issues access tokens that follow [RFC 9068](https://datatracker.ietf.org/doc/html/rfc9068), prefer `jwt_dynamic_jwks` for direct local validation. The `openid` processor is for opaque tokens (via userinfo and/or introspection) and for cases where you want to consult the IdP rather than validate locally. - -### Tokens cache -To reduce number of requests to IdP, tokens are cached internally for a maximum period of `token_cache_lifetime` seconds. -If token expires sooner than `token_cache_lifetime`, then cache entry for this token will only be valid while token is valid. -If token lifetime is longer than `token_cache_lifetime`, cache entry for this token will be valid for `token_cache_lifetime`. - -## Forwarding the token to external services {#token-forwarding} - -By default, the authenticated token is not retained in the session for forwarding. Set -`enable_token_forwarding` to `1` in `config.xml` to retain it for external-service authentication: - -```xml -1 -``` - -The setting is hot-reloadable and defaults to `false`. To use it with Iceberg REST or Glue, enable -`oauth_forward_user_token` on the [`DataLakeCatalog`](/engines/database-engines/datalakecatalog) -database. See [catalog token forwarding](/engines/database-engines/datalakecatalog#user-token-forwarding) -for configuration and examples. - -:::danger Restrict `CREATE DATABASE` -Database creators choose the endpoints that receive users' tokens. Grant `CREATE DATABASE` only -to trusted users and restrict `remote_url_allow_hosts` for catalog and token-exchange endpoints. -::: - -The forwarded token is sent to the catalog and token-exchange endpoints and, in passthrough mode, is -also valid for ClickHouse itself. Use HTTPS for these endpoints in production; an HTTP endpoint is -suitable only for local testing. - -Only the session's verified token is forwarded. It is not inherited by `EXECUTE AS` or -`DEFINER` views, forwarded to other ClickHouse nodes, or persisted to disk. - -HTTP requests authenticate separately, so a rotated token takes effect on the next request. -Native TCP sessions must reconnect with the new token. - -With `async_insert = 1`, each queued batch retains its token until the flush completes, even with -`wait_for_async_insert = 0`. Different tokens use separate batches; rotation does not change a -queued batch's token. - -## Enabling token authentication for a user in `users.xml` {#enabling-jwt-auth-in-users-xml} - -In order to enable token-based authentication for the user, specify `jwt` section instead of `password` or other similar sections in the user definition. - -Parameters: -- `claims` - An optional string containing a json object that should be contained in the token payload. - -Example (goes into `users.xml`): -```xml - - - - {"resource_access":{"account": {"roles": ["view-profile"]}}} - - - -``` - -Here, the JWT payload must contain `["view-profile"]` on path `resource_access.account.roles`, otherwise authentication will not succeed even with a valid JWT. - -:::note -Per-user `claims` are enforced only when the token is a JWT (validated by a JWT processor such as `jwt_static_key`, `jwt_dynamic_jwks`, or `entra`). When the user authenticates with an opaque (access) token (e.g. via OpenID or Google token processors), claims are not checked and authentication succeeds if the token is otherwise valid. -::: - -``` -{ -... - "resource_access": { - "account": { - "roles": ["view-profile"] - } - }, -... -} -``` - -:::note -A user cannot have JWT authentication together with any other authentication method. The presence of any other sections like `password` alongside `jwt` will force ClickHouse to shut down. -::: - -## Enabling token authentication using SQL {#enabling-jwt-auth-using-sql} - -Users with "JWT" authentication type cannot be created using SQL now. - -## Identity Provider as an External User Directory {#idp-external-user-directory} - -If there is no suitable user pre-defined in ClickHouse, authentication is still possible: Identity Provider can be used as source of user information. -To allow this, add `token` section to the `users_directories` section of the `config.xml` file. - -At each login attempt, ClickHouse tries to find the user definition locally and authenticate it as usual. -If a token is provided but the user is not defined, ClickHouse will treat the user as externally defined and will try to validate the token and get user information from the specified processor. -If validated successfully, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section. -All this implies that the SQL-driven [Access Control and Account Management](/docs/en/guides/sre/user-management/index.md#access-control) is enabled and roles are created using the [CREATE ROLE](/docs/en/sql-reference/statements/create/role.md#create-role-statement) statement. - -**Example** - -```xml - - - - token_processor_name - - - - my_profile - - - 8a1b2c3d-4e5f-6789-abcd-ef0123456789 - ch_admin - - - 9f8e7d6c-5b4a-3210-fedc-ba0987654321 - ch_analyst - - - - \bclickhouse-[a-zA-Z0-9]+\b - - s/-/_/g - - - -``` - -:::note -For now, no more than one `token` section can be defined inside `user_directories`. This _may_ change in future. -::: - -**Parameters** - -- `processor` — Name of one of processors defined in `token_processors` config section described above. This parameter is mandatory and cannot be empty. -- `common_roles` — Section with a list of locally defined roles that will be assigned to each user retrieved from the IdP. Optional. -- `default_profile` — Name of a locally defined settings profile that will be assigned to each user retrieved from the IdP. If the profile does not exist, a warning will be logged and the user will be created without a profile. Optional. -- `roles_mapping` — Explicit map from incoming group identifier (e.g. an Entra security-group object ID) to a ClickHouse role name. Each entry is a `` element with `` and `` children. Applied **before** `roles_filter` and `roles_transform`; groups absent from the map pass through unchanged, so the filter stage can be used to drop unmapped entries. Optional. -- `roles_filter` — Regex string for groups filtering. Only groups (after `roles_mapping` is applied) that match this regex will be considered. Optional. -- `roles_transform` — Sed-style transform pattern applied to group names (after `roles_mapping` and `roles_filter`) before mapping to roles. Format: `s/pattern/replacement/flags`. The `g` flag applies the replacement globally (all occurrences). Example: `s/-/_/g` converts `clickhouse-grp-dba` to `clickhouse_grp_dba`. Optional. - -The three stages run in this order: `roles_mapping` → `roles_filter` → `roles_transform`. Stages are independent and any of them may be omitted. diff --git a/docs/en/antalya/oauth.md b/docs/en/antalya/oauth.md new file mode 100644 index 0000000000000..6bf7ba4a8d385 --- /dev/null +++ b/docs/en/antalya/oauth.md @@ -0,0 +1,363 @@ +--- +description: 'How to authenticate ClickHouse users with OAuth / OpenID Connect tokens and JWTs, use an Identity Provider as an external user directory, and log in from clickhouse-client.' +sidebarTitle: 'OAuth' +slug: /antalya/oauth +title: 'Token-based authentication' +doc_type: 'guide' +--- + +# Token-based authentication {#token-based-authentication} + +ClickHouse can authenticate users with access tokens issued by an Identity Provider (IdP): JWTs or opaque OAuth 2.0 tokens. Clients pass the token with `clickhouse-client --jwt ` or the HTTP header `Authorization: Bearer `. + +A token can authenticate: + +- a user defined in ClickHouse with the `jwt` authentication method — see [Users defined in ClickHouse](#enabling-jwt-auth); +- a user that is not defined in ClickHouse at all, when the IdP is configured as an [external user directory](#idp-external-user-directory). Such users are created on first login. + +In both cases the token is validated by a **token processor**. + +## Token processors {#token-processors} + +Token processors are defined in the `token_processors` section of `config.xml`; each child element is one processor, and its name is the element name: + +```xml + + + + jwt_dynamic_jwks + https://idp.example.com/.well-known/jwks.json + + + +``` + +Token authentication is enabled by default. Set `0` in `config.xml` to disable it: token processors are then not parsed, the `token` user directory is unavailable, and all token logins are rejected. + +Parameters common to all processors: + +- `type` — Processor type, case-insensitive. Mandatory. One of: + - `jwt_static_key`, `jwt_static_jwks`, `jwt_dynamic_jwks` — validate a JWT locally ([JWT processors](#jwt)); + - `entra` — Microsoft Entra ID ([Entra](#entra)); + - `google` — Google OAuth 2.0 access tokens ([Google](#google)); + - `openid` — any OpenID Connect provider, through its `/userinfo` and introspection endpoints ([OpenID](#openid)). +- `username_claim` — Claim used as the ClickHouse user name. Default: `sub`. +- `groups_claim` — Claim with the list of the user's groups, read from the JWT or from the `/userinfo` response. Default: `groups`. +- `token_cache_lifetime` — How long, in seconds, a validated token is cached to avoid repeated requests to the IdP. A cache entry never outlives the token's own expiration. Default: `3600`. + +Parameters that do not apply to the configured `type` are ignored. + +:::tip +If the IdP issues JWT access tokens ([RFC 9068](https://datatracker.ietf.org/doc/html/rfc9068)), use `jwt_dynamic_jwks`: tokens are verified locally, without a request to the IdP. `google` and `openid` are for opaque tokens, or for when you want the IdP itself to confirm each token. +::: + +### JWT processors {#jwt} + +The token is a JWT whose signature is verified locally with a static key, a static JWKS (JSON Web Key Set), or a JWKS downloaded from the IdP. Parameters shared by `jwt_static_key`, `jwt_static_jwks`, `jwt_dynamic_jwks` and `entra`, all optional: + +- `expected_issuer` — Required value of the `iss` claim. +- `expected_audience` — Required value of the `aud` claim. +- `expected_typ` — Required value of the JWT header `typ`, for example `at+jwt` for RFC 9068 access tokens. +- `claims` — JSON object that the token payload must contain, for example `{"tenant":"acme"}`. +- `allow_no_expiration` — Accept tokens without an `exp` claim. Default: `false`. +- `verifier_leeway` — Allowed clock skew between ClickHouse and the IdP, in seconds. Default: `60`. + +A warning is logged at startup for each of `expected_issuer`, `expected_audience` and `expected_typ` that is not set. + +#### `jwt_static_key` {#jwt-static-key} + +```xml + + jwt_static_key + HS256 + my_static_secret + +``` + +- `algo` — Signature algorithm. Mandatory. Supported: `HS256`, `HS384`, `HS512`, `RS256`, `RS384`, `RS512`, `ES256`, `ES384`, `ES512`, `ES256K`, `PS256`, `PS384`, `PS512`, `Ed25519`, `Ed448`, and `None` (never use it in production). +- `static_key` — Key for `HS*` algorithms. `static_key_in_base64` — whether `static_key` is base64-encoded, default `false`. +- `public_key` — Public key for all other algorithms except `None`. `private_key`, `public_key_password`, `private_key_password` — optional. + +#### `jwt_static_jwks` {#jwt-static-jwks} + +```xml + + jwt_static_jwks + {"keys": [{"kty": "RSA", "alg": "RS256", "kid": "mykid", "n": "_public_key_mod_", "e": "AQAB"}]} + +``` + +- `static_jwks` — JWKS content as JSON, or `static_jwks_file` — path to a file with it. Exactly one of the two is required. + +JWKS-based processors (`jwt_static_jwks`, `jwt_dynamic_jwks`, `entra`) support the `RS*` and `ES*` algorithms. + +#### `jwt_dynamic_jwks` {#jwt-remote-jwks} + +```xml + + jwt_dynamic_jwks + https://idp.example.com/.well-known/jwks.json + https://idp.example.com + clickhouse + +``` + +- `jwks_uri` — JWKS endpoint. Mandatory. +- `jwks_cache_lifetime` — How often the JWKS is re-fetched, in seconds. Default: `3600`. + +### Entra {#entra} + +`entra` is a preset of `jwt_dynamic_jwks` for Microsoft Entra ID: tokens are verified locally against the tenant's JWKS, and the user name and groups are read from the token. The token's audience must be your own application (registered with *Expose an API*), not Microsoft Graph — Graph-audience tokens cannot be verified by third parties. + +```xml + + entra + aaaabbbb-0000-cccc-1111-dddd2222eeee + api://clickhouse + preferred_username + +``` + +- `tenant_id` — Tenant GUID or `*.onmicrosoft.com` domain. Mandatory. Multi-tenant aliases (`common`, `organizations`, `consumers`) are rejected. +- `expected_audience` — Your application's ID URI (for example `api://clickhouse`) or client ID. Strongly recommended: without it, any token from the tenant is accepted. +- `jwks_uri` — Default: `https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys`. Override only for sovereign clouds. +- `expected_issuer` — Default: `https://login.microsoftonline.com/{tenant_id}/v2.0`. Override for v1.0 tokens (`https://sts.windows.net/{tenant_id}/`) or sovereign clouds. +- `username_claim` — Default `sub`; common choices for Entra are `preferred_username`, `upn` and `oid`. +- All other [JWT processor](#jwt) parameters, `jwks_cache_lifetime` and `token_cache_lifetime`. + +`azure` is accepted as an alias for `entra`. A former `azure` configuration needs `tenant_id` (and should get `expected_audience`), and the application must issue tokens for its own audience rather than for Microsoft Graph. + +#### Mapping Entra groups to ClickHouse roles {#entra-group-mapping} + +By default, the `groups` claim contains group object IDs (GUIDs). To get readable values: + +- **App Roles (recommended).** In the app registration, create app roles whose `Value` is the string ClickHouse should receive (for example `ch_admin`). In the enterprise application, set *Assignment required* to *Yes* and assign users or groups to the roles (assigning groups requires Entra ID P1/P2). Then set `roles`. This is compact and unaffected by group renames. +- **Group names in the `groups` claim.** In the app registration, set `groupMembershipClaims` to `ApplicationGroup` (or `SecurityGroup`) and add the `groups` optional access-token claim with `additionalProperties`: `sam_account_name` or `dns_domain_and_sam_account_name` for groups synced from on-premises AD, `cloud_displayname` for cloud-only groups. Groups not covered still arrive as GUIDs, and renaming a group in Entra breaks the mapping. +- **Keep GUIDs** and translate them with `roles_mapping` in the [user directory](#idp-external-user-directory). + +If you switch from GUIDs to names, revisit `roles_filter`. + +### Google {#google} + +`google` validates Google OAuth 2.0 access tokens. They are opaque, so each token is checked with Google's `/tokeninfo` endpoint, and `/userinfo` is also called when `username_claim` is not in the `/tokeninfo` response. + +```xml + + google + 1234567890-abcdef.apps.googleusercontent.com + +``` + +- `expected_audience` — OAuth client ID the token must be issued to. Strongly recommended: without it, any Google access token is accepted. +- `username_claim` — Default `sub` (the Google account ID). `email` requires the `email` scope. +- `token_cache_lifetime`. + +**Groups.** When the token carries the user's email, the processor reads the user's direct Google Groups memberships through the Cloud Identity API and uses the group email addresses as groups. This requires the `https://www.googleapis.com/auth/cloud-identity.groups.readonly` scope in the token and the Cloud Identity API enabled in the project of the OAuth client. Groups inherited through nested groups are not returned. If the lookup fails, the user is authenticated without groups. To turn group emails into role names, use `roles_filter` and `roles_transform`; for example, this maps `ch-analysts@example.com` to the role `ch-analysts`: + +```xml + + google_prod + ch-[a-z-]+@example\.com + s/@.*// + +``` + +### OpenID {#openid} + +`openid` works with any OpenID Connect provider and is configured in one of two modes. + +**Discovery mode** reads the endpoints from the provider's discovery document. If the document has a `jwks_uri`, JWT access tokens are verified locally. If it has an `introspection_endpoint` and introspection credentials are set, every login is also checked with [RFC 7662](https://datatracker.ietf.org/doc/html/rfc7662) introspection, which detects revoked tokens. + +```xml + + openid + https://idp.example.com/.well-known/openid-configuration + my-clickhouse-client-id + clickhouse-rs + ... + +``` + +- `configuration_endpoint` — URL of `.well-known/openid-configuration`. Mandatory. +- `expected_issuer`, `expected_audience` — Required `iss` and `aud`, checked on the JWT or in the introspection response. +- `introspection_client_id`, `introspection_client_secret` — Credentials for the introspection endpoint. Set both or neither. +- `allow_no_expiration`, `verifier_leeway` (default `60`), `jwks_cache_lifetime` (default `3600`) — As for [JWT processors](#jwt). +- `allow_http_discovery_urls` — Allow non-HTTPS URLs in the discovery document. Default: `false`. + +**Manual mode** calls `/userinfo` for every new token. Without introspection, it cannot check the token's issuer, audience or expiration. + +```xml + + openid + https://idp.example.com/userinfo + https://idp.example.com/introspect + clickhouse-rs + ... + https://idp.example.com + clickhouse-rs + +``` + +- `userinfo_endpoint` — Mandatory. +- `token_introspection_endpoint`, `introspection_client_id`, `introspection_client_secret` — Optional, all three together. +- `expected_issuer`, `expected_audience` — Allowed only together with introspection. + +`configuration_endpoint` and `userinfo_endpoint` are mutually exclusive, and `jwks_uri` is rejected in both modes — use `jwt_dynamic_jwks` for an explicit JWKS URL. The introspection credentials accept `from_env` and `from_zk`. + +Introspection needs a separate confidential OAuth client representing ClickHouse, without redirect URIs: + +| IdP | How to create the introspection client | +|---|---| +| Keycloak | Clients → confidential client with *Service Accounts* enabled; the secret is on the *Credentials* tab. | +| Okta | Applications → Create App Integration → *API Services*. | +| Auth0 | No introspection for opaque tokens; for JWT access tokens of a custom API use `jwt_dynamic_jwks`. | +| Google, GitHub, Microsoft Entra ID | No RFC 7662 endpoint; use `google`, `entra` or `jwt_dynamic_jwks`. | + +## Users defined in ClickHouse {#enabling-jwt-auth} + +A user can authenticate with a token if `jwt` is one of its authentication methods. In `users.xml`, where a user has exactly one authentication method, the `jwt` section replaces `password`. + +```sql +CREATE USER alice IDENTIFIED WITH jwt PROCESSOR 'my_processor' CLAIMS '{"resource_access":{"account":{"roles":["view-profile"]}}}'; +``` + +In `users.xml`: + +```xml + + + + + my_processor + {"resource_access":{"account":{"roles":["view-profile"]}}} + + + + +``` + +Both clauses are optional: + +- `PROCESSOR` / `processor` — Accept only tokens validated by this processor. Without it, any configured processor may validate the token. +- `CLAIMS` / `claims` — JSON object the token payload must contain. In the example, `resource_access.account.roles` must contain `"view-profile"`. Claims can only be checked in JWTs validated by `jwt_static_key`, `jwt_static_jwks`, `jwt_dynamic_jwks` or `entra`; a user with `claims` cannot log in with a token validated by another processor. + +## IdP as an external user directory {#idp-external-user-directory} + +With a `token` section in `user_directories`, a user that is not defined in ClickHouse can log in with a valid token. The user is created on first login with the name taken from `username_claim`, and its roles are updated on every login. Roles and settings profiles must exist in ClickHouse, created with SQL-driven [access control](/concepts/features/security/access-rights). + +```xml + + + + my_processor + + + + token_users_profile + + + 8a1b2c3d-4e5f-6789-abcd-ef0123456789 + ch_admin + + + \bclickhouse-[a-zA-Z0-9]+\b + s/-/_/g + + + +``` + +Only one `token` section is allowed. + +- `processor` — Name of the token processor. Mandatory. +- `common_roles` — Roles granted to every user from this directory. +- `default_profile` — Settings profile assigned to every user from this directory. If the profile does not exist, users are created without it. +- `networks` — Addresses that users from this directory may connect from, in the same format as [`networks`](/concepts/features/configuration/settings/settings-users#user-namenetworks) in `users.xml`. Default: any address. + +The user's groups (from `groups_claim`) are turned into role names by three optional steps, applied in this order: + +1. `roles_mapping` — Explicit renames, one `` with `` and `` per group. Groups without an entry pass unchanged. +2. `roles_filter` — Regular expression; groups that do not match it are dropped. +3. `roles_transform` — Sed-style substitution `s/pattern/replacement/flags`; the `g` flag replaces all occurrences. For example, `s/-/_/g` turns `clickhouse-grp-dba` into `clickhouse_grp_dba`. + +The user is granted every resulting role that exists in ClickHouse. + +### Settings profiles {#idp-settings-profiles} + +To give different settings to different groups of users, for example to users connecting from Grafana, attach a settings profile to a role and grant the role through the group mapping: + +```sql +CREATE SETTINGS PROFILE grafana_profile SETTINGS max_execution_time = 60, readonly = 1; +CREATE ROLE grafana_users SETTINGS PROFILE 'grafana_profile'; +``` + +```xml + + my_processor + + + grafana-viewers + grafana_users + + + +``` + +Users with the `grafana-viewers` group get the `grafana_users` role and the settings of `grafana_profile`. This works for roles from `common_roles`, `roles_filter` and `roles_transform` as well. + +If a setting is defined in several places, `default_profile` overrides role profiles, which override the server's [default profile](/reference/settings/server-settings/settings/default#default_profile). Keep the settings that differ per group out of `default_profile`. + +## Forwarding the token to external services {#token-forwarding} + +The user's token can be forwarded to Iceberg REST or Glue catalogs. Enable it with `1` in `config.xml` (hot-reloadable, default `0`) and `oauth_forward_user_token` on the [`DataLakeCatalog`](/engines/database-engines/datalakecatalog) database; see [catalog token forwarding](/engines/database-engines/datalakecatalog#user-token-forwarding). + +:::danger Restrict `CREATE DATABASE` +Whoever creates the database chooses the endpoints that receive users' tokens. Grant `CREATE DATABASE` only to trusted users, restrict `remote_url_allow_hosts` for the catalog and token-exchange endpoints, and use HTTPS for them. In passthrough mode the forwarded token is also valid for ClickHouse itself. +::: + +- Only the session's verified token is forwarded. It is not inherited by `EXECUTE AS` or `DEFINER` views, not sent to other ClickHouse nodes, and not written to disk. +- HTTP requests authenticate separately, so a new token takes effect on the next request; native TCP sessions must reconnect. +- With `async_insert = 1`, each queued batch keeps its token until it is flushed, even with `wait_for_async_insert = 0`. Inserts with different tokens go to separate batches. + +## clickhouse-client {#clickhouse-client} + +`clickhouse-client` can get a token in three ways (see its [connection options](/concepts/features/interfaces/client#command-line-options-connection)): + +- `--jwt ` — Use the given token. +- `--login=browser` or `--login=device` — Log in to an OpenID Connect provider with the Authorization Code + PKCE flow (opens a browser) or the Device Authorization flow (prints a URL and a code). Requires an [OAuth credentials file](#oauth-credentials-file). +- `--jwt-command ` — Get the token from a [command](#jwt-command-details). + +### OAuth credentials file {#oauth-credentials-file} + +`--login=browser` and `--login=device` read the OAuth client from a JSON file, by default `~/.clickhouse-client/oauth_client.json`, or the path given with `--oauth-credentials `. The format is the one that the Google Cloud Console downloads for an OAuth client ID: + +```json +{ + "installed": { + "client_id": "YOUR_CLIENT_ID", + "client_secret": "YOUR_CLIENT_SECRET", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": "https://oauth2.googleapis.com/token", + "redirect_uris": ["http://127.0.0.1"] + } +} +``` + +- The top-level key is `installed` or `web`. +- `client_id`, `auth_uri`, `token_uri` — Mandatory. +- `client_secret` — Omit it or leave it empty for public clients; it is then not sent. +- `device_authorization_uri` — For `--login=device`. If absent, it is discovered through OIDC discovery from `issuer`. + +The refresh token is cached in `~/.clickhouse-client/oauth_cache.json` (mode `0600`), so the browser or device code is needed again only after it expires. + +### `--jwt-command` {#jwt-command-details} + +```bash +clickhouse-client --jwt-command "curl -sS https://idp.example/token | jq -r .access_token" +``` + +The command runs with `/bin/sh -c` and stdin closed. Its stdout, without the trailing newline, is the token; prompts, URLs and device codes must go to stderr, which is passed through to the client's stderr. + +The command runs on the first connection and runs again when the cached token is a JWT expiring within 30 seconds, or when the server rejects the token (the connection is then retried once). Opaque tokens and JWTs without `exp` are reused until the server rejects them. + +The command fails authentication if it exits with a non-zero code, prints nothing, or exceeds `--jwt-command-timeout` (default `30` seconds, also settable as `` in the client config). On timeout, the command's whole process tree is killed. `--jwt-command` cannot be combined with `--jwt`, `--login` or a non-default `--user`. diff --git a/docs/reference/engines/database-engines/datalake.mdx b/docs/reference/engines/database-engines/datalake.mdx index e91ef95cfcc7e..76669cf5031f8 100644 --- a/docs/reference/engines/database-engines/datalake.mdx +++ b/docs/reference/engines/database-engines/datalake.mdx @@ -109,7 +109,7 @@ This requires: setting, which is `false` by default; - `catalog_type = 'rest'` or `catalog_type = 'glue'`; - token authentication through an `Authorization: Bearer` HTTP header or `--jwt` for the native - protocol. See [Token-based authentication](/en/operations/external-authenticators/oauth). + protocol. See [Token-based authentication](/antalya/oauth). :::danger Restrict `CREATE DATABASE` Database creators choose the endpoints that receive users' tokens. Grant `CREATE DATABASE` only From e62d0c3eb21d83fa42e3784a02a57a4ec0094d98 Mon Sep 17 00:00:00 2001 From: Andrey Zvonov <32552679+zvonand@users.noreply.github.com> Date: Sat, 24 Oct 2026 18:35:05 +0200 Subject: [PATCH 2/3] Add `profiles_mapping` to the `token` user directory Map IdP groups directly to settings profiles for users created by the `token` user directory. Mapped profiles are assigned to the user itself after `default_profile`, in config order, so they override profiles carried by roles. The user's profiles are rebuilt on every login, so a profile of a group the user left is removed. The resulting settings are sent to other nodes with distributed queries. `roles_mapping` and `profiles_mapping` share one parser. Related: https://github.com/Altinity/ClickHouse/issues/1032 --- docs/en/antalya/oauth.md | 51 +++++-- src/Access/TokenAccessStorage.cpp | 125 ++++++++++-------- src/Access/TokenAccessStorage.h | 5 +- .../test_token_profiles_mapping/__init__.py | 0 .../configs/remote_servers.xml | 19 +++ .../configs/token.xml | 31 +++++ .../configs/users.xml | 7 + .../test_token_profiles_mapping/test.py | 110 +++++++++++++++ 8 files changed, 279 insertions(+), 69 deletions(-) create mode 100644 tests/integration/test_token_profiles_mapping/__init__.py create mode 100644 tests/integration/test_token_profiles_mapping/configs/remote_servers.xml create mode 100644 tests/integration/test_token_profiles_mapping/configs/token.xml create mode 100644 tests/integration/test_token_profiles_mapping/configs/users.xml create mode 100644 tests/integration/test_token_profiles_mapping/test.py diff --git a/docs/en/antalya/oauth.md b/docs/en/antalya/oauth.md index 6bf7ba4a8d385..1fe970e119cd0 100644 --- a/docs/en/antalya/oauth.md +++ b/docs/en/antalya/oauth.md @@ -8,14 +8,25 @@ doc_type: 'guide' # Token-based authentication {#token-based-authentication} -ClickHouse can authenticate users with access tokens issued by an Identity Provider (IdP): JWTs or opaque OAuth 2.0 tokens. Clients pass the token with `clickhouse-client --jwt ` or the HTTP header `Authorization: Bearer `. +ClickHouse can authenticate users with access tokens issued by an external Identity Provider (IdP). Clients pass the token with `clickhouse-client --jwt ` or the HTTP header `Authorization: Bearer `. A token can authenticate: - a user defined in ClickHouse with the `jwt` authentication method — see [Users defined in ClickHouse](#enabling-jwt-auth); - a user that is not defined in ClickHouse at all, when the IdP is configured as an [external user directory](#idp-external-user-directory). Such users are created on first login. -In both cases the token is validated by a **token processor**. +In both cases the token is validated by a **token processor** defined in `config.xml`. + +## Glossary {#glossary} + +- **Identity Provider (IdP)** — An external service that authenticates users and issues tokens: Keycloak, Okta, Microsoft Entra ID, Google, and others. +- **Access token** — A string the client presents to ClickHouse as proof of the user's identity. It is either a JWT or an opaque token, depending on the provider. +- **JWT (JSON Web Token)** — A signed token that carries data about the user. ClickHouse can verify it locally, without contacting the IdP. +- **Opaque token** — A token that is just a random string. Only the IdP can tell whether it is valid and whom it belongs to. +- **Claim** — A field in a JWT, or in the IdP's response about a token. Some of them are provider-specific, some are standard, e.g. `sub` is the user ID, `iss` (*issuer*) is the IdP that issued the token, `aud` (*audience*) is the application it was issued for and `exp` is its expiration time. +- **JWKS (JSON Web Key Set)** — The set of public keys that verify a JWT's signature, usually published by the IdP at a pre-defined URL. +- **Group** — A group (entity) the user belongs to at the IdP. ClickHouse can map groups to its own roles. +- **Token processor** — A bridge between ClickHouse and an IdP. It validates a token, either locally or by asking the IdP, and passes the user name and groups from the token to ClickHouse. ## Token processors {#token-processors} @@ -32,9 +43,9 @@ Token processors are defined in the `token_processors` section of `config.xml`; ``` -Token authentication is enabled by default. Set `0` in `config.xml` to disable it: token processors are then not parsed, the `token` user directory is unavailable, and all token logins are rejected. +Token authentication is enabled by default. Set `0` in `config.xml` to disable it system-wide: token processors are then not parsed, the `token` user directory is unavailable, and all token-based logins are rejected. -Parameters common to all processors: +Parameters common to all token processors: - `type` — Processor type, case-insensitive. Mandatory. One of: - `jwt_static_key`, `jwt_static_jwks`, `jwt_dynamic_jwks` — validate a JWT locally ([JWT processors](#jwt)); @@ -48,7 +59,7 @@ Parameters common to all processors: Parameters that do not apply to the configured `type` are ignored. :::tip -If the IdP issues JWT access tokens ([RFC 9068](https://datatracker.ietf.org/doc/html/rfc9068)), use `jwt_dynamic_jwks`: tokens are verified locally, without a request to the IdP. `google` and `openid` are for opaque tokens, or for when you want the IdP itself to confirm each token. +If the IdP issues JWT access tokens ([RFC 9068](https://datatracker.ietf.org/doc/html/rfc9068)), use `jwt_dynamic_jwks`: tokens are verified locally, without a request to the IdP. `openid` is a good fit for opaque tokens, or for when you want the IdP itself to confirm each token. ::: ### JWT processors {#jwt} @@ -215,7 +226,13 @@ Introspection needs a separate confidential OAuth client representing ClickHouse ## Users defined in ClickHouse {#enabling-jwt-auth} -A user can authenticate with a token if `jwt` is one of its authentication methods. In `users.xml`, where a user has exactly one authentication method, the `jwt` section replaces `password`. +A user can authenticate with a token if `jwt` is one of its authentication methods. + +:::tip +Locally-defined users are only _authenticated_ with tokens; no role mapping is done. +::: + +In `users.xml`, where a user has exactly one authentication method, the `jwt` section replaces `password`. ```sql CREATE USER alice IDENTIFIED WITH jwt PROCESSOR 'my_processor' CLAIMS '{"resource_access":{"account":{"roles":["view-profile"]}}}'; @@ -272,6 +289,7 @@ Only one `token` section is allowed. - `processor` — Name of the token processor. Mandatory. - `common_roles` — Roles granted to every user from this directory. - `default_profile` — Settings profile assigned to every user from this directory. If the profile does not exist, users are created without it. +- `profiles_mapping` — Settings profiles assigned by group; see [Settings profiles](#idp-settings-profiles). - `networks` — Addresses that users from this directory may connect from, in the same format as [`networks`](/concepts/features/configuration/settings/settings-users#user-namenetworks) in `users.xml`. Default: any address. The user's groups (from `groups_claim`) are turned into role names by three optional steps, applied in this order: @@ -284,28 +302,35 @@ The user is granted every resulting role that exists in ClickHouse. ### Settings profiles {#idp-settings-profiles} -To give different settings to different groups of users, for example to users connecting from Grafana, attach a settings profile to a role and grant the role through the group mapping: +To give different settings to different groups of users, for example to users connecting from Grafana, map groups to settings profiles with `profiles_mapping`: ```sql CREATE SETTINGS PROFILE grafana_profile SETTINGS max_execution_time = 60, readonly = 1; -CREATE ROLE grafana_users SETTINGS PROFILE 'grafana_profile'; ``` ```xml my_processor - + token_users_profile + grafana-viewers - grafana_users + grafana_profile - + ``` -Users with the `grafana-viewers` group get the `grafana_users` role and the settings of `grafana_profile`. This works for roles from `common_roles`, `roles_filter` and `roles_transform` as well. +Users with the `grafana-viewers` group get `grafana_profile` in addition to `default_profile`. `from` is the group exactly as it comes in the token: `roles_mapping`, `roles_filter` and `roles_transform` do not apply. Profiles are reassigned on every login, so a user who leaves a group loses its profile. + +If a setting is defined in several places, the first source in this list wins: + +1. Profiles from `profiles_mapping`. When the user is in several mapped groups, the entry that comes later in `profiles_mapping` wins. +2. `default_profile`. +3. Profiles of the user's roles. A profile can be attached to a role with `CREATE ROLE ... SETTINGS PROFILE '...'`; when several roles set the same setting, which one wins is undefined. +4. The server's [default profile](/reference/settings/server-settings/settings/default#default_profile). -If a setting is defined in several places, `default_profile` overrides role profiles, which override the server's [default profile](/reference/settings/server-settings/settings/default#default_profile). Keep the settings that differ per group out of `default_profile`. +In distributed queries, the resulting settings are sent to the other nodes with the query, so the profiles only need to exist on the node the user connects to. ## Forwarding the token to external services {#token-forwarding} diff --git a/src/Access/TokenAccessStorage.cpp b/src/Access/TokenAccessStorage.cpp index 8d22f9b7c67ea..560b354038fc5 100644 --- a/src/Access/TokenAccessStorage.cpp +++ b/src/Access/TokenAccessStorage.cpp @@ -120,6 +120,44 @@ namespace RE2::Replace(&result, re, replacement); return result; } + + /// Parse a list of XY children of `section`, keeping config order. + std::vector> parseGroupMapping( + const Poco::Util::AbstractConfiguration & config, const String & prefix_str, const String & section) + { + std::vector> result; + if (!config.has(prefix_str + section)) + return result; + + Poco::Util::AbstractConfiguration::Keys map_keys; + config.keys(prefix_str + section, map_keys); + + for (const auto & key : map_keys) + { + const String entry_prefix = prefix_str + section + "." + key; + if (!config.has(entry_prefix + ".from") || !config.has(entry_prefix + ".to")) + throw Exception(ErrorCodes::BAD_ARGUMENTS, + "{} entry '{}' must contain both 'from' and 'to' subelements", section, key); + + const String from = config.getString(entry_prefix + ".from"); + const String to = config.getString(entry_prefix + ".to"); + + if (from.empty()) + throw Exception(ErrorCodes::BAD_ARGUMENTS, "{} entry '{}': 'from' must not be empty", section, key); + if (to.empty()) + throw Exception(ErrorCodes::BAD_ARGUMENTS, "{} entry '{}': 'to' must not be empty", section, key); + + const auto it = std::find_if(result.begin(), result.end(), [&](const auto & entry) { return entry.first == from; }); + if (it != result.end()) + throw Exception(ErrorCodes::BAD_ARGUMENTS, + "{} has duplicate 'from' value '{}' (already mapped to '{}', cannot remap to '{}')", + section, from, it->second, to); + + result.emplace_back(from, to); + } + + return result; + } } TokenAccessStorage::TokenAccessStorage(const String & storage_name_, AccessControl & access_control_, const Poco::Util::AbstractConfiguration & config_, const String & prefix_) @@ -184,33 +222,11 @@ TokenAccessStorage::TokenAccessStorage(const String & storage_name_, AccessContr /// children. The mapping rewrites incoming group names BEFORE `roles_filter` / `roles_transform`, /// so each subsequent stage operates on the mapped value. Groups not listed here pass through /// to filter/transform unchanged. - if (config.has(prefix_str + "roles_mapping")) - { - Poco::Util::AbstractConfiguration::Keys map_keys; - config.keys(prefix_str + "roles_mapping", map_keys); + for (auto & [from, to] : parseGroupMapping(config, prefix_str, "roles_mapping")) + roles_mapping.emplace(std::move(from), std::move(to)); - for (const auto & key : map_keys) - { - const String entry_prefix = prefix_str + "roles_mapping." + key; - if (!config.has(entry_prefix + ".from") || !config.has(entry_prefix + ".to")) - throw Exception(ErrorCodes::BAD_ARGUMENTS, - "roles_mapping entry '{}' must contain both 'from' and 'to' subelements", key); - - const String from = config.getString(entry_prefix + ".from"); - const String to = config.getString(entry_prefix + ".to"); - - if (from.empty()) - throw Exception(ErrorCodes::BAD_ARGUMENTS, "roles_mapping entry '{}': 'from' must not be empty", key); - if (to.empty()) - throw Exception(ErrorCodes::BAD_ARGUMENTS, "roles_mapping entry '{}': 'to' must not be empty", key); - - auto [it, inserted] = roles_mapping.emplace(from, to); - if (!inserted) - throw Exception(ErrorCodes::BAD_ARGUMENTS, - "roles_mapping has duplicate 'from' value '{}' (already mapped to '{}', cannot remap to '{}')", - from, it->second, to); - } - } + /// `profiles_mapping` matches the raw incoming group, independently of the roles pipeline. + profiles_mapping = parseGroupMapping(config, prefix_str, "profiles_mapping"); provider_name = config.getString(prefix_str + "processor"); if (provider_name.empty()) @@ -529,40 +545,38 @@ void TokenAccessStorage::assignRolesNoLock(User & user, const std::set & user_external_roles[user_name] = external_roles; } -void TokenAccessStorage::assignProfileNoLock(User & user) const +void TokenAccessStorage::assignProfilesNoLock(User & user, const std::set & groups) const { - if (default_profile_name.empty()) - return; - const auto & user_name = user.getName(); - auto & settings = user.settings; - // Look up the profile ID once - const auto profile_id = access_control.find(default_profile_name); - if (!profile_id) + std::vector profile_names; + if (!default_profile_name.empty()) + profile_names.push_back(default_profile_name); + for (const auto & [group, profile_name] : profiles_mapping) { - LOG_TRACE(getLogger(), "Did not assign profile '{}' to user '{}': profile not found", default_profile_name, user_name); - return; + if (groups.contains(group)) + profile_names.push_back(profile_name); } - // Check if profile is already assigned - bool profile_already_assigned = false; - for (const auto & element : settings) + /// The user's settings consist only of the profiles assigned here, so they are rebuilt from scratch: + /// a profile of a group the user is no longer in is removed. + SettingsProfileElements settings; + for (const auto & profile_name : profile_names) { - if (element.parent_profile.has_value() && element.parent_profile == *profile_id) + const auto profile_id = access_control.find(profile_name); + if (!profile_id) { - profile_already_assigned = true; - break; + LOG_TRACE(getLogger(), "Did not assign profile '{}' to user '{}': profile not found", profile_name, user_name); + continue; } - } - if (!profile_already_assigned) - { SettingsProfileElement profile_element; profile_element.parent_profile = *profile_id; settings.push_back(std::move(profile_element)); - LOG_TRACE(getLogger(), "Assigned profile '{}' to user '{}'", default_profile_name, user_name); + LOG_TRACE(getLogger(), "Assigned profile '{}' to user '{}'", profile_name, user_name); } + + user.settings = std::move(settings); } std::optional TokenAccessStorage::authenticateImpl( @@ -631,7 +645,7 @@ std::optional TokenAccessStorage::authenticateImpl( /// the bearer token; grant common roles only. Session applies the /// roles pushed by the initiator. assignRolesNoLock(*new_user, /* external_roles= */ {}); - assignProfileNoLock(*new_user); + assignProfilesNoLock(*new_user, /* groups= */ {}); id = memory_storage.insert(new_user); lock.unlock(); @@ -656,6 +670,7 @@ std::optional TokenAccessStorage::authenticateImpl( /// Pipeline: incoming group --(roles_mapping)--> mapped name --(roles_filter)--> kept/dropped --(roles_transform)--> CH role name. /// Each stage is independent and optional; groups absent from `roles_mapping` pass through unchanged. std::set external_roles; + const std::set groups = token_credentials.getGroups(); /// Defensive: a broken filter regex must NEVER fall through to the permissive /// "grant everything that survives the rest of the pipeline" branch. Parse-time @@ -674,7 +689,7 @@ std::optional TokenAccessStorage::authenticateImpl( const bool has_filter = roles_filter.has_value(); const bool has_transform = roles_transform_pattern.has_value() && roles_transform_replacement.has_value(); - for (const auto & group : token_credentials.getGroups()) + for (const auto & group : groups) { String name = group; @@ -712,7 +727,7 @@ std::optional TokenAccessStorage::authenticateImpl( if (new_user) { assignRolesNoLock(*new_user, external_roles); - assignProfileNoLock(*new_user); + assignProfilesNoLock(*new_user, groups); id = memory_storage.insert(new_user); } else @@ -737,13 +752,13 @@ std::optional TokenAccessStorage::authenticateImpl( if (roles_changed) { - memory_storage.update(*id, [this, &external_roles] (const AccessEntityPtr & entity_, const UUID &) -> AccessEntityPtr + memory_storage.update(*id, [this, &external_roles, &groups] (const AccessEntityPtr & entity_, const UUID &) -> AccessEntityPtr { if (auto user_entity = typeid_cast>(entity_)) { auto changed_user = typeid_cast>(user_entity->clone()); assignRolesNoLock(*changed_user, external_roles); - assignProfileNoLock(*changed_user); + assignProfilesNoLock(*changed_user, groups); return changed_user; } return entity_; @@ -751,14 +766,14 @@ std::optional TokenAccessStorage::authenticateImpl( } else { - /// Roles are stable; just refresh the profile in case it was - /// added/changed in config since the last auth. - memory_storage.update(*id, [this] (const AccessEntityPtr & entity_, const UUID &) -> AccessEntityPtr + /// Roles are stable; just refresh the profiles, which depend on the + /// groups and on which profiles exist now. + memory_storage.update(*id, [this, &groups] (const AccessEntityPtr & entity_, const UUID &) -> AccessEntityPtr { if (auto user_entity = typeid_cast>(entity_)) { auto changed_user = typeid_cast>(user_entity->clone()); - assignProfileNoLock(*changed_user); + assignProfilesNoLock(*changed_user, groups); return changed_user; } return entity_; diff --git a/src/Access/TokenAccessStorage.h b/src/Access/TokenAccessStorage.h index fb2ee458d63b2..d92d8a63c882c 100644 --- a/src/Access/TokenAccessStorage.h +++ b/src/Access/TokenAccessStorage.h @@ -67,6 +67,9 @@ class TokenAccessStorage : public IAccessStorage std::set common_role_names; // role name that should be granted to all users at all times String default_profile_name; // settings profile name that should be assigned to all users + /// Incoming group -> settings profile name, in config order. Profiles of all matched groups are assigned + /// after `default_profile_name`, so for the same setting a later profile overrides an earlier one. + std::vector> profiles_mapping; /// Optional IP allowlist applied to auto-provisioned users at creation /// time. When unset, auto-created users inherit the default `AnyHostTag` /// (current behavior, no breakage). When set, only clients whose source @@ -87,7 +90,7 @@ class TokenAccessStorage : public IAccessStorage void applyRoleChangeNoLock(bool grant, const UUID & role_id, const String & role_name); void assignRolesNoLock(User & user, const std::set & external_roles) const; - void assignProfileNoLock(User & user) const; + void assignProfilesNoLock(User & user, const std::set & groups) const; protected: std::optional findImpl(AccessEntityType type, const String & name) const override; diff --git a/tests/integration/test_token_profiles_mapping/__init__.py b/tests/integration/test_token_profiles_mapping/__init__.py new file mode 100644 index 0000000000000..e69de29bb2d1d diff --git a/tests/integration/test_token_profiles_mapping/configs/remote_servers.xml b/tests/integration/test_token_profiles_mapping/configs/remote_servers.xml new file mode 100644 index 0000000000000..d0c4683b8a599 --- /dev/null +++ b/tests/integration/test_token_profiles_mapping/configs/remote_servers.xml @@ -0,0 +1,19 @@ + + + + profiles_mapping_cluster_secret + + + instance1 + 9000 + + + + + instance2 + 9000 + + + + + diff --git a/tests/integration/test_token_profiles_mapping/configs/token.xml b/tests/integration/test_token_profiles_mapping/configs/token.xml new file mode 100644 index 0000000000000..154ca728d0174 --- /dev/null +++ b/tests/integration/test_token_profiles_mapping/configs/token.xml @@ -0,0 +1,31 @@ + + + + jwt_static_key + HS256 + profiles_mapping_test_secret + false + true + + + + + + hs256_groups + + + + p_default + + + g_analysts + p_analysts + + + g_fast + p_fast + + + + + diff --git a/tests/integration/test_token_profiles_mapping/configs/users.xml b/tests/integration/test_token_profiles_mapping/configs/users.xml new file mode 100644 index 0000000000000..3399ef5915ada --- /dev/null +++ b/tests/integration/test_token_profiles_mapping/configs/users.xml @@ -0,0 +1,7 @@ + + + + 1 + + + diff --git a/tests/integration/test_token_profiles_mapping/test.py b/tests/integration/test_token_profiles_mapping/test.py new file mode 100644 index 0000000000000..3736894559d88 --- /dev/null +++ b/tests/integration/test_token_profiles_mapping/test.py @@ -0,0 +1,110 @@ +""" +Tests for `` in the `token` user directory: settings profiles assigned +directly to externally defined users by their IdP groups. + +Profiles are assigned to the user itself, so they override profiles carried by roles; +mapped profiles are assigned after `default_profile`, in config order. The profile settings +must reach other nodes of a cluster, which have neither the user's groups nor the profiles. +""" + +import jwt +import pytest + +from helpers.cluster import ClickHouseCluster + +SECRET = "profiles_mapping_test_secret" + +cluster = ClickHouseCluster(__file__) + +instance1 = cluster.add_instance( + "instance1", + main_configs=["configs/token.xml", "configs/remote_servers.xml"], + user_configs=["configs/users.xml"], +) + +instance2 = cluster.add_instance( + "instance2", + main_configs=["configs/token.xml", "configs/remote_servers.xml"], + user_configs=["configs/users.xml"], +) + +DEFAULT_MAX_QUERY_SIZE = "262144" + + +@pytest.fixture(scope="module", autouse=True) +def started_cluster(): + try: + cluster.start() + + instance1.query("CREATE SETTINGS PROFILE p_default SETTINGS max_block_size = 1001") + instance1.query("CREATE SETTINGS PROFILE p_role SETTINGS max_insert_block_size = 2002") + instance1.query( + "CREATE SETTINGS PROFILE p_analysts SETTINGS max_block_size = 1003, max_insert_block_size = 2003" + ) + instance1.query("CREATE SETTINGS PROFILE p_fast SETTINGS max_block_size = 1004, max_query_size = 300004") + instance1.query("CREATE ROLE r_common SETTINGS PROFILE 'p_role'") + + # instance2 has the role pushed by the initiator, but none of the profiles. + instance2.query("CREATE ROLE r_common") + + for node in (instance1, instance2): + node.query("GRANT SELECT, REMOTE ON *.* TO r_common") + + yield cluster + finally: + cluster.shutdown() + + +def query_with_token(node, sub, groups, sql): + token = jwt.encode({"sub": sub, "groups": groups}, SECRET, algorithm="HS256") + resp = node.http_request( + "", + method="POST", + data=sql, + headers={"Authorization": f"Bearer {token}"}, + ) + resp.raise_for_status() + return resp.text + + +def get_settings(sub, groups): + return query_with_token( + instance1, + sub, + groups, + "SELECT getSetting('max_block_size'), getSetting('max_insert_block_size'), getSetting('max_query_size') " + "FORMAT TabSeparated", + ).strip().split("\t") + + +def test_without_mapped_groups(): + assert get_settings("user_no_groups", ["g_unmapped"]) == ["1001", "2002", DEFAULT_MAX_QUERY_SIZE] + + +def test_mapped_profile_overrides_default_and_role_profiles(): + assert get_settings("user_analyst", ["g_analysts"]) == ["1003", "2003", DEFAULT_MAX_QUERY_SIZE] + + +def test_later_mapping_entry_wins(): + assert get_settings("user_both", ["g_fast", "g_analysts"]) == ["1004", "2003", "300004"] + + +def test_profile_removed_with_group(): + assert get_settings("user_leaving", ["g_analysts"]) == ["1003", "2003", DEFAULT_MAX_QUERY_SIZE] + assert get_settings("user_leaving", []) == ["1001", "2002", DEFAULT_MAX_QUERY_SIZE] + + +def test_mapped_profile_settings_reach_other_nodes(): + result = query_with_token( + instance1, + "user_cluster", + ["g_fast"], + "SELECT name, value FROM cluster('test_token_cluster', system.settings) " + "WHERE name IN ('max_block_size', 'max_query_size') ORDER BY name, value FORMAT TabSeparated", + ) + assert result.strip().split("\n") == [ + "max_block_size\t1004", + "max_block_size\t1004", + "max_query_size\t300004", + "max_query_size\t300004", + ] From 7da001978751a57377f71916e6301cfaa489b635 Mon Sep 17 00:00:00 2001 From: Andrey Zvonov <32552679+zvonand@users.noreply.github.com> Date: Sun, 25 Oct 2026 11:03:32 +0100 Subject: [PATCH 3/3] Document stale token profiles on interserver hops Related: https://github.com/Altinity/ClickHouse/pull/2664#discussion_r4370351275 --- docs/en/antalya/oauth.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/en/antalya/oauth.md b/docs/en/antalya/oauth.md index 1fe970e119cd0..74db823d0b9e0 100644 --- a/docs/en/antalya/oauth.md +++ b/docs/en/antalya/oauth.md @@ -330,7 +330,9 @@ If a setting is defined in several places, the first source in this list wins: 3. Profiles of the user's roles. A profile can be attached to a role with `CREATE ROLE ... SETTINGS PROFILE '...'`; when several roles set the same setting, which one wins is undefined. 4. The server's [default profile](/reference/settings/server-settings/settings/default#default_profile). -In distributed queries, the resulting settings are sent to the other nodes with the query, so the profiles only need to exist on the node the user connects to. +In distributed queries, the resulting settings that differ from their defaults are sent to the other nodes with the query, so the profiles only need to exist on the node the user connects to. + +Limitation: if the user has logged in directly to another node, that node keeps the profiles and roles from that login until the user's next direct login there. Its part of a distributed query uses those profiles' settings and constraints, even if the user's groups have changed. ## Forwarding the token to external services {#token-forwarding}