Commit c95ca3f
feat(kernel): route Azure Entra auth (azure-sp-m2m + azure-oauth) through the auth bridge (#919)
* feat(kernel): thread Azure Entra OAuth (U2M + SP M2M) through the auth bridge
The kernel auth bridge rejected azure-oauth and had no azure-sp-m2m path.
Route both Azure auth types onto the kernel's generic OAuth flows (the
kernel needs no Azure-specific code; PR databricks/databricks-sql-kernel#263
added the token_url/scope override plumbing this relies on):
- azure-oauth (Azure AD U2M) -> oauth-u2m with the Azure app client id
(96eecda7-...), redirect port 8030, and the {app_id}/user_impersonation
offline_access delegated scope (via AzureOAuthEndpointCollection, honoring
DATABRICKS_AZURE_TENANT_ID). The kernel discovers endpoints via the
workspace /oidc redirector. (PECOBLR-4120)
- azure-sp-m2m (Azure service principal) -> oauth-m2m with the Azure creds,
an Entra v2.0 token_url, and the {effective_app_id}/.default scope.
Requires an explicit azure_tenant_id (the kernel path does not
auto-discover it). The management-token header / azure_workspace_resource_id
are not applied on the kernel path -- no SQL connector uses them, matching
Go and Node. (PECOBLR-4141)
kernel_auth_kwargs now takes hostname (for the effective Azure app id);
the client passes self._server_hostname.
TDD: replaced the azure-oauth NotSupportedError test with routing tests and
added a TestKernelAzureSpM2M suite (routing, required tenant/creds,
federation client id). 50 bridge tests pass; black clean.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* docs: update CONNECTION_PARAMETERS.md for kernel Azure OAuth support
The auth table marked the azure_* fields as Kernel-unsupported and
claimed azure-oauth 'still works on the kernel' (it was actually
rejected). Reflect the new routing: azure-sp-m2m + azure-oauth now work
on the kernel path; azure_tenant_id is required there; the management
token / azure_workspace_resource_id are not applied (matching Go/Node).
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* refactor(kernel): thin azure-oauth — kernel owns Azure resolution
The kernel is the auth core now: for azure-oauth the bridge forwards only
auth_type='azure-oauth' (+ optional client_id/redirect_port passthrough),
and the kernel pins the workspace v2.0 authorize/token endpoints, the
Azure app client id, port 8030, and the user_impersonation scope. Drops
the connector-side endpoint/scope construction (and the
AzureOAuthEndpointCollection / PYSQL_OAUTH_AZURE_* imports) from the
kernel path. Live-verified end-to-end against an Azure workspace.
azure-sp-m2m still routes to oauth-m2m here pending the kernel's
dedicated azure-sp-m2m variant.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* refactor(kernel): thin azure-sp-m2m — kernel owns Azure resolution + optional mgmt token
Make the `azure-sp-m2m` bridge thin, matching the kernel becoming the
Azure-aware auth core. The connector now forwards
`auth_type='azure-sp-m2m'` + `azure_client_id` / `azure_client_secret`
(and optional `azure_tenant_id` / `azure_workspace_resource_id`) straight
to the kernel Session, instead of constructing the Entra token endpoint
and `{app_id}/.default` scope itself.
Behavior changes on the kernel path (Thrift parity):
- `azure_tenant_id` is now OPTIONAL — the kernel auto-discovers the
tenant from the workspace's `/aad/auth` redirect when omitted, exactly
as the Thrift backend does. (Previously the kernel path required it.)
- `azure_workspace_resource_id` is now honored as an optional add-on:
forward it and the kernel fetches an Azure-management token and sends
the `X-Databricks-Azure-SP-Management-Token` +
`X-Databricks-Azure-Workspace-Resource-Id` pair, so an SP with only an
Azure RBAC role (not a workspace member) can authenticate. (Previously
it was dropped with a warning.)
Also thread the `azure_*` connection kwargs into `kernel_auth_options`
in session.py — without this the bridge never received them and
`azure-sp-m2m` failed at session-open with "requires azure_client_id".
Adds a regression test for that threading, and rewrites the bridge tests
for thin forwarding (tenant optional, resource id forwarded). Drops the
now-unused `get_effective_azure_login_app_id` import and
`_AZURE_AAD_LOGIN_HOST` constant.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* refactor(kernel): drop azure-sp-m2m management token on the kernel path
Mirror the kernel-side simplification: the kernel's `azure-sp-m2m` no longer
supports the Azure management-token flow (RBAC-only SPs), so stop forwarding
`azure_workspace_resource_id` to the kernel. The SP must be a workspace member
on `use_kernel=True`.
`azure_workspace_resource_id` is still a valid connection parameter for the
Thrift path (unchanged). On the kernel path, setting it now logs a warning
(rather than silently dropping a security-relevant auth parameter and failing
later with an opaque 403) and the SP authenticates with the Databricks-audience
data token alone. Drops the threading through `kernel_auth_options` in
session.py. RBAC-only-SP support is a documented follow-up.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* refactor(kernel): reject azure-oauth on the kernel path; point to databricks-oauth
The kernel dropped its dedicated Azure U2M flow, so `auth_type="azure-oauth"`
on `use_kernel=True` is now rejected with a clear pointer to
`databricks-oauth` — whose in-house OAuth browser flow works against Azure
workspaces (the workspace federates login to Entra). This replaces the thin
forward-the-selector branch. `azure-sp-m2m` is unchanged; `azure-oauth` on the
Thrift path is unaffected. Rejecting (rather than silently remapping) keeps the
user's selected flow explicit.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* feat(kernel): forward azure_workspace_resource_id for complete Azure SP M2M
The kernel's azure-sp-m2m now always sends the Azure SP management token and,
when a workspace resource id is set, the X-Databricks-Azure-Workspace-Resource-Id
header — matching the Thrift connector. So the bridge forwards
azure_workspace_resource_id to the kernel (previously warned-and-ignored), and
session.py threads it into kernel_auth_options.
This completes Azure SP M2M parity: a service principal with an Azure RBAC role
but no workspace membership now authenticates on the kernel path too.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* docs(kernel): azure-sp-m2m management token is gated on azure_workspace_resource_id
Correct the wording (bridge docstring/comments, CHANGELOG, CONNECTION_PARAMETERS)
to match the kernel: the Azure SP management token +
X-Databricks-Azure-Workspace-Resource-Id header are sent only when
azure_workspace_resource_id is set (JDBC parity), not always. A workspace-member
SP authenticates with the data token alone; the management token is only needed
for an RBAC-only SP (Azure role, not a workspace member), which is what the
resource id signals.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* feat(kernel): route azure-oauth (Azure AD U2M) to the kernel OAuth U2M flow
On use_kernel=True, auth_type='azure-oauth' was rejected with a pointer to
databricks-oauth. Instead route it to the kernel's oauth-u2m, identically to
databricks-oauth: the kernel runs the in-house workspace-federated browser
flow, which Azure Databricks workspaces support (the workspace federates the
login to Microsoft Entra).
It forwards the connector's databricks-sql-python app bundle, NOT the Thrift
Azure app (96eecda7 / redirect port 8030): that app is registered for Thrift's
direct-Entra flow (login.microsoftonline.com config + {tenant}/user_impersonation
scopes), which the kernel does not perform. So on the kernel path azure-oauth
and databricks-oauth are the same flow.
The U2M-vs-M2M ambiguity guard now also covers azure-oauth, so
azure-oauth + oauth_client_secret is rejected as ambiguous rather than
silently falling through to the M2M branch.
Connector-only; the kernel already exposes oauth-u2m. (PECOBLR-4120.)
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* ai: apply changes for #919 (2 review threads)
Addresses:
- #3811121476 at src/databricks/sql/backend/kernel/auth_bridge.py:158
- #3824120741 at tests/unit/test_kernel_auth_bridge.py:36
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* ai: apply changes for #919 (1 review thread)
Addresses:
- #3828326882 at src/databricks/sql/backend/kernel/auth_bridge.py:200
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* ai: apply changes for #919 (1 review thread)
Addresses:
- #3828367785 at src/databricks/sql/backend/kernel/auth_bridge.py:265
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* ai: apply changes for #919 (1 review thread)
Addresses:
- #3828404026 at src/databricks/sql/backend/kernel/auth_bridge.py:279
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* ai: apply changes for #919 (1 review thread)
Addresses:
- #3828445075 at src/databricks/sql/backend/kernel/auth_bridge.py:285
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* ci(kernel): bump KERNEL_REV to the build carrying the pyo3 Azure SP surface
Point kernel-e2e's KERNEL_REV at the kernel commit that adds the pyo3
azure-sp-m2m surface (databricks-sql-kernel#263), so the connector's kernel-e2e
builds a kernel wheel that can accept auth_type='azure-sp-m2m'. Temporary pin to
the unmerged #263 branch tip; re-point to a kernel main SHA once #263 merges and
a release is cut. Customer-facing pin (pyproject databricks-sql-kernel ^0.2.0)
still needs a release bump to a published kernel with #263.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
* ai: apply changes for #919 (1 review thread)
Addresses:
- #3828603835 at src/databricks/sql/backend/kernel/auth_bridge.py:18
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
* fix(kernel): make auth_bridge.py pass black + mypy
Two quality-check regressions from the azure-sp-m2m addition:
- mypy [no-redef]: the new azure-sp-m2m branch's `kwargs` assignment now
precedes the JWT branch's annotated `kwargs: Dict[str, Any]`. Move the
annotation onto the first (azure-sp-m2m) assignment so the later ones
are plain reassignments to the same annotated name.
- black: split the over-long `ignored_signals` tuple.
Verified locally: `black --check src` clean, `mypy src` reports no issues.
Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
---------
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Co-authored-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>1 parent e550cb7 commit c95ca3f
8 files changed
Lines changed: 386 additions & 83 deletions
File tree
- src/databricks/sql
- backend/kernel
- tests/unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
72 | | - | |
| 72 | + | |
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | | - | |
| 85 | + | |
| 86 | + | |
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
0 commit comments