Skip to content

feat(feature-flags): generalize driver-side cache - #974

Open
cathleeny wants to merge 9 commits into
mainfrom
cathleeny/general-driver-flags
Open

cathleeny wants to merge 9 commits into
mainfrom
cathleeny/general-driver-flags

Conversation

@cathleeny

@cathleeny cathleeny commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Generalize the existing feature-flag cache for driver-owned flags on the Thrift path and before kernel/session initialization, once authenticated transport is available.

  • Share raw flag values by workspace ID, with normalized host as a fallback; keep credentials and HTTP clients caller-owned.
  • Add typed Boolean, int32, int64, double, string, and string-list getters. Integer validation uses standard-library fixed-width types.
  • Keep the blocking initial fetch and background refresh, retain stale values on refresh failures, and avoid queuing duplicate refreshes.

How is this tested?

  • Full non-real-kernel unit suite: 1,032 passed, 5 skipped, 1 deselected; 351 subtests passed.
  • Focused telemetry/feature-flag suite: 58 passed.
  • Black and mypy checks passed.
  • Covers all six types, defaults/range validation, workspace sharing, and caller-owned refresh/authentication.

Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:30 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:30 — with GitHub Actions Active
@cathleeny cathleeny added the kernel-e2e Trigger preview run of the Kernel E2E workflow on this PR label Oct 7, 2026
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:32 — with GitHub Actions Active
@cathleeny
cathleeny marked this pull request as ready for review October 7, 2026 17:38
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:39 — with GitHub Actions Active
@cathleeny
cathleeny requested review from a team, jay-xiao446 and vuanhphung and removed request for a team October 7, 2026 17:39

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Solid, well-tested generalization of the feature-flag cache — the typed getters correctly exclude bool-as-int, enforce fixed-width ranges, and reject NaN/inf, and the shared _CacheState + RLock refresh coordination is sound. One low-severity lifecycle note: get_instance is now created eagerly per non-kernel connection while remove_instance has no production caller, so the refresher executor/cache is never cleaned up.

Comment thread src/databricks/sql/session.py Outdated
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@github-actions github-actions Bot removed the kernel-e2e Trigger preview run of the Kernel E2E workflow on this PR label Oct 7, 2026

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks solid overall — the typed getters are correct and well-covered (bool/int range via ctypes, double finiteness, strict list typing), and the workspace-keyed sharing with host fallback is sensible and tested. One low-severity lifecycle concern: remove_instance (which shuts down the shared refresh executor and evicts cache state) has no production caller, so the executor and per-workspace cache leak for the process lifetime. Minor nit below.

Nit (no anchor needed): the typed getters' default_value parameters (get_int32/get_int64/get_double/get_string/get_string_list, and _get_int) lack type hints, unlike the annotated return types; adding them would match the repo's type-hint convention (CONTRIBUTING.md).

Comment thread src/databricks/sql/common/feature_flag.py
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 22:53 — with GitHub Actions Active

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a clean, well-tested generalization of the feature-flag cache (shared _CacheState per workspace, typed JSON getters, refresh dedup, pre-session reader). One low-severity note: get_bool is a stricter parser than the str(...).lower() == "true" telemetry gate it replaces, so the telemetry flag now depends on the server emitting a bare lowercase JSON boolean. Nit (summary-only): the typed getters annotate return types but leave default_value/integer_type/name params untyped in _get_int, get_int32/64, get_double, etc. — CONTRIBUTING.md asks for type hints; adding default_value: Optional[int] = None style annotations would keep them consistent with get_bool/get_string.

Comment thread src/databricks/sql/common/feature_flag.py Outdated
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@jay-xiao446 jay-xiao446 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg in general. left a comment

Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 2 Low

Solid refactor that generalizes the feature-flag cache to a per-workspace _CacheState with caller-owned transport and typed getters; the type(value) is int/bool checks correctly exclude bools, and stale-on-refresh-failure semantics are preserved. Two low concerns noted inline: a latent stale-executor RuntimeError path after remove_instance shuts the pool down, and the first-fetch network call now running under the shared per-workspace lock. Nits (summary-only): _refresh_flags swallows all exceptions with no logging (the module imports no logger), so failed fetches are completely silent; and the get_double integer-OverflowError branch (float(value) on an out-of-float-range int) appears untested — the 1e400 case exercises the JSON-inf path instead.

Comment thread src/databricks/sql/common/feature_flag.py
Comment thread src/databricks/sql/common/feature_flag.py

This branch was successfully deployed

1 active deployment
azure-prod — fb647df0 Deployed Oct 9, 2026 by peco-review-bot[bot] via followup #983
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants