Skip to content

[needs visual review] #1138 FinOps index/object heatmap redesign (both phases, both apps)#1237

Merged
erikdarlingdata merged 2 commits into
devfrom
feature/1138-finops-heatmaps
Jun 25, 2026
Merged

[needs visual review] #1138 FinOps index/object heatmap redesign (both phases, both apps)#1237
erikdarlingdata merged 2 commits into
devfrom
feature/1138-finops-heatmaps

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Implements #1138 — the FinOps index/object heatmap redesign, both phases, both apps (Dashboard + Lite), on existing collected data. Follows the 3-review-round plan in plans/finops-index-heatmap-redesign.md.

This PR is code-complete but needs your visual sign-off — I cannot launch the apps from the worktree (single-instance would evict your running Dashboard/Lite and kill the MCP servers), so the heatmap rendering, cell shading, and drill navigation need an eyes-on pass. The code itself is done: both apps build green, tests pass, and all new T-SQL was validated live against a real PerformanceMonitor database (SQL2025).

Phase 1 — Object-growth heatmap + Storage Growth drill

Replaces the two flat tabs Object Sizes & Growth (#5, removed) and Index Usage (#6, folded) with a drill rooted at Storage Growth (#4):

  • Storage Growth grid → double-click a database row (or the new "Show objects" context-menu item) → that database's object-growth heatmap: rows = top-N objects by reserved-MB growth over the window, X = daily buckets, color = absolute reserved MB (reuses the query-heatmap renderer's 0→NaN behavior unchanged, so a blank cell = object absent/empty that day). A companion per-object grid sits below it.
  • Double-click an object → its per-index detail (size + the old Index Usage seeks/scans/lookups/updates).
  • Back/breadcrumb navigation; a window selector (7/30/90d, default 30); an in-UI substrate caveat ("object / reserved footprint … may not explain file-level growth").
  • Producer is two-step + DB-scoped for perf (plan §3A): rank the top-N cheaply via the 2-snapshot diff, then series-scan only those — never the uncovered all-objects 90-day scan that caused [BUG] index_object_stats collector times out (30s) as a single all‑or‑nothing cross‑database sweep - yields no index data #1135. Rollup key is name-based (database, schema, table) (plan K1).
  • The standalone Object Sizes / Index Usage MCP read methods are untouched (kept for the MCP tools).

Phase 2 — Locking color-scaled wait-type grid + drill

Turns Locking & Contention (#7) into a heatmap-in-grid (plan §3B — not a ScottPlot heatmap, so no M1/M2 issues):

  • The four *_wait_in_ms cells are shaded per-column (log scale, max=0 guarded) by their wait category's hue — row/page lock = Lock, page latch = Buffer Latch, page-io-latch = Buffer IO — via a shared themed value→brush converter. The numeric values stay visible in every cell.
  • A database selector (per-database latest, so "all databases" shows each DB's newest row); double-click an index → full lock/latch counters + identity, with back.
  • Read layer gains an optional DB filter, a per-DB-latest CTE, and the two latch *_count columns; a new GetIndexLockingDatabasesAsync feeds the selector.
  • Doc-drift fix (plan R7): install/02, install/55, and the Lite collector header all claimed lock-wait deltas are "computed in the read layer" — no such logic exists and M1 shows it wouldn't be safe; corrected to "raw cumulative totals, no delta".

Shared / parity

To keep the two hand-duplicated apps from drifting, the parity-critical logic is shared once:

  • PerformanceMonitor.Common/FinOpsHeatmap.cs — pure ranking, long→matrix pivot, per-column log color-scale.
  • PerformanceMonitor.Ui/FinOpsHeatmapRenderer.cs — the query-heatmap renderer with count couplings stripped (MB colorbar, M/d axis).
  • PerformanceMonitor.Ui/HeatIntensityToBrushConverter.cs — themed cell shading.

Tests

  • Pure unit tests in both Dashboard.Tests + Lite.Tests: top-N ranking, the long→matrix pivot (incl. missing-cell=0 + ordering), and the per-column color-scale math (incl. the max=0 guard).
  • Lite integration tests (DuckDB): the object-growth producer (growth ranking + daily series), the per-index detail, and the locking DB selector/filter.
  • Both apps build green; new T-SQL validated live on SQL2025.

Reviewer-driven fixes already folded in

  • Deterministic ORDER BY tiebreaker on the growth ranking so Lite's two queries (and Dashboard vs Lite) always pick the same top-N set + order; the heatmap rows and companion grid now share one ranking authority.
  • T-SQL polish on the producer (COUNT_BIG, unique-clustered #ranked, TABLOCK, OPTION hints, CONVERT over CAST).

Not in this PR

No merge — the only thing gating merge is your visual sign-off. No newly collected columns (read-layer + UI only). The other adjacent tabs (#8 Database Sizes is per-file, #9/#10/#11) are unchanged per the plan's scope.

🤖 Generated with Claude Code

erikdarlingdata and others added 2 commits June 25, 2026 11:33
Replace the standalone Object Sizes & Growth (#5) and Index Usage (#6) FinOps
tabs with a Storage Growth (#4) parent -> object -> index drill:

- Storage Growth grid: double-click a database row (or "Show objects") drills to
  that DB's object-growth heatmap (rows = top-N objects by reserved-MB growth over
  the window, X = daily buckets, color = absolute reserved MB) plus a companion
  per-object grid; double-click an object drills to its per-index size + usage.
  Back/breadcrumb navigation; an in-UI substrate caveat (object reservation vs
  file allocation).
- Shared foundations so both apps render identically (no drift): pure
  PerformanceMonitor.Common/FinOpsHeatmap (top-N ranking, long->matrix pivot,
  per-column log color-scale); PerformanceMonitor.Ui/FinOpsHeatmapRenderer (the
  query-heatmap renderer with count couplings stripped: MB colorbar, M/d axis);
  PerformanceMonitor.Ui/HeatIntensityToBrushConverter (themed cell shading, used
  by Phase 2).
- Read layer (both apps): GetObjectGrowthHeatmapDataAsync ranks the top-N cheaply
  (2-snapshot diff) then series-scans only those, DB-scoped on the covering index
  lead column to avoid the #1135 uncovered scan; GetObjectIndexDetailAsync for the
  index leaf. Existing GetObjectSizeGrowthAsync / GetIndexUsageAsync stay for MCP.
- Tests: FinOpsHeatmapBuilder ranking + long->matrix pivot, in both test projects.

#7 Locking & Contention is untouched here (it is Phase 2's home).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dex drill (both apps)

Turn the Locking & Contention tab into a heatmap-in-grid: the four *_wait_in_ms
cells are shaded per-column (log scale, max=0 guarded) by their wait category's
hue (row/page lock = Lock, page latch = Buffer Latch, page-io-latch = Buffer IO)
via the shared HeatIntensityToBrushConverter, with the numeric values still
visible. Cumulative snapshot, no delta (plan 3B), so this sidesteps the M1/M2
issues a lock-wait-over-time chart would hit.

- DB selector (per-database latest, so "all databases" shows every DB's newest
  row); double-click an index -> full lock/latch counters + identity, with back.
- Read layer (both apps): GetIndexLockingAsync gains an optional database filter,
  a per-database-latest CTE, and the two latch *_count columns; new
  GetIndexLockingDatabasesAsync feeds the selector. Existing no-arg call kept for MCP.
- Fix the R7 doc-drift: install/02, install/55, and the Lite collector header all
  claimed lock-wait deltas are "computed in the read layer" - no such logic exists
  and M1 shows it would not be safe; corrected to "raw cumulative totals, no delta".
- Tests: ColumnLogIntensities (the per-column color-scale math) in both projects;
  Lite integration tests for the object-growth producer, the index detail, and the
  locking DB selector/filter.

Review fixes folded in: deterministic ORDER BY tiebreaker on the object-growth
ranking so Lite's two queries (and Dashboard/Lite) always pick the same top-N set
and order; the heatmap rows + companion grid now share one ranking authority
(FinOpsHeatmapBuilder.RankTopGrowers); T-SQL polish on the Phase-1 producer
(COUNT_BIG, unique clustered #ranked, TABLOCK, OPTION hints, CONVERT). All new
T-SQL validated live against a real PerformanceMonitor database.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@erikdarlingdata erikdarlingdata merged commit d17c741 into dev Jun 25, 2026
6 checks passed
@erikdarlingdata erikdarlingdata deleted the feature/1138-finops-heatmaps branch June 25, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant