Migrate ISF consumers from retired ccmstorageprod CSVs to toolkit open data

[RolandKrummenacher]

Summary

Fixes #2200 (the consumer side of #2090): the ccmstorageprod.blob.core.windows.net ISF files (AutofitComboMeterData.csv, isfratioblob.csv) stop updating 9 May 2026 and are removed 30 Aug 2026. This PR points every consumer at the InstanceSizeFlexibility.csv open data file introduced in #2199, so the switch ships in the same release that first publishes the CSV as a release asset.

Changes

Power BI shared datasets (storage + kql)

• InstanceSizeFlexibility table now loads the 3-column open data file from releases/latest/download/… — the same pattern as PricingUnits/Regions/ResourceTypes.
• Dropped the 6 connector-only columns (RegionId, combo/hardware/software meters, composite key, normalized SKU) that the Catalogs API doesn't expose, plus the derived normalized-key column.
• x_CommitmentDiscountFlexRatio changed int64 → double (real ratios are fractional, e.g. 4.44).
• Rewired the InstanceSizeFlexibility ↔ ReservationRecommendations relationship from the removed composite key to ReservationRecommendations.x_CommitmentDiscountNormalizedSize → InstanceSizeFlexibility.x_SkuType (many-to-one) — safe because [Open Data] Add instance size flexibility dataset from Azure Catalogs API #2199 guarantees and test-enforces ArmSkuName uniqueness.
• Removed the obsolete ccmstorageprod authorization bullet from all 12 report help pages (the github.com bullet already covers open data).

Optimization Engine

• Swapped 21 externaldata URLs across the reservations-usage, reservations-potential, and benefits-simulation workbooks. These use the raw GitHub URL (raw.githubusercontent.com/microsoft/finops-toolkit/dev/src/open-data/…) rather than the release-asset URL because externaldata does not follow the 302 redirect that release-asset downloads return. Column order (group, SKU, ratio) is identical to the old blob, so the queries themselves are unchanged.

Docs

• Removed the ccmstorageprod bullet from the Power BI setup and hubs deploy authorization lists; updated ms.date; added changelog entries under Unreleased.

Notes for reviewers

• The Q&A linguistic metadata (cultures/en-US.tmdl) still contains entries for the dropped columns. Power BI ignores bindings to removed columns and prunes them on next model save — hand-editing the 60k-line generated JSON seemed riskier than leaving it.
• releases/latest/download/InstanceSizeFlexibility.csv will 404 until the next release ships (the packaging script already includes all open-data/*.csv); published templates are unaffected since they ship with that release.
• Validation: all 15 edited JSON files parse (outer + embedded visual configs), docs link tests pass (1,509/1,509), TMDL changes follow the existing open-data table pattern. A Power BI Desktop refresh of both shared datasets would be a welcome extra check.

🤖 Generated with Claude Code

[Copilot]

🟢 Ready to approve

The changes consistently repoint ISF consumers to the new open-data source and the remaining feedback is limited to a minor consistency nit in the Power Query CSV import settings.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Migrates all in-repo Instance Size Flexibility (ISF) consumers away from the retiring ccmstorageprod CSVs to the toolkit’s ISF open data CSV, ensuring Power BI datasets and Optimization Engine workbooks continue to function after the upstream removal deadline.

Changes:

• Repointed Power BI shared datasets (storage + KQL) InstanceSizeFlexibility to releases/latest/download/InstanceSizeFlexibility.csv, reduced schema to the 3-column open-data shape, and updated relationships to join via ARM SKU name.
• Updated Optimization Engine reservation workbooks to load ISF data from raw.githubusercontent.com/.../src/open-data/InstanceSizeFlexibility.csv (redirect-free for externaldata).
• Removed obsolete ccmstorageprod authorization guidance from relevant report textboxes and Microsoft Learn setup/deploy docs; added changelog entries.

File summaries

File	Description
src/power-bi/storage/WorkloadOptimization.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/storage/Shared.Dataset/definition/tables/InstanceSizeFlexibility.tmdl	Switches ISF import to toolkit open data release asset; trims columns; updates ratio type to double.
src/power-bi/storage/Shared.Dataset/definition/relationships.tmdl	Rewires ISF ↔ reservation recommendations relationship to use normalized size → ARM SKU name.
src/power-bi/storage/RateOptimization.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/storage/Invoicing.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/storage/Governance.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/storage/DataIngestion.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/storage/CostSummary.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps.
src/power-bi/kql/WorkloadOptimization.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/power-bi/kql/Shared.Dataset/definition/tables/InstanceSizeFlexibility.tmdl	Same ISF open data repoint/schema reduction as storage dataset (KQL version).
src/power-bi/kql/Shared.Dataset/definition/relationships.tmdl	Same relationship rewiring as storage dataset (KQL version).
src/power-bi/kql/RateOptimization.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/power-bi/kql/Invoicing.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/power-bi/kql/Governance.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/power-bi/kql/DataIngestion.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/power-bi/kql/CostSummary.Report/report.json	Removes deprecated ISF authorization mention from in-report setup steps (KQL version).
src/optimization-engine/views/workbooks/reservations-usage.json	Replaces externaldata ISF URL with raw GitHub open data URL.
src/optimization-engine/views/workbooks/reservations-potential.json	Replaces externaldata ISF URL with raw GitHub open data URL.
docs-mslearn/toolkit/power-bi/setup.md	Removes deprecated ccmstorageprod authorization requirement and updates ms.date.
docs-mslearn/toolkit/hubs/deploy.md	Removes deprecated ccmstorageprod authorization requirement and updates ms.date.
docs-mslearn/toolkit/changelog.md	Adds Unreleased changelog entries describing the ISF consumer migration.

Review details

• Files reviewed: 22/22 changed files
• Comments generated: 2
• Review effort level: Low

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[MSBrett]

Review result: approve

Requirement	Evidence	Result
Remove retired ISF consumer URLs	All 15 changed JSON files parse; migrated optimization workbook endpoint is reachable (200).	Met
Power BI open-data source	Both shared datasets use the documented release asset and match the 3-column schema. The URL is currently 404 because latest is v14 and does not contain the asset; this is the PR's disclosed release-timed dependency. The next release must publish the CSV before its Power BI artifacts are used.	Met for the intended release rollout
Data-model correctness	The ratio is now double; the new ArmSkuName relationship key is unique in the current open-data file.	Met
Security and supply chain	No dependencies or secrets added; no actionable security finding.	Met
Existing inline comments	Both QuoteStyle suggestions were reviewed and replied to. They are behaviorally no-ops for this CSV, so no change is requested.	Addressed

Residual pre-release check: refresh both shared datasets and confirm the published release asset returns 200 after release publication.