Skip to content

feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists - #328

Open
eisber wants to merge 2 commits into
apache:mainfrom
eisber:dev/marcozo/settle-the-microsoft-power-bi-ossie-vend-tjrqxn
Open

feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists#328
eisber wants to merge 2 commits into
apache:mainfrom
eisber:dev/marcozo/settle-the-microsoft-power-bi-ossie-vend-tjrqxn

Conversation

@eisber

@eisber eisber commented Aug 14, 2026

Copy link
Copy Markdown

Summary

Registers the Microsoft Tabular model (TMSL / TMDL / TOM) as a well-known custom_extensions vendor, so Power BI and Fabric semantic models can be a first-class spoke in the hub-and-spoke converter ecosystem.

This is the registration slice only — no converter implementation. It is deliberately small so the naming question can be settled before code depends on the answer.

The token is MICROSOFT, not POWER_BI

There are currently two tokens in flight for the same vendor: #250 adds MICROSOFT to Vendor.examples, while work in progress elsewhere uses POWER_BI in the converters table. If both land, the spec ships contradictory guidance and custom_extensions written by one tool are silently invisible to the other.

This PR settles it on MICROSOFT, agreeing with #250:

  1. One object model, four products. The Tabular model — scripted as TMSL, authored as TMDL, manipulated through TOM — is the storage format for Power BI semantic models, Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services. A model.bim does not record which produced it, and a converter reading one cannot tell. A product-named token is therefore wrong for three of the four inputs.
  2. It matches existing precedent. Ossie tokens name the organization — SNOWFLAKE, DATABRICKS, GOODDATA, HONEYDEW — and SALESFORCE already covers a differently-branded product, the Tableau semantic layer.
  3. It costs no renames. [OSSIE][NVIDIA_GSF] Rename GSF converter directory and package to NVIDIA #299 is currently renaming the GSF converter directory and package to nvidia to match its token. Under MICROSOFT a converters/microsoft/ directory and an ossie_microsoft package already agree; under POWER_BI they would need exactly the churn [OSSIE][NVIDIA_GSF] Rename GSF converter directory and package to NVIDIA #299 is doing.

The honest objection to a company token is that it over-covers: Microsoft ships semantic surfaces that are not Tabular models. That is answered inside the payload, with a format discriminator in the envelope, rather than by minting a second vendor token. This picks the recoverable error over the unrecoverable one — a payload labelled too broadly can be narrowed within the envelope, whereas a token that is too narrow cannot be widened once it is written into files in the wild.

"Register both" is explicitly rejected. MICROSOFT is canonical for reading and writing, and readers must not accept POWER_BI as an alias. A converter that stashes under one token and reads the other loses data on round-trip, and a reader that quietly accepts two spellings makes a disagreeing writer undetectable — the first symptom would be a silently lost round trip.

Reconciling the vendor lists

Five places enumerated well-known vendors, and all five already disagreed with each other on main. This PR reconciles them rather than appending to one:

Location Was missing Now
core-spec/spec.md "Vendor Names" OMNI, NVIDIA_GSF canonical list, + MICROSOFT
converters/README.md "Supported Vendors" GOODDATA, HONEYDEW + MICROSOFT, and now points at the canonical list
core-spec/osi-schema.json Vendor.examples HONEYDEW, OMNI, NVIDIA_GSF + MICROSOFT
core-spec/spec.yaml comment 4 tokens resynced to the canonical list
python/src/ossie/models.py OSIVendor HONEYDEW, OMNI, NVIDIA_GSF + MICROSOFT, + docstring

core-spec/spec.md is now named as the single canonical list, and the other four point at it, so they cannot silently drift apart again.

vendor_name remains a free-form string. Nothing here turns it into an enum — the schema keeps {"type": "string", "examples": [...]}, the description now says outright that the examples are not a closed set, and OSICustomExtension.vendor_name stays typed str. OSIVendor is documented as advisory.

Drive-by fix

The Custom Extensions example block used vendor_name: Databricks — the only mixed-case token in a document that is SCREAMING_SNAKE throughout, and inconsistent with the DATABRICKS row in the table directly above it. Changed to DATABRICKS. Flagging it explicitly rather than burying it: it is a one-word docs change with no schema effect (vendor_name is free-form, so both were always valid), but it is a change to a published example.

What gives the registration substance

docs/vendor_extensions/microsoft.md documents what actually belongs in custom_extensions[MICROSOFT] — otherwise a registration is just a table row. It covers the constructs Ossie core cannot express, per level (model / dataset / field / metric / relationship):

  • DAX expressions (and where they don't go), format strings, display folders, isHidden visibility
  • hierarchies, calculation groups, perspectives
  • RLS and OLS roles
  • partitions and storage mode — Import / DirectQuery / Dual / Direct Lake
  • lineage tags, sortByColumn, summarizeBy, relationship cardinality and cross-filter behaviour

Design points worth reviewing:

  • Keys are TMSL property names spelled exactly as TMSL spells them. No synonyms, no re-casing — so any key in a payload can be looked up directly in the public reference.
  • Preservation is a deny-list, not an allow-list. A converter should carry across TMSL properties it has never heard of. New properties appear over time; an allow-list turns each one into silent data loss.
  • A _v version and a format discriminator in the envelope, with "fail loudly, never guess" rules for both.
  • The extension carries only what has no core home. A measure's DAX belongs in metrics[].expression under the DAX dialect, not in the extension. There is an explicit table of constructs that must stay in core fields, so the two cannot become competing sources of truth.
  • Security is called out as absent. An Ossie document converted from a model with RLS/OLS describes the unsecured shape and must not be treated as a security boundary.

Everything in it is reconstructible from public documentation — learn.microsoft.com's TMSL/TMDL/TOM/DAX/Direct Lake references and the public microsoft/semantic-link-labs repository, all cited at the bottom of the file.

examples/microsoft_extension.yaml is a worked example covering model, dataset, field, metric and relationship placement.

Relationship to #250

This PR agrees with and depends on nothing from #250, and the two can merge in either order.

Worth knowing while #250 is open: validate.py currently hands a DAX expression to sqlglot as if it were SQL. Simple DAX happens to parse — CALCULATE(SUM('Sales'[Amount]), 'Date'[Year] = 2024) is accepted — while realistic DAX using VAR / RETURN fails. #250's skip-list entry fixes this; noting it here as supporting evidence for that part of #250.

Other overlapping PRs

Open questions for reviewers

  1. SEMANTIDO appears in OSIVendor and in no other list — not in the spec, the schema, or any converter. I left it untouched rather than guess. Someone who knows what it is should either document or remove it.
  2. RLS role membership can name identities. Should a converter preserve members, or drop it as unnecessary personal data? The doc flags the question rather than deciding it.
  3. ORIONBELT and POLARIS have converters in-repo but no registered vendor token.
  4. Is docs/vendor_extensions/ the right home for per-vendor payload conventions, or should this live under core-spec/? It is a new convention introduced by this PR.
  5. Should converters/README.md keep its own vendor table at all, or just link the canonical one? Two tables that must agree is what caused the current drift.

Related Issues

Relates to #250.

Checklist

Specification

  • Spec changes are included in core-spec/ and follow the existing structure
  • Spec changes have been discussed on the mailing list or in a linked issue — not yet done. Per CONTRIBUTING.md, changes touching core-spec/ need a dev@ossie.apache.org thread and a minimum 7-day discussion period. This PR is opened to give that discussion something concrete to point at; the dev@ announcement still needs to be sent before it can merge.
  • Breaking changes to the spec are clearly called out in the summary — there are none. Adding examples to a free-form string field and adding rows to a documentation table cannot invalidate an existing document.

Ontology

  • Ontology changes in ontology/ are consistent with spec changes — n/a, no ontology change.
  • New or modified terms are defined and documented

Converters

  • Converter logic in converters/ is updated to reflect spec or ontology changes — n/a by design; this is the registration slice only, no converter is added or changed.
  • New converters include tests under the converter's test directory — n/a, no new converter.

Validation

  • Validation rules in validation/ are updated if the spec changed — no change needed; vendor_name is free-form, so no validation rule keys off the token. (The DAX skip-list entry belongs to feat: Add DAX, and examples to the Dialects #250.)
  • New validation cases are covered by tests

Documentation

  • docs/ is updated to reflect any user-facing changes
  • New features or behaviors are documented with examples where appropriate
  • CONTRIBUTING.md is updated if the contribution process changed — n/a, unchanged.

Examples

  • examples/ are added or updated for any new spec constructs or converter support

Tests

  • All existing tests pass — python validation/validate.py passes on examples/microsoft_extension.yaml, examples/tpcds_semantic_model.yaml and examples/flights.yaml; pytest python/tests is 9 passed.
  • New functionality is covered by tests — the new example is exercised by the validator.

Compliance

  • ASF license headers are present on all new source files — both new files carry the standard header.
  • No third-party dependencies are added without PMC/IPMC approval — none added.

Registers the Microsoft Tabular model (TMSL / TMDL / TOM) as a well-known
`custom_extensions` vendor, so Fabric and Power BI semantic models become a
recognized spoke in the hub-and-spoke converter ecosystem.

The token is `MICROSOFT`, not `POWER_BI`. One object model backs Power BI
Desktop, Fabric semantic models, Azure Analysis Services and SQL Server
Analysis Services; a `model.bim` does not record which produced it, so a
product-named token mislabels the other three. This also matches how Ossie
already registers vendors -- tokens name the organization, and `SALESFORCE`
already covers the differently-branded Tableau semantic layer. It agrees with
apache#250, which adds `MICROSOFT` to `Vendor.examples`.

The cost of a company token is that it is broader than the payload. That is
handled by a `format` discriminator inside the envelope rather than by minting
a second vendor token, because a payload labelled too broadly can be narrowed
inside the envelope, whereas a token that is too narrow cannot be widened once
it is written into files in the wild.

Five places enumerated well-known vendors and all five disagreed, so this
reconciles rather than appends:

- `core-spec/spec.md` gained `OMNI` and `NVIDIA_GSF`, and is now declared the
  canonical list.
- `converters/README.md` gained `GOODDATA` and `HONEYDEW`, and now points at
  the canonical list so the two cannot drift again.
- `core-spec/osi-schema.json` `Vendor.examples` gained `HONEYDEW`, `OMNI` and
  `NVIDIA_GSF`.
- `core-spec/spec.yaml`'s example comment was resynced.
- `python/src/ossie/models.py` `OSIVendor` gained `HONEYDEW`, `OMNI` and
  `NVIDIA_GSF`, plus a docstring noting it is advisory (`vendor_name` is typed
  `str`, not this enum).

`vendor_name` stays a free-form string; nothing here turns it into an enum.

Also fixes `vendor_name: Databricks` in the spec's example block, which was the
only mixed-case token in a document that is otherwise SCREAMING_SNAKE
throughout.

`docs/vendor_extensions/microsoft.md` documents what belongs in the payload:
DAX and format strings, display folders, `isHidden`, hierarchies, calculation
groups, RLS/OLS roles, partitions and storage mode (Import / DirectQuery /
Dual / Direct Lake), perspectives, and lineage tags -- keyed by TMSL property
name so every key is directly lookup-able in the public reference. It defers to
apache#250 for the `DAX` dialect rather than duplicating it, and cites
public documentation only.

`examples/microsoft_extension.yaml` is a worked example that passes
`python validation/validate.py` today; it carries no `DAX` dialect entry
because that enum member arrives with apache#250.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 14, 2026 17:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Registers MICROSOFT as a well-known custom_extensions.vendor_name token (for Microsoft Tabular / TMSL/TMDL/TOM) and reconciles the previously drifting vendor-token lists across spec/docs/schema/Python, adding vendor-extension documentation and a worked example.

Changes:

  • Adds MICROSOFT (plus missing existing vendors) to the canonical vendor-token registry in core-spec/spec.md, and syncs other vendor lists to reference it.
  • Introduces docs/vendor_extensions/microsoft.md defining the MICROSOFT extension envelope, placement rules, and key catalog.
  • Adds examples/microsoft_extension.yaml as a worked, validator-tested example of MICROSOFT vendor extensions.
Show a summary per file
File Description
python/src/ossie/models.py Expands OSIVendor advisory enum and clarifies it is not normative.
examples/microsoft_extension.yaml Adds a worked MICROSOFT vendor-extension example across model/dataset/field/metric/relationship levels.
docs/vendor_extensions/microsoft.md Defines the MICROSOFT extension envelope, placement, and round-trip rules with references.
docs/index.md Adds a docs index entry for vendor extension conventions.
core-spec/spec.yaml Updates vendor-name comment to point at the canonical registry in spec.md.
core-spec/spec.md Establishes canonical vendor token table, adds MICROSOFT entry + Microsoft extension example, and fixes Databricks example token casing.
core-spec/osi-schema.json Updates Vendor.examples and clarifies examples are non-exhaustive.
converters/README.md Reconciles supported vendor table and points to canonical vendor-token list + vendor extension docs.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 8/8 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread examples/microsoft_extension.yaml
Comment thread examples/microsoft_extension.yaml
Review found the example's `sortByColumn` naming `MonthNumber`, a column no
field in the `date` dataset defines. The same class of defect appeared a second
time, unflagged: the `Calendar` hierarchy's `Year` level names `CalendarYear`,
also undefined. Both now exist as ordinary fields -- `month_number` hidden, as
a sort key normally is -- so every reference in the example resolves.

This mattered more than a tidy-up. A sort key preserved *only* inside a vendor
payload is not preserved at all: it dangles for every consumer that reads core
fields, and the construct it supports silently loses its target. The doc never
stated the rule, so the example had nothing to be wrong against. It now does,
under "Which name space a value refers to", along with the neighbouring trap --
these references are TMSL names, not Ossie field names, and translating them
into Ossie names breaks replay just as thoroughly as omitting the field.

Also corrects the relationship example, whose comment described preserving an
inactive relationship while the payload set `isActive: true`. The payload was
right and the comment wrong: an inactive relationship must not be emitted as an
ordinary Ossie relationship at all, since that asserts a join the engine does
not apply by default. `isActive` is written on every relationship, active ones
included, because a reader that finds no flag cannot tell "active" from
"converted by a tool that dropped it". Comment and the `isActive` row in the
doc both say this now.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants