feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists - #328
Open
eisber wants to merge 2 commits into
Open
Conversation
Registers the Microsoft Tabular model (TMSL / TMDL / TOM) as a well-known `custom_extensions` vendor, so Fabric and Power BI semantic models become a recognized spoke in the hub-and-spoke converter ecosystem. The token is `MICROSOFT`, not `POWER_BI`. One object model backs Power BI Desktop, Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services; a `model.bim` does not record which produced it, so a product-named token mislabels the other three. This also matches how Ossie already registers vendors -- tokens name the organization, and `SALESFORCE` already covers the differently-branded Tableau semantic layer. It agrees with apache#250, which adds `MICROSOFT` to `Vendor.examples`. The cost of a company token is that it is broader than the payload. That is handled by a `format` discriminator inside the envelope rather than by minting a second vendor token, because a payload labelled too broadly can be narrowed inside the envelope, whereas a token that is too narrow cannot be widened once it is written into files in the wild. Five places enumerated well-known vendors and all five disagreed, so this reconciles rather than appends: - `core-spec/spec.md` gained `OMNI` and `NVIDIA_GSF`, and is now declared the canonical list. - `converters/README.md` gained `GOODDATA` and `HONEYDEW`, and now points at the canonical list so the two cannot drift again. - `core-spec/osi-schema.json` `Vendor.examples` gained `HONEYDEW`, `OMNI` and `NVIDIA_GSF`. - `core-spec/spec.yaml`'s example comment was resynced. - `python/src/ossie/models.py` `OSIVendor` gained `HONEYDEW`, `OMNI` and `NVIDIA_GSF`, plus a docstring noting it is advisory (`vendor_name` is typed `str`, not this enum). `vendor_name` stays a free-form string; nothing here turns it into an enum. Also fixes `vendor_name: Databricks` in the spec's example block, which was the only mixed-case token in a document that is otherwise SCREAMING_SNAKE throughout. `docs/vendor_extensions/microsoft.md` documents what belongs in the payload: DAX and format strings, display folders, `isHidden`, hierarchies, calculation groups, RLS/OLS roles, partitions and storage mode (Import / DirectQuery / Dual / Direct Lake), perspectives, and lineage tags -- keyed by TMSL property name so every key is directly lookup-able in the public reference. It defers to apache#250 for the `DAX` dialect rather than duplicating it, and cites public documentation only. `examples/microsoft_extension.yaml` is a worked example that passes `python validation/validate.py` today; it carries no `DAX` dialect entry because that enum member arrives with apache#250. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Registers MICROSOFT as a well-known custom_extensions.vendor_name token (for Microsoft Tabular / TMSL/TMDL/TOM) and reconciles the previously drifting vendor-token lists across spec/docs/schema/Python, adding vendor-extension documentation and a worked example.
Changes:
- Adds
MICROSOFT(plus missing existing vendors) to the canonical vendor-token registry incore-spec/spec.md, and syncs other vendor lists to reference it. - Introduces
docs/vendor_extensions/microsoft.mddefining the MICROSOFT extension envelope, placement rules, and key catalog. - Adds
examples/microsoft_extension.yamlas a worked, validator-tested example of MICROSOFT vendor extensions.
Show a summary per file
| File | Description |
|---|---|
| python/src/ossie/models.py | Expands OSIVendor advisory enum and clarifies it is not normative. |
| examples/microsoft_extension.yaml | Adds a worked MICROSOFT vendor-extension example across model/dataset/field/metric/relationship levels. |
| docs/vendor_extensions/microsoft.md | Defines the MICROSOFT extension envelope, placement, and round-trip rules with references. |
| docs/index.md | Adds a docs index entry for vendor extension conventions. |
| core-spec/spec.yaml | Updates vendor-name comment to point at the canonical registry in spec.md. |
| core-spec/spec.md | Establishes canonical vendor token table, adds MICROSOFT entry + Microsoft extension example, and fixes Databricks example token casing. |
| core-spec/osi-schema.json | Updates Vendor.examples and clarifies examples are non-exhaustive. |
| converters/README.md | Reconciles supported vendor table and points to canonical vendor-token list + vendor extension docs. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 8/8 changed files
- Comments generated: 2
- Review effort level: Lite
Review found the example's `sortByColumn` naming `MonthNumber`, a column no field in the `date` dataset defines. The same class of defect appeared a second time, unflagged: the `Calendar` hierarchy's `Year` level names `CalendarYear`, also undefined. Both now exist as ordinary fields -- `month_number` hidden, as a sort key normally is -- so every reference in the example resolves. This mattered more than a tidy-up. A sort key preserved *only* inside a vendor payload is not preserved at all: it dangles for every consumer that reads core fields, and the construct it supports silently loses its target. The doc never stated the rule, so the example had nothing to be wrong against. It now does, under "Which name space a value refers to", along with the neighbouring trap -- these references are TMSL names, not Ossie field names, and translating them into Ossie names breaks replay just as thoroughly as omitting the field. Also corrects the relationship example, whose comment described preserving an inactive relationship while the payload set `isActive: true`. The payload was right and the comment wrong: an inactive relationship must not be emitted as an ordinary Ossie relationship at all, since that asserts a join the engine does not apply by default. `isActive` is written on every relationship, active ones included, because a reader that finds no flag cannot tell "active" from "converted by a tool that dropped it". Comment and the `isActive` row in the doc both say this now. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Registers the Microsoft Tabular model (TMSL / TMDL / TOM) as a well-known
custom_extensionsvendor, so Power BI and Fabric semantic models can be a first-class spoke in the hub-and-spoke converter ecosystem.This is the registration slice only — no converter implementation. It is deliberately small so the naming question can be settled before code depends on the answer.
The token is
MICROSOFT, notPOWER_BIThere are currently two tokens in flight for the same vendor: #250 adds
MICROSOFTtoVendor.examples, while work in progress elsewhere usesPOWER_BIin the converters table. If both land, the spec ships contradictory guidance andcustom_extensionswritten by one tool are silently invisible to the other.This PR settles it on
MICROSOFT, agreeing with #250:model.bimdoes not record which produced it, and a converter reading one cannot tell. A product-named token is therefore wrong for three of the four inputs.SNOWFLAKE,DATABRICKS,GOODDATA,HONEYDEW— andSALESFORCEalready covers a differently-branded product, the Tableau semantic layer.nvidiato match its token. UnderMICROSOFTaconverters/microsoft/directory and anossie_microsoftpackage already agree; underPOWER_BIthey would need exactly the churn [OSSIE][NVIDIA_GSF] Rename GSF converter directory and package to NVIDIA #299 is doing.The honest objection to a company token is that it over-covers: Microsoft ships semantic surfaces that are not Tabular models. That is answered inside the payload, with a
formatdiscriminator in the envelope, rather than by minting a second vendor token. This picks the recoverable error over the unrecoverable one — a payload labelled too broadly can be narrowed within the envelope, whereas a token that is too narrow cannot be widened once it is written into files in the wild."Register both" is explicitly rejected.
MICROSOFTis canonical for reading and writing, and readers must not acceptPOWER_BIas an alias. A converter that stashes under one token and reads the other loses data on round-trip, and a reader that quietly accepts two spellings makes a disagreeing writer undetectable — the first symptom would be a silently lost round trip.Reconciling the vendor lists
Five places enumerated well-known vendors, and all five already disagreed with each other on
main. This PR reconciles them rather than appending to one:core-spec/spec.md"Vendor Names"OMNI,NVIDIA_GSFMICROSOFTconverters/README.md"Supported Vendors"GOODDATA,HONEYDEWMICROSOFT, and now points at the canonical listcore-spec/osi-schema.jsonVendor.examplesHONEYDEW,OMNI,NVIDIA_GSFMICROSOFTcore-spec/spec.yamlcommentpython/src/ossie/models.pyOSIVendorHONEYDEW,OMNI,NVIDIA_GSFMICROSOFT, + docstringcore-spec/spec.mdis now named as the single canonical list, and the other four point at it, so they cannot silently drift apart again.vendor_nameremains a free-form string. Nothing here turns it into an enum — the schema keeps{"type": "string", "examples": [...]}, the description now says outright that the examples are not a closed set, andOSICustomExtension.vendor_namestays typedstr.OSIVendoris documented as advisory.Drive-by fix
The
Custom Extensionsexample block usedvendor_name: Databricks— the only mixed-case token in a document that is SCREAMING_SNAKE throughout, and inconsistent with theDATABRICKSrow in the table directly above it. Changed toDATABRICKS. Flagging it explicitly rather than burying it: it is a one-word docs change with no schema effect (vendor_nameis free-form, so both were always valid), but it is a change to a published example.What gives the registration substance
docs/vendor_extensions/microsoft.mddocuments what actually belongs incustom_extensions[MICROSOFT]— otherwise a registration is just a table row. It covers the constructs Ossie core cannot express, per level (model / dataset / field / metric / relationship):isHiddenvisibilitysortByColumn,summarizeBy, relationship cardinality and cross-filter behaviourDesign points worth reviewing:
_vversion and aformatdiscriminator in the envelope, with "fail loudly, never guess" rules for both.metrics[].expressionunder theDAXdialect, not in the extension. There is an explicit table of constructs that must stay in core fields, so the two cannot become competing sources of truth.Everything in it is reconstructible from public documentation — learn.microsoft.com's TMSL/TMDL/TOM/DAX/Direct Lake references and the public
microsoft/semantic-link-labsrepository, all cited at the bottom of the file.examples/microsoft_extension.yamlis a worked example covering model, dataset, field, metric and relationship placement.Relationship to #250
This PR agrees with and depends on nothing from #250, and the two can merge in either order.
MICROSOFTtoVendor.examples; this PR'sVendor.examplesline is a strict superset of feat: Add DAX, and examples to the Dialects #250's, so the conflict is a trivial one-line rebase whichever lands first.DAXdialect —$defs.Dialect, the spec dialect table,OSIDialect, and thevalidate.pyskip list — comes entirely from feat: Add DAX, and examples to the Dialects #250 and is not duplicated here.docs/vendor_extensions/microsoft.mddefers to feat: Add DAX, and examples to the Dialects #250 by reference.examples/microsoft_extension.yamldeliberately carries noDAXdialect entry yet, so it validates on today'smain. Once feat: Add DAX, and examples to the Dialects #250 lands, the DAX-dialect form documented in the vendor doc becomes usable.Worth knowing while #250 is open:
validate.pycurrently hands aDAXexpression to sqlglot as if it were SQL. Simple DAX happens to parse —CALCULATE(SUM('Sales'[Amount]), 'Date'[Year] = 2024)is accepted — while realistic DAX usingVAR/RETURNfails. #250's skip-list entry fixes this; noting it here as supporting evidence for that part of #250.Other overlapping PRs
python/src/ossie/models.py, but removesvendors: Optional[list[OSIVendor]]from the root model, which is different lines from this change.OSIVendorremains an exported public enum, so both changes are compatible.ai_context/custom_extensions) — no conflict; this PR deliberately places nothing at the document root.Open questions for reviewers
SEMANTIDOappears inOSIVendorand in no other list — not in the spec, the schema, or any converter. I left it untouched rather than guess. Someone who knows what it is should either document or remove it.members, or drop it as unnecessary personal data? The doc flags the question rather than deciding it.ORIONBELTandPOLARIShave converters in-repo but no registered vendor token.docs/vendor_extensions/the right home for per-vendor payload conventions, or should this live undercore-spec/? It is a new convention introduced by this PR.converters/README.mdkeep its own vendor table at all, or just link the canonical one? Two tables that must agree is what caused the current drift.Related Issues
Relates to #250.
Checklist
Specification
core-spec/and follow the existing structureCONTRIBUTING.md, changes touchingcore-spec/need adev@ossie.apache.orgthread and a minimum 7-day discussion period. This PR is opened to give that discussion something concrete to point at; the dev@ announcement still needs to be sent before it can merge.Ontology
ontology/are consistent with spec changes — n/a, no ontology change.Converters
converters/is updated to reflect spec or ontology changes — n/a by design; this is the registration slice only, no converter is added or changed.Validation
validation/are updated if the spec changed — no change needed;vendor_nameis free-form, so no validation rule keys off the token. (TheDAXskip-list entry belongs to feat: Add DAX, and examples to the Dialects #250.)Documentation
docs/is updated to reflect any user-facing changesCONTRIBUTING.mdis updated if the contribution process changed — n/a, unchanged.Examples
examples/are added or updated for any new spec constructs or converter supportTests
python validation/validate.pypasses onexamples/microsoft_extension.yaml,examples/tpcds_semantic_model.yamlandexamples/flights.yaml;pytest python/testsis 9 passed.Compliance