Render the database ERD as a searchable Mermaid diagram instead of an image

[ColtenOuO]

Sumarry

Render the database ERD reference pages (core, FAB, Edge3) as a Mermaid erDiagram instead of an SVG image, so table and column names are indexed by search engines and in-page search instead of being locked inside a picture.

closes: #36842

Before / after

Before — the current production docs (apache-airflow 3.4.0) embed the ERD as a flat SVG image. On a schema this size (57 core tables) the image is dense and hard to read even at full screen, and none of the table/column text is selectable or searchable:

After — same page, rendered as a Mermaid diagram from this branch. Every table/column name is now real page text: here the browser's in-page search (Cmd+F) for "INTEGER" finds 50 matches on the FAB provider's ERD page:

Same on the Edge3 provider's ERD page (a much smaller schema, so also easier to read as a bonus) — 7 matches for "INTEGER":

What changed

• generate_erd (the Sphinx extension that (re)builds the diagram from the live SQLAlchemy models on every doc build) now writes Mermaid ER-diagram markup (.mmd) instead of an SVG.
• The three database-erd-ref.rst pages (airflow-core, providers/fab, providers/edge3) use the .. mermaid:: directive (already available via sphinxcontrib-mermaid, already used elsewhere in the docs) instead of .. image::.
• Dropped the graphviz/dot availability check and its SVG placeholder — Mermaid markup is emitted directly by eralchemy without shelling out to graphviz, so that whole fallback path no longer applies.
• Updated .gitignore and contributing-docs/14_metadata_database_updates.rst for the new generated filename.

Why this wasn't done sooner

An earlier attempt (#42323) stalled because it tried to introduce a new tool (paracelsus) and wasn't sure whether generating from live models vs. the post-migration database would let the diagram drift out of sync with the models. Both concerns are now moot: generate_erd already regenerates from the live SQLAlchemy MetaData on every doc build (introduced after that PR), and the eralchemy version already pinned in devel-common (eralchemy==1.7.0) can emit Mermaid ER-diagram markup directly — no new dependency needed.

Note: eralchemy's own render_er(..., mode="mermaid_er") wraps its output in an HTML comment plus a mermaid.ink-hosted image link (meant for GitHub-flavored markdown READMEs), which isn't right for a Sphinx .. mermaid:: directive and would reintroduce an external-image dependency. This PR calls eralchemy's lower-level all_to_intermediary/filter_resources/_intermediary_to_mermaid_er functions directly to get the raw erDiagram ... markup instead.

Testing

• Ran the extension against the real core/FAB/Edge3 SQLAlchemy models and confirmed clean Mermaid markup is produced (57 core tables).
• Built all three affected packages with breeze build-docs --package-filter apache-airflow --package-filter apache-airflow-providers-fab --package-filter apache-airflow-providers-edge3 — build and spellcheck succeeded for all three, and the rendered HTML contains the ERD text (e.g. dag_run) directly in the page rather than in an image, with no mermaid.ink reference.
• prek run --stage pre-commit passes on the changed files, including mypy for devel-common.

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Sonnet 5)

[ColtenOuO]

Two follow-up items I'd like reviewer input on — neither blocks this PR.

1. CI doesn't catch changes to generate_erd.py in isolation

selective-checks' DOC_FILES group (in selective_checks.py) is what decides whether the docs-build job runs — and that job is the only thing that actually exercises generate_erd end-to-end (importing the models, building the MetaData, emitting Mermaid markup, and feeding it through the .. mermaid:: directive). Its patterns include ^devel-common/src/docs but not ^devel-common/src/sphinx_exts, where generate_erd.py actually lives.

This PR's own docs-build run is triggered fine, because it also touches airflow-core/docs/ and providers/*/docs/. But a future PR that touches only generate_erd.py (or anything else under devel-common/src/sphinx_exts/) wouldn't trigger a docs build at all — it could break the extension (import error, bad Mermaid syntax, whatever) and CI would stay green.

Proposal: add ^devel-common/src/sphinx_exts to the DOC_FILES pattern list in selective_checks.py, plus the matching doc/test updates (04_selective_checks.md and test_selective_checks.py). Happy to open that as a separate PR if the direction sounds right.

2. The ERD is dense and doesn't have real zoom/pan

On the core schema (57 tables) the diagram is dense enough that even full-screen isn't very readable.

sphinxcontrib-mermaid supports real client-side pan/zoom via mermaid_d3_zoom = True, which loads d3.js and lets users scroll-to-zoom / drag-to-pan the rendered SVG (as opposed to whole-page browser zoom, which just scales the already-cramped layout uniformly and doesn't help). Two reasons I left it out of this PR:

• It's a Sphinx-wide conf.py setting (app.config), not something scoped per-diagram — turning it on would affect every .. mermaid:: diagram across the docs (e.g. the flowcharts in security_model.rst / jwt_token_authentication.rst), not just the three ERD pages.
• By default it pulls d3.min.js from cdn.jsdelivr.net. These docs otherwise seem to lean toward mirroring external JS in (see swagger_mirror_external_resources = True in airflow-core/docs/conf.py), so doing this properly would mean also setting d3_use_local and vendoring/hosting d3.min.js ourselves — more surface area than I wanted to add unreviewed in the same PR as the SVG→Mermaid switch.

Proposal: if there's appetite for this, I'd want to confirm the CDN-vs-local-hosting call with a maintainer before implementing, given the existing convention around external resources. Let me know if it's worth a follow-up PR, and if so which way to go on that.