Round trip spec-built models through netcdf and benchmark from_spec

[FabianHofmann]

Step 4 of the plan in #919, stacked on #922: a spec-built model survives to_netcdf / read_netcdf and Model.copy(), and Model.from_spec gets a codspeed benchmark.

Note

The following content was generated by AI.

Changes proposed in this Pull Request

• to_netcdf writes the spec YAML text as the top-level attribute _linopy_spec, next to _linopy_version, following the relaxed-registry and piecewise-formulation precedent. read_netcdf re-lowers the Program from that text; a file without a spec loads without math-spec installed.
• One extra sub-dataset under the prefix spec- carries what model.parameters cannot round-trip on its own: master coordinates as coords__<dim> data variables (an index coordinate no retained parameter reaches would otherwise be dropped on read), and lookups plus every string-kind parameter as pandas.factorize output, codes__<name> (int32, -1 for a hole) plus cats__<name> with the in-memory dtype in a variable attribute. No netcdf type carries an object array of strings with NaN: netCDF4 returns <U and turns a hole into an empty string, scipy narrows int64 to int32. Codes plus categories round-trip str->str, str->int, int->str and int->int lookups, full and partial, exactly on both engines. This resolves the two lookup items left open in feat(spec): xarray data binder for math-spec programs #921.
• Model.copy() carries the spec; the accessor is rebound to the copy. On read, every parameter is cast back to its in-memory dtype and the master coordinates are stamped onto all containers, so one dimension has one dtype even after the scipy engine narrows int64 to int32. assert_model_equal now compares parameters with dtypes. linopy/spec/testing.py holds the synthetic_sources generator shared by tests and benchmark.
• assert_model_equal compares the spec text; parameters, coordinates and lookups are already covered through model.parameters.
• test/test_spec_io.py: round trips for retain="report"|"all"|"none" on both engines, a solved model whose named expression is identical before and after, evaluate on a restored retain="none" model, exact-dtype assertions per lookup kind, copy semantics, and math-spec's examples/pypsa.yaml with synthetic data when MATH_SPEC_EXAMPLES points at a math-spec checkout.
• benchmarks/models/spec_pypsa.py: from_spec on examples/pypsa.yaml plus the two netcdf phases; skips without math-spec or MATH_SPEC_EXAMPLES. Running it in CI needs math-spec installed and the examples directory available in the codspeed workflow, which waits for a math-spec PyPI release.

Verification

uv run --with <math-spec> pytest test/test_spec_binder.py test/test_spec_builder.py test/test_spec_io.py test/test_io.py -q
465 passed, 155 skipped
uv run pytest test/test_io.py test/test_spec_io.py -q          (without math-spec)
93 passed, 16 skipped
uv run --with <math-spec> mypy linopy benchmarks
Success
MATH_SPEC_EXAMPLES=<math-spec>/examples uv run --with <math-spec> --with pytest-benchmark pytest benchmarks -k spec --benchmark-disable
6 passed, 1 skipped

Checklist

• AI-generated content is marked (see AGENTS.md).
• Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
• Unit tests for new features were added (if applicable).
• A note for the release notes doc/release_notes.rst of the upcoming release is included.

[github-actions]

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04516 (1.08)   │   15.03 (1.00) 
 (v1)        0.04195 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04851 (1.03)   │   12.04 (1.00) 
 (v1)        0.04728 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0202 (1.12)   │   15.20 (1.0) 
 (v1)       0.01803 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03288 (1.17)   │   44.93 (1.0) 
 (v1)        0.02801 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0236 (1.20)   │   11.51 (1.0) 
 (v1)       0.01972 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04954 (1.01)   │   24.34 (1.06) 
 (v1)         0.0492 (1.0)   │    23.05 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05658 (1.03)   │   16.12 (1.00) 
 (v1)        0.05489 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.01183 (1.06)   │   752.18 (1.10) 
 (v1)        0.01112 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.01175 (1.03)   │   3.12 (1.33) 
 (v1)        0.01136 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03965 (1.41)   │   126.16 (1.44) 
 (v1)        0.02809 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04025 (1.42)   │   126.16 (1.44) 
 (v1)        0.02843 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03863 (1.32)   │   126.16 (1.44) 
 (v1)        0.02925 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02825 (1.03)   │    715.14 (1.0) 
 (v1)        0.02736 (1.0)   │   787.74 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02639 (1.02)   │   4.55 (1.27) 
 (v1)        0.02586 (1.0)   │    3.59 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1948 (1.12)   │   704.12 (1.09) 
 (v1)        0.1738 (1.0)   │    647.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.1971 (1.07)   │   18.34 (1.0) 
 (v1)        0.1837 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.2042 (1.12)   │   9.54 (1.0) 
 (v1)         0.182 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03763 (1.08)   │   3.77 (1.12) 
 (v1)        0.03498 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03861 (1.08)   │   216.59 (1.10) 
 (v1)        0.03587 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.02042 (1.16)   │   938.49 (1.0) 
 (v1)        0.01753 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02064 (1.16)   │   9.66 (1.0) 
 (v1)        0.01787 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02029 (1.15)   │   5.32 (1.0) 
 (v1)        0.01763 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0979 (1.05)   │   946.85 (1.06) 
 (v1)       0.09283 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0982 (1.13)   │   12.01 (1.0) 
 (v1)       0.08655 (1.0)   │   12.01 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02371 (1.05)   │   147.70 (1.06) 
 (v1)        0.02262 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02489 (1.10)   │   2.60 (1.09) 
 (v1)         0.0226 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02032 (1.15)   │   696.75 (1.03) 
 (v1)        0.01766 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05881 (1.24)   │   137.97 (1.0) 
 (v1)        0.04724 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03482 (1.13)   │   69.22 (1.0) 
 (v1)        0.03084 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02401 (1.17)   │   402.33 (1.00) 
 (v1)        0.02046 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02251 (1.07)   │   3.19 (1.19) 
 (v1)        0.02102 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.02597 (1.13)   │   29.00 (1.54) 
 (v1)        0.02296 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0342 (1.22)   │   37.95 (1.43) 
 (v1)       0.02792 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05199 (1.05)   │    410.93 (1.0) 
 (v1)        0.04963 (1.0)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05431 (1.05)   │     9.94 (1.0) 
 (v1)        0.05184 (1.0)   │   10.22 (1.03) 

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

[FabianHofmann]

Folded into #922 (fast-forwarded spec-builder onto this branch) so build, fold and persistence are reviewed together.