feat(roofmodel): derive PV array geometry from Lantmäteriet open geodata - #735
Open
HuggeK wants to merge 21 commits into
Open
feat(roofmodel): derive PV array geometry from Lantmäteriet open geodata#735HuggeK wants to merge 21 commits into
HuggeK wants to merge 21 commits into
Conversation
This was referenced Aug 4, 2026
This was referenced Aug 7, 2026
Member
|
This PR currently has right of way on Please rebase onto current |
This was referenced Aug 7, 2026
6 tasks
Member
|
Release coordination for the 2.0 candidate: the exact beta-to-stable artifact promotion fix touches Is it OK to land #876 first and then rebase this draft? I will not change the RoofModel branch or code. If you need a different order, please say so. |
This was referenced Aug 16, 2026
Member
|
Rebased onto #734, which sits on #962 (
Force-pushed HuggeK |
…coverage STRANG becomes a real irradiance source rather than an unreferenced client, every external data source now says where in the world it works, and the location picker moves off Leaflet. STRANG's parameter set was mapped against the live API because SMHI's apidocs pages 404: exactly 116-122 exist. Identification was confirmed by physics rather than by guessing - at solar noon 121 + 122 = 723.0 + 87.5 = 810.5, exactly parameter 117, and 119 caps at 60, i.e. minutes within the hour. STRANG publishes no cloud cover; it is a radiation model. Cloudiness is instead derived from sunshine duration as 1 - minutes/60, which is observed rather than inferred but coarser: blind to thin cirrus, undefined at night. CloudCover() therefore returns an explicit unknown instead of defaulting to clear, because those two lead to opposite decisions. The new coverage registry makes an existing silence explicit. STRANG is Nordic-only and every price provider is European, so sites elsewhere were getting empty results with no explanation (srcfl#726). GET /api/data-sources now reports area, countries, licence and whether each source reaches this site, and the Weather tab renders it under the map. Bounds are advisory: STRANG's grid is rotated, so a lat/lon box can only ever be a superset - all four in-box corners were probed and returned no data. False is definitive, true means worth trying. Scoring now declines to start outside the domain instead of retrying nightly forever. Stacked on srcfl#718, which carries the plane-of-array wiring this builds on. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
…a CDN Upstream ships Leaflet on the box (srcfl#910) for a reason that applies just as hard to its replacement: the box UI must not execute third-party JS from a CDN, and the map must load when the gateway cannot reach the internet. So MapLibre GL JS 6.0.0 moves into web/vendor/maplibre — the ESM entry, its code-split chunk, its worker, the stylesheet and the license — and the loader imports it same-origin, which also spares MapLibre its blob: worker detour. The now-unused vendored Leaflet goes away with its test; maplibre-vendor.test.mjs pins the new contract, including the per-request referrerPolicy opt-in that keeps OSM volunteer tile servers from 403ing under the box's Referrer-Policy: no-referrer. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
The package comment linked the legacy strang.smhi.se site and claimed the apidocs 404 — they live at opendata.smhi.se/metanalys/strang, with the model described under smhi.se/forskning. The empirical parameter-code confirmation stays: the docs name the quantities, not the code-to-quantity mapping. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
http.ServeFile resolves Content-Type through the operating system's MIME table — the registry on Windows — and a host that maps .mjs to text/plain does not merely mislabel the vendored MapLibre entry: the app sends X-Content-Type-Options: nosniff, so the browser is required to refuse the module, and the weather map dies with 'failed to fetch dynamically imported module'. Seen live on a Windows host serving the demo build. handleStatic now pins the Content-Type for every asset kind the web tree ships (.html .css .js .mjs .svg .png .jpg), so nothing depends on host MIME state. Covered by a test that would have caught this wherever it ran. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Same four-file code-split layout as 6.0.0, from the maplibre-gl npm package's dist/. The entry still imports its shared and worker chunks by the vendored names and the shared chunk still honours the per-request referrerPolicy the OSM tiles depend on. Map mount, building picker and the TerraDraw PV drawing tool re-verified in headless Edge on the new build; served entry hash matches the tarball byte for byte. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
A Swedish site can stop typing panel angles in by hand. The new roofmodel/ module reads Lantmateriet LiDAR around the site and recovers each roof face's tilt, azimuth and usable area, which pre-fill weather.pv_arrays. Structured as a separate Python package alongside optimizer/, reached at arm's length: core spawns it, passes coordinates and the operator's own Geotorget credentials, and reads one versioned roof_model.json from stdout. Segmentation drags in a compiled point-cloud stack and runs for minutes, so a time-boxed subprocess keeps it off the control tick and lets it be absent entirely - the normal case, since the data exists only for Sweden. The pipeline follows the SPAN method: iterative RANSAC pulls one surface at a time out of the cloud, then DBSCAN splits faces sharing a plane equation but not a location, because two wings of a building fit the same plane and are not the same roof. Method only - no code is taken from SPAN's GPL QGIS plugin, and the dependencies (numpy, scikit-learn, requests) are all BSD. SWEREF 99 TM is implemented directly rather than via pyproj: it is one projection with fixed parameters, and a full PROJ build is disproportionate for that. Verified against the projection's exact analytic properties - easting on the central meridian is exactly 500000 at every latitude - and by sub-millimetre round trips from Smygehuk to Treriksroeset. Applying derived arrays to config is deliberately a separate act. Derivation is a best guess from a point cloud that may be years old, and silently rewriting an operator's panel config is a change they should make knowingly. Also generalises the __pycache__ ignore rule, which was optimizer-specific and would otherwise need repeating per module. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
The roofmodel tests passed on Windows and failed on Linux, which is where CI
runs them. Two separate bugs, both caused by stubbing the module with a shell
script:
- The stub handed back "sh" as the command, and the service invokes it as
`<cmd> -m ftw_roofmodel ...`. dash reads that as "run the script named
ftw_roofmodel", cannot open it, and exits 2. The stub never ran at all, so
five tests failed against an error that had nothing to do with what they
were asserting.
- The module-error stub echoed an unquoted JSON document. dash strips the
double quotes, so `{"error":"..."}` reached the parser as `{error:...}` and
could not be unmarshalled. cmd.exe keeps them, which is the only reason
that test ever passed.
The stub is now this test binary re-executed with an environment variable
telling it which behaviour to play. Nothing goes through a shell, so there is
no quoting to get wrong and no per-platform behaviour to diverge. The timeout
test consequently runs everywhere instead of being skipped on Windows.
Two assertions the shell stub could not make: that the site, credentials and
radius actually survive the process boundary, and that PYTHONPATH is set --
without it the module is only importable if installed system-wide, which on a
Pi it is not. Also that --vostok stays absent unless configured, so a GPL tool
is never invoked by default.
The Python module's tests did not run in CI at all: the workflow runs
`pytest -q optimizer/tests` and nothing else, so 75 tests covering the plane
fitting every derived tilt and azimuth depends on were never executed upstream.
Added a roofmodel job mirroring the optimizer's, wired into the required-check
gate so a failure blocks rather than being reported and ignored. It installs
without the `geo` extra on purpose: LAZ decoding pulls a compiled backend, and
everything except the point-cloud read is exercised without it.
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
Settings -> Weather gains a Geotorget credential form and a building picker. Press "Find buildings here" and the footprints near the marker are drawn on the map and listed beside it; click yours, press "Read roof from LiDAR", and the PV arrays fill in with one entry per usable roof face. The form is filled but nothing is saved. The operator sees the numbers, corrects what is wrong and presses Save. FTW does not rewrite a panel configuration on its own: the derivation is a guess from a scan that may be years old, and only the operator knows whether that face has panels on it. Picking a building is not cosmetic. Without a footprint the module segments whatever stands inside its search radius, and the plane fitting is global -- a fitted plane is infinite, so a roof at azimuth 180 is z = f(y) with no x term and extends across the whole tile. A second building sharing that ridge orientation lands inside its inlier band however far away it is, and the two lose returns to each other. Measured on a synthetic pair: a detached garage recovered 93% of its true area and split into two fragments while coplanar with the house, against 100% and one clean face once clipped to its own footprint. The clip buffers the outline by a metre first, because roofs overhang their walls and the eaves carry the lowest returns. Frames are detected rather than assumed. GeoJSON mandates WGS84 but Lantmateriet publishes this catalogue in SWEREF 99 TM and its STAC search takes a SWEREF bbox, so ring coordinates are classified by magnitude -- six- and seven-figure numbers are projected metres, degrees never are. New GET /api/roofmodel/buildings lists footprints as GeoJSON, honouring an explicit lat/lon so the picker can search where the marker is rather than where the last save put it. POST /api/roofmodel/derive accepts a building_id. GET /api/roofmodel reports has_credentials so the UI can stop asking. The Geotorget token now masks and restores like every other secret. It never appears in an API response, and saving an unrelated setting no longer wipes it -- the settings form returns the blank it was given, which without PreserveMaskedSecrets would have deleted the stored credential. docs/roof-geometry.md covers ordering the two Geotorget products, what the derived kWp does and does not mean (an upper bound on what fits, not what is installed), and what each failure message is telling you. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
Both roof-geometry products are STAC APIs behind one Geotorget account, and they differ only in what their items point at: Byggnad Nedladdning, vektor delivers GeoPackage, Laserdata Nedladdning, Skog delivers LAZ organised as COPC. Assets are now selected by declared media type rather than by guessing at asset key names, so a catalogue that calls its asset "punktmoln" instead of "data" keeps working, and a thumbnail is never handed back as a point cloud. Building footprints are read straight out of the GeoPackage with the standard library. A GeoPackage is a SQLite database holding geometry as WKB, both published formats with fixed layouts, so this costs about a hundred lines against a GDAL dependency that will not install on a Pi without a compiler. Previously only inline STAC geometry was handled and the asset-backed case -- the normal one -- returned nothing at all. Because COPC indexes points into an octree, picking a building now also makes the download small: only the octree nodes covering that footprint are range-requested, instead of a 2.5 km tile running to hundreds of megabytes. Plain .laz assets, hosts that ignore Range, and builds of laspy without COPC support all fall back to reading the tile whole -- slower, same answer -- and the model records which path ran as source.fetch, since that also decides what returns_in_radius is counted over. Also corrects the coplanar-absorption measurement quoted in buildings.py and docs/roof-geometry.md. A fresh run does not reproduce the 93%/two-fragment figures: one pass over a house and a garage 40 m apart consumes all 576 of the house's south-face returns and all 256 of the garage's as one surface, and gives identical output at every separation from 3 m to 40 m -- which is the signature of a global infinite-plane fit rather than a proximity effect. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
… from Rings read from a GeoPackage arrive x=easting first, the way GIS files store them. Rings converted from inline WGS84 geometry were stored the way wgs84_to_sweref99tm returns them: northing first. Everything downstream assumed the second convention, so every GeoPackage-sourced building -- the normal Lantmateriet case -- reported its centroid near (4 N, 63 E) in the Indian Ocean, an 8 000 km "distance", and a picker map with nothing visible on it. The suite missed it because every assertion was axis-blind: areas and SWEREF centroids survive a consistent swap. It surfaced the first time the module ran against a server and a human looked at where the buildings landed. ring_sweref now always holds (easting, northing): rings from degrees are swapped into it, and projected rings are normalised per point -- eastings stay under a million metres and northings start above six, so each point states its own order and an EPSG-registry-ordered export (north first) is folded in rather than mis-read. The WGS84 accessors and the site distance unpack accordingly. A regression test feeds the same square in both orders and requires real Stockholm coordinates and a sub-50 m distance back from each; the fixture files mis-unpacked the projection the same way the code did, and now match reality (LAZ x is an easting). Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
The roofmodel section's help text ends with two quoted product names, and the bubble cut off mid-sentence right before them. escHtml used the textContent/innerHTML trick, which never escapes quotes -- so every caller that builds an attribute (data-help, title, value) had its text terminated at the first embedded quote, with the remainder parsed as junk attribute names. Plain string replaces now cover both quote kinds. With the full text rendering, the second failure appeared: the bubble opens downward and the modal body is the scroll container that clips it, so a badge low in the last section has less room below than a long help text needs. settings.js now toggles .help-up on hover when the badge sits in the lower part of the visible modal, and the bubble grows upward into room that exists. Measured on hover because scroll position, not the badge, decides which way is open. Verified in headless Edge: the full text renders at both scroll positions, upward when low, downward when high. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
Derived arrays pre-fill weather.pv_arrays, which now stores rated_w. The Python module and Go host speak watts; kWp stays a test helper.
Lantmateriet provides no OAuth for its STAC download APIs, so the
credential is the operator's own Geotorget account username and password,
sent as HTTP Basic auth. The config keys become roofmodel.stac_username /
stac_password; the old geotorget_username/geotorget_token keys keep
working as aliases and migrate on the next save through the API.
While renaming the keys, the client sheds its Lantmateriet hardcodes:
stac_base_url, stac_buildings_collection, stac_lidar_collection and
stac_bbox_epsg point FTW at any STAC-conformant catalog (search is the
spec's POST {base}/search; the bbox CRS is per-catalog, 4326 per spec,
3006 for Lantmateriet). A custom catalog lifts the Sweden-only gate.
Point-cloud data must still arrive in SWEREF 99 TM metres.
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Open catalogs are the rule, not the exception: France's LiDAR HD (COPC via the MTD STAC API), Carinthia's KAGIS and swisstopo all answer STAC searches and serve point-cloud assets with no credentials at all. So a custom stac_base_url no longer demands a username and password: with neither half of a credential stored, core omits --username/--password entirely and the module's session goes anonymous. The default Lantmäteriet catalog still requires the operator's own Geotorget account, with the ordering instructions raised before any request. Half a credential stays an error everywhere, and a 401/403 on an anonymous catalog says to configure credentials rather than to check them. has_credentials on GET /api/roofmodel now reports "the catalog is usable as configured", so the UI stops asking for credentials an open catalog does not need. docs/roof-geometry.md gains the table of catalogs verified live today, with the honest caveats per catalog. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
… line Two faults that live testing with a real Geotorget account surfaced. Credentials typed into Settings never reached the service: it was built once at boot — nil when disabled — and no reload ever touched it, while GET /api/roofmodel read the live config and claimed has_credentials. The service is now always constructed, keeps its config behind a mutex, and the hot-reload applier swaps it like every other service. And when the module failed for a real reason, the operator saw 'exit status 1': requests' RequestsDependencyWarning shares stderr with the module's JSON error document, and parsing the whole stream failed. The contract is one JSON document as the final line, so parse that. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Verified end-to-end against the real API with a real account: the catalogue differs from what the module assumed in four ways. The service is two STAC roots, not one — byggnader on stac-vektor/v1, the point clouds on stac-hojd/v1 — so the pipeline runs one client per root for the defaults (a custom single-root catalog is unchanged). Laserdata Skog is published as dsm-skoglig-copc. Searches take the spec's WGS84 bbox, which the live probe confirmed. And buildings arrive as one ZIP-wrapped GeoPackage per municipality: Stockholm's holds 93,716 features with no per-row envelopes, so the reader unwraps the zip and clips rows to the search window by parsed geometry bounds — without that, the row limit truncated the table before it reached the site. A tile item's own geometry (the municipality outline) is no longer read as a building; a data asset wins, inline geometry is the fallback for catalogues whose items are buildings. Live result: 'Find buildings here' at the demo site returns 39 real footprints, nearest first, in ~4 s including the authenticated 13 MB municipality download. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
The roof section gains a data-catalog select: Lantmäteriet (the default, Geotorget account), the live-verified open catalogs — IGN LiDAR HD for France, KAGIS for Carinthia — and a custom entry that reveals the STAC API root and collection fields. Presets fill the same roofmodel.stac_* config keys the config file uses; open catalogs need no credentials, and the credential labels and the attribution line follow the pick. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
MapLibre rejected both footprint layers without throwing: the case condition needed a typed boolean, and the theme's oklch() tokens are unparseable to its colour parser, so themeColor now bakes the resolved value to sRGB bytes through a 1x1 canvas. A find also fits the camera to the nearby candidates (city zoom leaves a footprint smaller than a pixel), retries the draw if the style is still loading, and a click on a footprint selects that building instead of dragging the site pin - and the saved coordinates - to wherever you clicked. Verified in headless Edge against the live Geotorget service: 39 footprints rendered, map-click selection enables the derive button, pin coordinates unchanged. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Live derive with laspy[lazrs] returned 47 points for a 4,760 m2 window of a 217M-point tile. The tile's octree nodes carry z voxel keys from a different origin than the file's own cube - a level-6 node keyed to a slab at -1698..-1542 m holds points at +18..+42 m, x/y keys exact - so laspy's bounds pruning discarded every dense level and kept the sparse preview. The window query now spans the octree cube vertically; x/y pruning and laspy's exact post-filter still bound the read. Verified live end to end afterwards: the picked building derived 8 arrays from 9 roof planes out of the 2021-03-23 scan, over HTTP range requests (~7 MB moved, not the multi-GB tile). Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Reported live from a 522 m2 barn at an unsaved pin: 'building ... was not found near this site'. Two causes, both fixed: the derive now sends the picker's own (possibly dragged, unsaved) coordinates instead of silently using the stored site, and its re-find of the picked footprint reaches as far as the picker's 150 m search did rather than the 40 m LiDAR radius. The LiDAR tile lookup centres on the picked building, not the pin. The module also accepts --footprint-json, a hand-drawn [lon, lat] ring that clips exactly like a picked building - the UI for it rides the drawing PR above this one. Verified live at the reported site: the exact failing pick now derives 2 arrays from 2 roof planes out of the 2019-04-04 scan. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
HuggeK
marked this pull request as ready for review
September 2, 2026 19:57
miravoss26
reviewed
Sep 3, 2026
miravoss26
left a comment
miravoss26
left a comment
Contributor
There was a problem hiding this comment.
Adds an optional Lantmäteriet roof-geometry module: a Go service spawns a Python subprocess (STAC search + LiDAR/COPC segmentation + RANSAC plane fit) and turns the result into candidate PV arrays that pre-fill weather.pv_arrays. Also fixes a COPC octree z-key bug that was pruning nearly all LiDAR returns, a derive-vs-picker coordinate mismatch, and vendors MapLibre 6.7.0 (most of the +11257 is the vendored lib + generated test fixtures, not hand-written code).
Findings:
- Medium —
go/internal/roofmodel/roofmodel.go:293. The Geotorget/STAC password is passed as a subprocess CLI argument (args = append(args, "--password", p)). Process arguments are visible to other local processes viaps//proc/<pid>/cmdlinefor the life of the derive (up to the 10-minute default timeout). Prefer an env var or piping it via stdin instead. - Medium —
roofmodel/ftw_roofmodel/geotorget.py:260+:311.StacClientsets HTTP Basic-Auth on the wholerequests.Session(session.auth = (...)), anddownload(url)sends that session at whateverurlit's given with no check that the host matchesbase_url. Sinceroofmodel.stac_base_urlis explicitly designed to point at any third-party STAC catalog, a malicious or compromised catalog could return an asset href on an attacker-controlled host in a search response and receive the operator's Geotorget credentials on the follow-up download. Low risk on the default Lantmäteriet path (own hosts only), real risk the moment a custom catalog is configured.
Secret-masking (config.go MaskSecrets/PreserveMaskedSecrets) itself is solid and well tested (roofmodel_secrets_test.go) — the finding is specifically about the two places the plaintext secret leaves Go's process boundary. Safe to merge from my read; worth tightening the credential handling before or shortly after.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Derives PV array tilt, azimuth and rated watts from Lantmäteriet open geodata,
so a Swedish site can stop typing panel angles in by hand — and speaks plain
STAC, so it is not actually Sweden-only anymore.
Builds on #734 (the coverage registry this registers into). Based on
master.Shape
roofmodel/is a separate Python package alongsideoptimizer/, reached atarm's length: core spawns it, passes coordinates and the catalog credentials,
and reads back one versioned
roof_model.json.That boundary is the point. LiDAR segmentation drags in heavy dependencies,
runs for minutes, and is onboarding-time work rather than runtime work. In a
time-boxed subprocess it cannot stall the control tick, cannot leak into the
daemon, and can be absent entirely.
Method
Follows the SPAN paper
(Yavuzdoğan, Renewable Energy 2023): iterative RANSAC pulls one roof surface
at a time out of the cloud, then DBSCAN splits faces that satisfy the same
plane equation but sit in different places. The method only — no code from
SPAN's GPL QGIS plugin; dependencies are numpy, scikit-learn and requests, all
BSD.
You pick your building first, and the cloud is clipped to that footprint
before segmenting. That is a correctness fix, not a nicety: RANSAC fits
infinite planes, so a neighbour sharing pitch and ridge orientation lands on
the same plane at any distance (measured: byte-identical contamination from
3 m to 40 m of separation). Clipping restores the isolated result exactly.
Two implementation choices worth flagging
Plane fitting is total-least-squares, not
z = ax + by + c. An ordinaryregression of height on position minimises vertical error, which
systematically biases steep roofs. The smallest singular vector of the
mean-centred points minimises perpendicular distance instead, and stays
well-conditioned as the surface approaches vertical.
SWEREF 99 TM is implemented directly rather than via pyproj. It is exactly
one projection with fixed parameters, and shipping a full PROJ build for that
is disproportionate on a Pi. The Krüger series is ~120 lines and verified two
ways: against the projection's exact analytic properties (easting on the
central meridian is exactly 500000 at every latitude), and by
sub-millimetre round trips from Smygehuk to Treriksröset.
Found while demoing: buildings rendered in the Indian Ocean (fixed)
Driving the picker against a locally served GeoPackage put both candidate
buildings at lat ≈ 4°, lon ≈ 63°, "8 352 577 m away". The cause was an
axis-order split: rings converted from inline WGS84 geometry were stored
north-first, while rings decoded from a GeoPackage passed through east-first,
and everything downstream assumed one convention. 133 tests missed it because
every assertion was axis-blind — areas and centroids survive a consistent
swap, and the fixtures encoded the bug the same way the code did. Fixed in
a07fc0c0:ring_swerefalways holds (easting, northing), projected rings arenormalised per point by magnitude (Swedish eastings stay under a million
metres, northings start above six), and the regression test feeds the same
square in both orders and requires real Stockholm coordinates and a
sub-50 m distance back from each.
Credentials and other catalogs
Lantmäteriet provides no OAuth for its STAC download APIs, so the credential
is the operator's own Geotorget account username and password, sent as HTTP
Basic:
roofmodel.stac_username/stac_password. The legacygeotorget_username/geotorget_tokenkeys keep working as aliases andmigrate to the canonical keys on the next save through the API. The secret
masks and preserves like every other secret — never echoed by
GET /api/config, an empty password on save means "unchanged", and alegacy-stored token stays exactly as private (tested on both keys).
The client is otherwise catalog-agnostic:
stac_base_url,stac_buildings_collection,stac_lidar_collectionandstac_bbox_epsgpoint FTW at any STAC-conformant catalog (search is the spec's
POST {base}/search), and an open catalog needs no credentials at all — themodule goes anonymous, and only the Lantmäteriet default insists on an
account. A custom catalog also lifts the Sweden-only coordinate gate. The
Settings UI exposes this as a data-catalog picker: Lantmäteriet (default),
the live-verified open catalogs (IGN LiDAR HD for France, KAGIS for
Carinthia), or a custom endpoint with its own root and collection ids.
docs/roof-geometry.md
carries the table of catalogs verified live (France's LiDAR HD via the MTD
STAC API and Carinthia's KAGIS work today; swisstopo needs an unzip step;
USGS 3DEP needs a small SAS-token extension), with per-catalog caveats.
Verified live against the real service (2026-09-02)
Running the buildings flow end-to-end with a real Geotorget account surfaced
four differences between the assumed catalogue and the live one — each fixed
and pinned by tests:
api.lantmateriet.se/stac-vektor/v1, the point clouds onapi.lantmateriet.se/stac-hojd/v1. The pipeline now runs one client perroot for the defaults; a custom single-root catalog is unchanged.
byggnaderanddsm-skoglig-copc("LaserdataSkog", ytmodell som punktmoln in LAZ/COPC) — not the product-page slugs.
4326is nowthe default;
3006remains available for catalogs that want SWEREF.Stockholm's holds 93,716 features with no per-row envelopes. The reader
unwraps the zip and clips rows to the search window by parsed geometry
bounds — without that, the 5000-row limit truncated the table before it
reached the site — and a tile item's own geometry (the municipality
outline) is no longer mistaken for a building; a data asset wins, inline
geometry stays the fallback for catalogs whose items are buildings.
Catalogue metadata turned out to be anonymously readable; the credentials are
enforced where it matters, on the
dl1.lantmateriet.sedownloads (anonymousHEAD → 401). Live result: Find buildings here in central Stockholm returns
39 real footprints, nearest first, in ~4 s including the authenticated
13 MB municipality download.
The same session surfaced two core-side faults, fixed here too: credentials
typed into Settings never reached the module until a restart (the service
kept its boot-time config snapshot; it now follows config hot-reload like
every other service), and a real module error was reduced to
exit status 1whenever a third-party library warning shared stderr (the module's JSON
document is the final line, and is now parsed as such).
Design notes on the catalog boundary
(
stac_bbox_epsg: 4326), which the live Lantmäteriet service follows, so4326is the default and3006remains for a catalog that wants SWEREF.fitting works in that frame; going further would mean carrying a projection
stack the module deliberately avoids (see
sweref.py).byggnaderdelivers aZIP-wrapped GeoPackage per municipality (unzipped with the stdlib, decoded
with ~100 lines of
sqlite3+structagainst the OGC layouts, no GDAL),the point clouds are LAZ organised as COPC, whose
octree index turns the picked footprint into a kilobytes-sized range
request instead of a hundreds-of-MB tile. Assets are selected by declared
media type, not key names.
What it produces
GET /api/roofmodelreports availability and coverage,GET /api/roofmodel/buildingslists footprints nearest-first on the map, andPOST /api/roofmodel/deriveclips, segments and returns the proposal.Lantmäteriet also joins
/api/data-sourcesas a Sweden-only,credential-gated source.
A successful derive fills the arrays into the editor — real numbers in
the form, 3D preview updating — and stops at "Review them and press Save."
Config never changes behind the operator's back: derivation is a best guess
from a point cloud that may be years old, and a derived rated W is a
proposal, never a quote.
Testing
128 Python module tests. Every geometry case is a synthetic roof whose tilt,
azimuth and area are known exactly by construction, and the generator is the
inverse of the code under test rather than a copy of it. Covered: six
orientations, gable splitting, two buildings sharing a plane, sloped-vs-
footprint area, LiDAR noise, wall rejection, determinism, the GeoPackage and
WKB decoders against spec-assembled fixtures, both bbox CRSs, anonymous and
credentialed catalogs, the axis-order regression above, the municipality
zip + search-window clipping, and the tile-outline-vs-asset rule.
The Go suite exercises the subprocess contract end-to-end against a stub
module — argument passing, stdout parsing, stderr error surfacing, schema
rejection, the timeout, the Sweden gate and its custom-catalog lift, that
no empty
--username/--passwordever reaches a command line, confighot-reload of credentials and enablement, and module errors surviving
library warnings above them on stderr.
A limitation the tests pin rather than hide
Sweden is a long diagonal, so no lat/lon rectangle traces its border — any
box containing Sweden also contains parts of Norway; Oslo is the clearest
case. Rather than tighten the box until it excludes real border addresses,
the box stays generous and the STAC search is the authority: no tiles come
back, and the module reports "Sweden only". Tests cover both halves — Oslo
admitted by design, and Strömstad, Haparanda, Karesuando and Smygehuk never
excluded.
Covered live since 2026-09-02: the buildings half ran against the real
service with a real account (see Verified live above). The LiDAR derive
still stops at the point-cloud read on a box without
laspy; that path andthe documented quirk that
properties.datetimeis being backfilled through2026 (laser strip
datumas fallback) stay fake-session tested. To try itlive:
Part of the roadmap in discussion #717.
Seen running
Find buildings here → STAC search → GeoPackage decoded with sqlite3 + raw
WKB → two candidates at their real distances (local stand-in; the live run
against Lantmäteriet reproduces this with 39 buildings — that screenshot is
withheld because it shows the account username):
Read roof from LiDAR on the picked footprint: two planes found, the north
face dropped per the rule, the form pre-filled — "Filled in 1 array(s) from
2 roof plane(s). Laser data from 2019-06-30. Review them and press Save."
The capture date came through the
datumfallback:The section's help bubble, which used to truncate at the first quoted product
name and clip at the modal edge, now renders whole and flips upward when it
sits low (
29fb3e23):Runbook: how this was demoed (reproducible)
go build -o ftw-demo.exe ./cmd/ftwfrom the branch, plusgo run ./cmd/sim-ferroamp.provider: open_meteo, twopv_arrays,roofmodel.enabledwithcommand: python,module_dir: roofmodel.a local server spoke real STAC and served a real GeoPackage
(byte-assembled per OGC 12-128r19, EPSG:3006) and a synthetic LiDAR tile
through the module's documented decode seam.
screenshots are unretouched viewport captures.
Everything else — every line of core, the module, the UI — was the shipped
code from this branch.
Buildings render on the map, and the LiDAR derive runs live
Two later rounds of live testing hardened this further.
The found buildings now actually draw on the MapLibre map, selectable by
click (
cff76e93). Threestacked defects had kept the map empty while the list worked:
["get", "selected"]where MapLibrerequires a typed boolean — the layer is rejected through the error event,
without throwing;
oklch()colour tokens reached MapLibre's parser unconverted("Could not parse color"); the resolved colours are now baked to sRGB
through a 1×1 canvas;
A find now fits the view to the nearby candidates, and clicking a footprint
selects that building without dragging the site pin (and the saved
coordinates) along.
The full LiDAR derive now works against the real service
(
249411b7): picking aStockholm building and pressing Read roof from LiDAR returns
8 proposed arrays from 9 roof planes out of the 2021-03-23 national scan,
moving ~7 MB over HTTP range requests instead of the multi-GB tile.
New module dependency exercised: laspy[lazrs] (the geo extra)
The point-cloud read uses laspy
(docs) with the
laz-rs-based
lazrsbackend for LAZ/COPCdecoding — pure prebuilt wheels, no compiler. It was already declared as the
module's opt-in
geoextra inroofmodel/pyproject.toml(
pip install -e roofmodel[geo]); nothing else on the box needs it, and everynon-LiDAR path keeps working without it. This round is the first time the
extra ran against real Lantmäteriet data end-to-end. No pure-Go LAZ decoder
exists, so the Python module remains the right home for this step.
Found upstream: Lantmäteriet's COPC octree z-keys are broken
First live derive returned 47 points for a 4,760 m² window of a 217-million
point tile. The tile's octree nodes carry z voxel keys measured from some
other origin than the file's own COPC cube: a level-6 node whose key implies
a slab at −1698…−1542 m holds points at +18…+42 m, while its x/y keys are
exact. laspy prunes by those keys, silently discarding every dense level.
read_copc_windownow queries the octree cube's full vertical range, sopruning happens on x/y only — laspy's exact post-filter still bounds the
result. Pinned by
roofmodel/tests/test_pointcloud.py.The derive now searches where the picker searched
(
43924e16) — found live byan operator whose pick died with "building … was not found near this site".
Two causes, both fixed: the derive request now carries the picker's own
(possibly dragged, unsaved) map coordinates instead of silently using the
stored site, and the derive's re-find of the picked footprint reaches as far
as the picker's 150 m search did — it used to re-search with the 40 m LiDAR
radius, so anything found beyond that reach vanished on derive. The LiDAR
tile lookup also centres on the picked building rather than the pin, and the
module accepts
--footprint-json(a hand-drawn [lon, lat] ring that clipsexactly like a picked building — the UI for it rides the drawing PR above).
Re-verified on the reported case: a 522 m² barn picked via an unsaved pin
derives 2 arrays from 2 roof planes out of the 2019-04-04 scan.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Wepiu9kFxZkLTEZhoTJTx4