Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ to use it, not a requirement.
> break the previous one. Pin an exact version (`simantic==0.1.0`) if you
> depend on it. Not recommended for production pipelines yet.

> **A Simantic account is required.** Installing the package gets you the
> Python code, but the simulators it drives are fetched from our backend and
> every request is authenticated. Without `simantic auth`, nothing runs.

```bash
pip install simantic
simantic auth # required: authenticates against your account
```

That is the whole setup for Python. The first `Sim(...)` fetches the
simulation engine (Simantic.Core plus a private .NET runtime — nothing else
to install) into `~/.simantic/engine/<version>/`, verified against the
release manifest, through the same authenticated gate the CLIs use.
`simantic install` fetches it up front, along with the `sim` and
`analog-cli` binaries if you also want the command-line tools.
to install) into `~/.simantic/engine/<version>/`, checksum-verified against
the public release manifest. `simantic install` fetches it up front, along
with the `sim` and `analog-cli` binaries if you also want the command-line
tools.

A Simantic account (`simantic auth`) is needed for one thing: resolving MCU
models by name (`mcu="STM32F401RE"`), which are fetched from your account
and cached in `~/.sim_cache`. A platform file you supply (`repl=`) needs no
account at all.

`simantic auth` opens a browser tab to sign in — like `gh auth login` — and
stores the resulting token in `~/.sim_id`, the same file the CLIs use, so one
Expand All @@ -36,9 +36,8 @@ on the dashboard's `/account/api` page. `--no-browser` falls back to an
interactive prompt for a pasted token.

Every download — engine or binary — is verified against the checksum in the
release manifest and fails closed: with no stored credentials nothing is
fetched and the error says to authenticate. The package on PyPI contains
only Python; the simulators are never in the wheel.
release manifest. The package on PyPI contains only Python; the simulators
are never in the wheel.

Already have the binaries? Point `$SIMANTIC_ANALOG_CLI` and `$SIMANTIC_SIM`
at them, or put them on PATH — both take precedence over a managed install.
Expand Down
2 changes: 1 addition & 1 deletion docs/session-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ the `sim` CLI is built on. There is no subprocess and no protocol: method
calls are method calls, records are objects. The engine is located in this
order: `$SIMANTIC_ENGINE_DIR`; a development `sim` publish directory
(`$SIMANTIC_SIM`); the managed install under `~/.simantic/engine/<version>/`
— and if none exists and credentials are stored, it is fetched on the spot.
— and if none exists it is fetched from the public release on the spot.
The managed engine bundles its own .NET runtime, so a machine needs only
Python.

Expand Down
15 changes: 4 additions & 11 deletions src/simantic/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
from functools import cache
from pathlib import Path

from . import auth, install
from . import install
from ._locate import BinaryNotFound
from .mcu import sim_binary

Expand All @@ -37,9 +37,9 @@ def engine_dir(explicit: str | os.PathLike[str] | None = None, *, fetch: bool =

Order: an explicit path, $SIMANTIC_ENGINE_DIR, a development `sim` whose
publish directory is beside it ($SIMANTIC_SIM), then the managed install
under ~/.simantic/engine. When nothing is there and credentials are
stored, the engine is fetched — so the first `Sim(...)` after
`simantic auth` just works. Without credentials it says what to do.
under ~/.simantic/engine. When nothing is there, the engine is fetched
from the public release — so the first `Sim(...)` after `pip install`
just works.
"""
candidates = []
if explicit is not None:
Expand All @@ -57,13 +57,6 @@ def engine_dir(explicit: str | os.PathLike[str] | None = None, *, fetch: bool =
if managed is not None:
return managed
if fetch:
try:
auth.load()
except auth.NotAuthenticated:
raise EngineNotFound(
"no simulation engine installed and no credentials stored: run `simantic auth` "
"(then the engine is fetched on first use, or run `simantic install engine`)."
) from None
try:
return install.install_engine()
except install.InstallError as exc:
Expand Down
179 changes: 28 additions & 151 deletions src/simantic/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
written into site-packages: an installed package may be read-only, and a
binary there would vanish on the next upgrade.

Fetching requires an account: every request carries the stored token, and
an unauthenticated install stops before it reaches the network.
Releases are public objects, keyed by version, so fetching needs no account;
checksums from the manifest are what make a download trustworthy.
"""

from __future__ import annotations
Expand All @@ -36,50 +36,19 @@
from dataclasses import dataclass
from pathlib import Path

from . import auth

RELEASES_URL = "https://drjdhqfvrttolueolzif.supabase.co/storage/v1/object/public/releases"


def releases_url() -> str:
"""Where manifests are served from. $SIMANTIC_RELEASES_URL overrides.

Overridable so a release can be rehearsed against a staging host before
it is published, and so the base can later move behind an endpoint that
checks the token this client already sends.
Overridable so a release can be rehearsed against a staging host (or a
plain directory served over HTTP) before it is published.
"""
return os.environ.get("SIMANTIC_RELEASES_URL", RELEASES_URL).rstrip("/")


#: Trades the token for a short-lived signed URL into a private bucket.
#:
#: The token check has to happen on the server. This installer is readable —
#: it names its own download URL — so a check performed here is a check any
#: reader can skip by fetching that URL directly. The engine is about a
#: megabyte; re-hosting it is a `curl` and an upload. Only an object that
#: cannot be fetched without a signature makes the check mean anything.
GATE_URL = "https://drjdhqfvrttolueolzif.supabase.co/functions/v1/get-release"


def gate_url() -> str:
"""The signing endpoint, or "" to fetch straight from a public bucket.

Set $SIMANTIC_GATE_URL="" together with $SIMANTIC_RELEASES_URL to install
from a plain directory of files — used by the tests and by anyone serving
their own mirror. Unset, the gated path is what runs.
"""
configured = os.environ.get("SIMANTIC_GATE_URL")
return (GATE_URL if configured is None else configured).rstrip("/")

#: Binary name -> release product prefix. A product that has published no
#: manifest yet fails with a clear message rather than a stray 404.
PRODUCTS = {
"sim": "cli",
"analog-cli": "analog",
"pyrite": "pyrite",
"pyrite-mcp": "pyrite",
}

#: The manifest to read. $SIMANTIC_CHANNEL selects a pre-release channel.
DEFAULT_CHANNEL = "latest"

Expand All @@ -97,12 +66,6 @@ class Artifact:
version: str
url: str
sha256: str | None
#: Set when the manifest names an object in a gated bucket rather than a
#: public URL. Signed at download time, because a signature minted when
#: the manifest was read may have expired by the time the bytes are
#: wanted.
path: str | None = None
channel: str | None = None


def simantic_home() -> Path:
Expand Down Expand Up @@ -141,76 +104,14 @@ def current_rid() -> str:
return f"{system}-{arch}"


def _headers(url: str) -> dict[str, str]:
"""Authorization for a release request.

Raises NotAuthenticated rather than falling back to an anonymous fetch:
an install must fail closed, and failing here costs nothing but a clear
message before any network round trip.

The token is only attached to the release host itself. Artifact URLs come
out of a manifest, which is data rather than code — a manifest naming
another host would otherwise have this client hand that host the user's
credentials. Off-host downloads still happen; they happen anonymously.
"""
credentials = auth.load() # fail closed before any request, wherever it goes
target, home = urllib.parse.urlparse(url), urllib.parse.urlparse(releases_url())
# Same host, and never in the clear: a bearer token on http is readable by
# anything on the path, so a staging or local host gets an anonymous fetch
# rather than the user's credentials.
if target.netloc != home.netloc or target.scheme != "https":
return {}
return {"Authorization": f"Bearer {credentials.api_key}"}


def signed_url(path: str, *, channel: str | None = None, timeout: float = 30) -> str:
"""Ask the gate to sign `path`, proving the token before anything is served.

A 401 here is the gate doing its job, so it is reported as such rather
than as a download failure.
"""
credentials = auth.load() # fail closed before the request
endpoint = (
f"{gate_url()}?path={urllib.parse.quote(path)}"
f"&channel={urllib.parse.quote(channel or default_channel())}"
)
request = urllib.request.Request(
endpoint, headers={"Authorization": f"Bearer {credentials.api_key}"}
)
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
body = json.loads(response.read())
except urllib.error.HTTPError as exc:
if exc.code in (401, 403):
raise auth.NotAuthenticated(
f"the backend rejected your credentials (HTTP {exc.code}) — "
"run `smtc auth`"
) from None
if exc.code == 404:
raise InstallError(f"no release artifact at {path!r}") from None
raise InstallError(f"release gate failed: HTTP {exc.code}") from None
except urllib.error.URLError as exc:
raise InstallError(f"cannot reach the release gate: {exc.reason}") from None
except json.JSONDecodeError as exc:
raise InstallError(f"release gate returned invalid JSON: {exc}") from None

url = body.get("url")
if not isinstance(url, str) or not url:
raise InstallError("release gate returned no URL")
return url


def _object_url(path: str, *, channel: str | None = None) -> tuple[str, dict[str, str]]:
"""Where to fetch a release object from, and what to send with it.

A signed URL carries its own authorisation in the query string, and the
storage endpoint expects a JWT in an Authorization header — sending the
PAT alongside it would be rejected. So a signed fetch sends no headers.
"""
if gate_url():
return signed_url(path, channel=channel), {}
url = f"{releases_url()}/{path}"
return url, _headers(url)
#: Binary name -> release product prefix. A product that has published no
#: manifest yet fails with a clear message rather than a stray 404.
PRODUCTS = {
"sim": "cli",
"analog-cli": "analog",
"pyrite": "pyrite",
"pyrite-mcp": "pyrite",
}


def fetch_manifest(
Expand All @@ -222,17 +123,11 @@ def fetch_manifest(
f"unknown binary {binary!r}; expected one of {sorted(PRODUCTS)}"
)
channel = channel or default_channel()
url, headers = _object_url(f"{product}/{channel}.json", channel=channel)
request = urllib.request.Request(url, headers=headers)
request = urllib.request.Request(f"{releases_url()}/{product}/{channel}.json")
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
return json.loads(response.read())
except urllib.error.HTTPError as exc:
if exc.code in (401, 403):
raise auth.NotAuthenticated(
f"the backend rejected your credentials for {binary!r} "
"(HTTP {}) — run `simantic auth`".format(exc.code)
) from None
raise InstallError(
f"no published releases for {binary!r} (HTTP {exc.code} from {url}). "
"Install the binary yourself and point $SIMANTIC_* at it."
Expand All @@ -255,38 +150,21 @@ def resolve(

rid = rid or current_rid()
entry = artifacts.get(rid)
# A gated manifest names `path` (an object in a private bucket); a public
# one names `url`. Either is enough to locate the build.
if not entry or not (entry.get("url") or entry.get("path")):
if not entry or not entry.get("url"):
available = ", ".join(sorted(artifacts)) or "none"
raise InstallError(
f"no {rid} build in {binary} release {version} (available: {available})"
)
return Artifact(
version=version,
url=entry.get("url", ""),
sha256=entry.get("sha256"),
path=entry.get("path"),
channel=channel or default_channel(),
)
return Artifact(version=version, url=entry["url"], sha256=entry.get("sha256"))


def download(artifact: Artifact, *, timeout: float = 300) -> bytes:
"""Fetch the artifact and verify its checksum before it is trusted."""
if artifact.path:
url, headers = _object_url(artifact.path, channel=artifact.channel)
else:
url, headers = artifact.url, _headers(artifact.url)
request = urllib.request.Request(url, headers=headers)
request = urllib.request.Request(artifact.url)
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
payload = response.read()
except urllib.error.HTTPError as exc:
if exc.code in (401, 403):
raise auth.NotAuthenticated(
f"the backend rejected your credentials (HTTP {exc.code}) — "
"run `simantic auth`"
) from None
raise InstallError(f"download failed: HTTP {exc.code}") from None
except urllib.error.URLError as exc:
raise InstallError(f"download failed: {exc.reason}") from None
Expand Down Expand Up @@ -394,37 +272,36 @@ def _version_key(name: str) -> tuple:


def install_engine(*, force: bool = False, channel: str | None = None) -> Path:
"""Download the engine archive for this machine into engine_root()/<version>.
"""Download the engine for this machine into engine_root()/<version>.

The archive is the `sim` publish directory (Simantic.Core.dll and friends)
plus a `dotnet/` runtime, published under the manifest key
`engine-<rid>` of the `sim` release, so it needs the same credentials and
goes through the same gate as the binaries.
One zip from the `sim` release manifest, key `engine-<rid>`: the
Simantic.Core publish directory plus a private .NET runtime under
`dotnet/`, laid out exactly as published.
"""
artifact = resolve("sim", rid=f"{ENGINE_KEY}-{current_rid()}", channel=channel)
target = engine_root() / artifact.version
if (target / "Simantic.Core.dll").exists() and not force:
return target

payload = download(artifact)
if not payload.startswith(b"\x1f\x8b"):
raise InstallError("engine artifact is not a gzipped tar archive")
if not payload.startswith(b"PK\x03\x04"):
raise InstallError("engine artifact is not a zip archive")
incoming = target.with_name(f".{artifact.version}.incoming")
if incoming.exists():
shutil.rmtree(incoming)
incoming.mkdir(parents=True)
try:
with tarfile.open(fileobj=io.BytesIO(payload), mode="r:gz") as archive:
for member in archive.getmembers():
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
for name in archive.namelist():
# Refuse anything that would land outside the target.
dest = (incoming / member.name).resolve()
dest = (incoming / name).resolve()
if not str(dest).startswith(str(incoming.resolve())):
raise InstallError(f"engine archive has an unsafe path: {member.name}")
archive.extractall(incoming, filter="data")
raise InstallError(f"engine archive has an unsafe path: {name}")
archive.extractall(incoming)
if target.exists():
shutil.rmtree(target)
os.replace(incoming, target)
except (OSError, tarfile.TarError) as exc:
except (OSError, zipfile.BadZipFile) as exc:
shutil.rmtree(incoming, ignore_errors=True)
raise InstallError(f"cannot unpack the engine into {target}: {exc}") from None
return target
Expand Down
Loading
Loading