From 59b1fb8a8754c5b33122ff8193ce473c35b24886 Mon Sep 17 00:00:00 2001 From: alexeybe1kin <210597588+alexeybe1kin@users.noreply.github.com> Date: Fri, 25 Sep 2026 21:08:26 +0300 Subject: [PATCH] Rewrite the README as a front page and move deep dives into docs Links pointed at the old personal repository, and the backup and runtime inventory contracts buried the endpoint list. The front page now shows what SystemGate reports, a quick start and the security model; the backup, runtime inventory and full security sections moved verbatim into docs/. Co-Authored-By: Claude Opus 5.5 --- README.md | 175 +++++++++++++------------------------- docs/backups.md | 24 ++++++ docs/runtime-inventory.md | 48 +++++++++++ docs/security.md | 18 ++++ 4 files changed, 147 insertions(+), 118 deletions(-) create mode 100644 docs/backups.md create mode 100644 docs/runtime-inventory.md create mode 100644 docs/security.md diff --git a/README.md b/README.md index 3eda270..55479d0 100644 --- a/README.md +++ b/README.md @@ -1,134 +1,73 @@ -# SystemGate +

+

SystemGate

+

A read-only window onto the machine. Not a remote control.
+Host vitals, containers, processes, packages, logs and verified backups, with no way to change any of them.

+

+ CI + Python 3.12 + MIT license + Part of Conker +

+ +SystemGate lets a dashboard or an agent see how the machine is doing without ever receiving write +access to it. By design there is no write, exec, restart, install or file-read +endpoint. Part of [Conker](https://github.com/Conker-AI/conker), and usable on its own. + +## Where it fits + +```mermaid +flowchart LR + Pi[Conker's Pi
or any dashboard] -->|admin key, server-side| SG[SystemGate] + SG -.->|read-only mounts| Host["/proc · Docker socket · disk · backups"] + classDef focus fill:#e36b2c,color:#fff,stroke:#b4521f + class SG focus +``` + +## Endpoints -Part of **[Conker](https://github.com/alexeybe1kin/conker)**, independently usable and deployable. [Project map](https://github.com/alexeybe1kin/conker/blob/feat/dashboard/docs/conker-project.md) · [Connected local setup](https://github.com/alexeybe1kin/conker/blob/feat/dashboard/docs/local-windows-startup.md). +| Route | What it reports | +|---|---| +| `GET /health` | Real probes of procfs, the Docker socket and the key store. No key. | +| `GET /vitals` | CPU, memory, disk and uptime, and whether they describe the **host** or the container. | +| `GET /containers` | Docker containers and their state. | +| `GET /processes` · `GET /services` | Processes, and listening services (metadata only). | +| `GET /packages` · `GET /logs/errors` | Installed packages and recent errors. | +| `GET /backups` | Snapshots whose manifest, hashes and archives verify. [Details](docs/backups.md) | +| `GET /runtime` | Processes, containers and ports in one bounded inventory. [Details](docs/runtime-inventory.md) | -Read-only local system telemetry API. +Every route except `/health` needs `X-SystemGate-Key`. Every response is bounded. -SystemGate exposes host, container, package, log, and backup status without adding any write, exec, or mutation endpoint. It is a standalone gate: run it by itself, or let a dashboard such as Conker consume it through a server-side proxy. +## Quick start -## Run +Requires Docker with Compose. ```bash -cp .env.example .env +cp .env.example .env # then set SYSTEMGATE_ADMIN_KEY to a long random value docker compose up -d --build +curl -H "X-SystemGate-Key: $KEY" http://127.0.0.1:8040/vitals ``` -API: `http://127.0.0.1:8040` - -Authenticated endpoints require `X-SystemGate-Key: `. +The API listens on `127.0.0.1:8040` only. Backups are read from `~/systemgate-backups` by default; +set `SYSTEMGATE_BACKUP_ROOT` to use another host directory. -By default the standalone compose file creates `systemgate_net` and mounts `~/systemgate-backups` read-only. Larger stacks can mount a different backup directory with `SYSTEMGATE_BACKUP_ROOT`. +## Security model, briefly -In standalone Compose, that variable selects the **host** directory. Inside the container, -Compose explicitly sets `SYSTEMGATE_BACKUP_ROOT: /backups`, matching the mount. Other Compose -stacks must set the same container environment entry alongside their `/backups:ro` mount. -Without an environment override, Python also defaults to `/backups`; direct host runs can set -`SYSTEMGATE_BACKUP_ROOT` to their local backup directory. +- **Read-only by construction.** Package and log collection use a fixed command set; no caller input + reaches a shell. +- **Read-only mounts.** The Docker socket, `/proc`, the host root and backups are mounted `:ro`. +- **Stated trade-off.** Host disk figures need the host root visible inside the container. Drop the + mount for container-only figures, and `/vitals` will say so. +- **Keys** are stored as PBKDF2 hashes. Use it from a server-side proxy so browsers never hold one. -### Backup integrity telemetry +Full model: [security](docs/security.md). -`GET /backups` verifies the `conker-snapshot-1` format from Companion's recovery script. It checks -the complete manifest, required stores, image identities, exact file inventory, sizes, SHA-256 -hashes, PostgreSQL dump header and archive structure/required SQLite headers. Archives are read, -never extracted. A snapshot changing during verification is rejected. +## Development -`results` and `latest` contain only verified snapshots, ordered by the manifest's timezone-aware -`created_at`, never filesystem modification time. Every candidate is checked before the newest -20 verified entries are selected, so incomplete directories cannot hide a valid snapshot. -`rejected` separately lists up to 20 incomplete, invalid, unsupported or unverifiable directories; -`verified_count` and `rejected_count` give the full counts. Missing/unreadable roots report -`unavailable`, an empty root reports `empty`, and rejected candidates make the response `degraded`. -An incomplete directory does not become a recovery point merely because it has a recent name. - -Verification reads the snapshot files in full on each request; large model archives can make -this endpoint slow. Poll it deliberately rather than at dashboard animation frequency. Responses -include `checked_at` and per-snapshot `verified_at`; there is no stale success cache. Hashes establish -consistency with the manifest, not authenticity against an attacker who can rewrite both. -`restore_status: not_tested` is explicit: this telemetry does not test decryption, restore a stack, -replay later deletions or reconcile actions. Manifest deletion-ledger and execution-journal states -are preserved instead of interpreting a complete snapshot as permission to resume. +```bash +pip install -r requirements.txt pytest httpx +python -m pytest tests -q +``` -## Endpoints +## License -- `GET /health` -- `GET /vitals` -- `GET /containers` -- `GET /processes` -- `GET /logs/errors` -- `GET /packages` -- `GET /backups` -- `GET /services` — listening processes, metadata only (no pid, cmdline, env or user) - -All non-health endpoints are read-only, bounded, and require the admin key. - -`GET /health` takes no key and runs real dependency probes — procfs, the Docker socket and the -admin key store. It reports `degraded` with the failing dependency named, never a hardcoded `ok`. -Probe detail is deliberately coarse because the endpoint is unauthenticated. - -## Security Layers - -SystemGate is intentionally narrow. It exists so a local dashboard or agent harness can show operational state without receiving host write access. - -- **Read-only API**: there are no write, exec, restart, package-install, file-edit, or Docker mutation endpoints. Package and log collection shell out to a fixed, non-injectable command set; no caller input reaches a shell. -- **Admin-key auth**: every endpoint except `/health` requires `X-SystemGate-Key`. -- **PBKDF2 key storage**: the first configured `SYSTEMGATE_ADMIN_KEY` is stored as a PBKDF2 hash under `data/admin-key.pbkdf2`; the raw key is not stored by SystemGate. -- **Server-side proxy friendly**: a dashboard can call SystemGate from its backend so the browser never receives the SystemGate key. -- **Loopback binding**: compose publishes the API on `127.0.0.1:8040` only. -- **Private Docker network**: the standalone compose file uses `systemgate_net` for local service-to-service traffic. -- **Read-only host mounts**: the Docker socket, `/proc`, the host root and the backup directory are all mounted `:ro`. -- **The host root mount is a deliberate trade-off, stated plainly**: reporting the host's disk usage requires the host filesystem to be visible, so `/` is mounted at `/host/root` read-only. This is what host telemetry exporters do, and it means anything able to read files from inside this container can read any file on the host. SystemGate exposes no file-read endpoint, and adding one would turn this mount into a serious hole. Drop the mount if you would rather have container-scoped disk figures; `/vitals` will say `scope: container` and remain truthful. -- **Says which machine it measured**: `/vitals` reports `source.scope` as `host` or `container`, with the procfs and disk paths actually in use - read back from psutil, not from the configuration, so it reports where the numbers came from rather than where they were asked to come from. Host figures require `SYSTEMGATE_PROCFS_PATH=/host/proc`, the host root mount and `uts: host`, all set by the bundled compose file. psutil honours no environment variable for procfs redirection, so SystemGate assigns `psutil.PROCFS_PATH` itself at import; setting an environment variable alone silently does nothing. -- **Bounded outputs**: process, log, package, backup, and container responses are capped so host telemetry cannot become an unbounded data leak. -- **No secret logging**: endpoints return system telemetry only; admin keys and upstream service secrets are not returned in responses. -- **Telemetry only**: SystemGate can observe Docker/container state, process lists, packages, logs, vitals, and backup timestamps, but it cannot change them. - -In short: SystemGate is a read-only window, not a remote control. - -## Runtime inventory - -`GET /runtime?limit=100` requires the existing `X-SystemGate-Key`; it returns -`Cache-Control: no-store`. It is an observation endpoint, not a control surface. -`systemgate/runtime.py` projects injected psutil/Docker collectors; no command, -restart, kill, container mutation, port edit, terminal, or file API is added. - -The response includes `mode: observed`, `sampledAt` (UTC collection start), -`ageSeconds`, `collectionSeconds`, and source scopes. Process scope identifies -configured procfs versus the collector namespace; network scope is reported as -the collector namespace, and Docker describes its configured daemon. These do not -assert that all three inventories describe the same host or namespace. - -`processes`, `containers`, and `ports` each contain `results`, `status` -(`ok`, `partial`, `unavailable`), `truncated`, and bounded static error codes. -Failures retain successfully collected rows and never expose raw exceptions. -Top-level status is partial if any section fails or truncates. Empty successful -inventory remains distinguishable from unavailable collection. - -Process IDs combine PID with exact floating-point creation time, checked again -against a fresh Process object to reject reused PIDs. Listener associations also -recheck that identity. Process RSS is bytes; CPU interval measurements, usernames, -and command lines are not collected. Their fields are null. Unmanaged processes -have no claimed start/restart definition or restart count. Container identity is -the full Docker ID; sparse list metadata avoids per-container inspect/stats calls. -Container-to-process association and restart count remain null. - -Port rows distinguish observed TCP listeners/UDP bindings from Docker's published -port mappings. IPv6 addresses are preserved. A Docker mapping never establishes -reachability: `listening` and `bound` are null. UDP `bound: true` does not invent a -TCP listening state. Docker exposed-only ports without host publication are not -presented as host mappings. No claim is made that stopped configured mappings are -fully discoverable from sparse Docker inventory. - -Result limits clamp to 1–200 rows **per section**, and process/listener scanning -stops after 2,000 entries. Ports share a row budget (listeners first, then Docker -bindings); truncation is explicit. Strings are bounded; Docker transport timeout -is five seconds. Collector internals may materialize full OS socket/container -lists before projection: these limits bound scanning/output, not OS collector -allocation or an overall wall-clock deadline. Results are fresh per request and -not persisted. Capabilities explicitly advertise all mutation, terminal, and file -operations as unavailable. The frontend's fixture-only schema still requires a -separate adapter; this endpoint does not claim P14 completion. - -`python -m pytest tests/test_runtime.py -q` uses injected fake collectors and a -temporary authentication directory to verify identities/PID reuse, auth, no -mutation routes, partial failures, bounded scanning/output, and truthful bindings. -No live host inventory, Docker daemon, or external service is needed by these tests. +[MIT](LICENSE) diff --git a/docs/backups.md b/docs/backups.md new file mode 100644 index 0000000..91839a1 --- /dev/null +++ b/docs/backups.md @@ -0,0 +1,24 @@ +# Backup integrity telemetry + +What `GET /backups` verifies, and what it deliberately does not claim. + +`GET /backups` verifies the `conker-snapshot-1` format from Companion's recovery script. It checks +the complete manifest, required stores, image identities, exact file inventory, sizes, SHA-256 +hashes, PostgreSQL dump header and archive structure/required SQLite headers. Archives are read, +never extracted. A snapshot changing during verification is rejected. + +`results` and `latest` contain only verified snapshots, ordered by the manifest's timezone-aware +`created_at`, never filesystem modification time. Every candidate is checked before the newest +20 verified entries are selected, so incomplete directories cannot hide a valid snapshot. +`rejected` separately lists up to 20 incomplete, invalid, unsupported or unverifiable directories; +`verified_count` and `rejected_count` give the full counts. Missing/unreadable roots report +`unavailable`, an empty root reports `empty`, and rejected candidates make the response `degraded`. +An incomplete directory does not become a recovery point merely because it has a recent name. + +Verification reads the snapshot files in full on each request; large model archives can make +this endpoint slow. Poll it deliberately rather than at dashboard animation frequency. Responses +include `checked_at` and per-snapshot `verified_at`; there is no stale success cache. Hashes establish +consistency with the manifest, not authenticity against an attacker who can rewrite both. +`restore_status: not_tested` is explicit: this telemetry does not test decryption, restore a stack, +replay later deletions or reconcile actions. Manifest deletion-ledger and execution-journal states +are preserved instead of interpreting a complete snapshot as permission to resume. diff --git a/docs/runtime-inventory.md b/docs/runtime-inventory.md new file mode 100644 index 0000000..9b4c5f0 --- /dev/null +++ b/docs/runtime-inventory.md @@ -0,0 +1,48 @@ +# Runtime inventory + +`GET /runtime?limit=100` requires the existing `X-SystemGate-Key`; it returns +`Cache-Control: no-store`. It is an observation endpoint, not a control surface. +`systemgate/runtime.py` projects injected psutil/Docker collectors; no command, +restart, kill, container mutation, port edit, terminal, or file API is added. + +The response includes `mode: observed`, `sampledAt` (UTC collection start), +`ageSeconds`, `collectionSeconds`, and source scopes. Process scope identifies +configured procfs versus the collector namespace; network scope is reported as +the collector namespace, and Docker describes its configured daemon. These do not +assert that all three inventories describe the same host or namespace. + +`processes`, `containers`, and `ports` each contain `results`, `status` +(`ok`, `partial`, `unavailable`), `truncated`, and bounded static error codes. +Failures retain successfully collected rows and never expose raw exceptions. +Top-level status is partial if any section fails or truncates. Empty successful +inventory remains distinguishable from unavailable collection. + +Process IDs combine PID with exact floating-point creation time, checked again +against a fresh Process object to reject reused PIDs. Listener associations also +recheck that identity. Process RSS is bytes; CPU interval measurements, usernames, +and command lines are not collected. Their fields are null. Unmanaged processes +have no claimed start/restart definition or restart count. Container identity is +the full Docker ID; sparse list metadata avoids per-container inspect/stats calls. +Container-to-process association and restart count remain null. + +Port rows distinguish observed TCP listeners/UDP bindings from Docker's published +port mappings. IPv6 addresses are preserved. A Docker mapping never establishes +reachability: `listening` and `bound` are null. UDP `bound: true` does not invent a +TCP listening state. Docker exposed-only ports without host publication are not +presented as host mappings. No claim is made that stopped configured mappings are +fully discoverable from sparse Docker inventory. + +Result limits clamp to 1–200 rows **per section**, and process/listener scanning +stops after 2,000 entries. Ports share a row budget (listeners first, then Docker +bindings); truncation is explicit. Strings are bounded; Docker transport timeout +is five seconds. Collector internals may materialize full OS socket/container +lists before projection: these limits bound scanning/output, not OS collector +allocation or an overall wall-clock deadline. Results are fresh per request and +not persisted. Capabilities explicitly advertise all mutation, terminal, and file +operations as unavailable. The frontend's fixture-only schema still requires a +separate adapter; this endpoint does not claim P14 completion. + +`python -m pytest tests/test_runtime.py -q` uses injected fake collectors and a +temporary authentication directory to verify identities/PID reuse, auth, no +mutation routes, partial failures, bounded scanning/output, and truthful bindings. +No live host inventory, Docker daemon, or external service is needed by these tests. diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..5ebc8b9 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,18 @@ +# SystemGate security model + +SystemGate is intentionally narrow. It exists so a local dashboard or agent harness can show operational state without receiving host write access. + +- **Read-only API**: there are no write, exec, restart, package-install, file-edit, or Docker mutation endpoints. Package and log collection shell out to a fixed, non-injectable command set; no caller input reaches a shell. +- **Admin-key auth**: every endpoint except `/health` requires `X-SystemGate-Key`. +- **PBKDF2 key storage**: the first configured `SYSTEMGATE_ADMIN_KEY` is stored as a PBKDF2 hash under `data/admin-key.pbkdf2`; the raw key is not stored by SystemGate. +- **Server-side proxy friendly**: a dashboard can call SystemGate from its backend so the browser never receives the SystemGate key. +- **Loopback binding**: compose publishes the API on `127.0.0.1:8040` only. +- **Private Docker network**: the standalone compose file uses `systemgate_net` for local service-to-service traffic. +- **Read-only host mounts**: the Docker socket, `/proc`, the host root and the backup directory are all mounted `:ro`. +- **The host root mount is a deliberate trade-off, stated plainly**: reporting the host's disk usage requires the host filesystem to be visible, so `/` is mounted at `/host/root` read-only. This is what host telemetry exporters do, and it means anything able to read files from inside this container can read any file on the host. SystemGate exposes no file-read endpoint, and adding one would turn this mount into a serious hole. Drop the mount if you would rather have container-scoped disk figures; `/vitals` will say `scope: container` and remain truthful. +- **Says which machine it measured**: `/vitals` reports `source.scope` as `host` or `container`, with the procfs and disk paths actually in use - read back from psutil, not from the configuration, so it reports where the numbers came from rather than where they were asked to come from. Host figures require `SYSTEMGATE_PROCFS_PATH=/host/proc`, the host root mount and `uts: host`, all set by the bundled compose file. psutil honours no environment variable for procfs redirection, so SystemGate assigns `psutil.PROCFS_PATH` itself at import; setting an environment variable alone silently does nothing. +- **Bounded outputs**: process, log, package, backup, and container responses are capped so host telemetry cannot become an unbounded data leak. +- **No secret logging**: endpoints return system telemetry only; admin keys and upstream service secrets are not returned in responses. +- **Telemetry only**: SystemGate can observe Docker/container state, process lists, packages, logs, vitals, and backup timestamps, but it cannot change them. + +In short: SystemGate is a read-only window, not a remote control.