diff --git a/README.md b/README.md index 3eda270..55479d0 100644 --- a/README.md +++ b/README.md @@ -1,134 +1,73 @@ -# SystemGate +

+

SystemGate

+

A read-only window onto the machine. Not a remote control.
+Host vitals, containers, processes, packages, logs and verified backups, with no way to change any of them.

+

+ CI + Python 3.12 + MIT license + Part of Conker +

+ +SystemGate lets a dashboard or an agent see how the machine is doing without ever receiving write +access to it. By design there is no write, exec, restart, install or file-read +endpoint. Part of [Conker](https://github.com/Conker-AI/conker), and usable on its own. + +## Where it fits + +```mermaid +flowchart LR + Pi[Conker's Pi
or any dashboard] -->|admin key, server-side| SG[SystemGate] + SG -.->|read-only mounts| Host["/proc · Docker socket · disk · backups"] + classDef focus fill:#e36b2c,color:#fff,stroke:#b4521f + class SG focus +``` + +## Endpoints -Part of **[Conker](https://github.com/alexeybe1kin/conker)**, independently usable and deployable. [Project map](https://github.com/alexeybe1kin/conker/blob/feat/dashboard/docs/conker-project.md) · [Connected local setup](https://github.com/alexeybe1kin/conker/blob/feat/dashboard/docs/local-windows-startup.md). +| Route | What it reports | +|---|---| +| `GET /health` | Real probes of procfs, the Docker socket and the key store. No key. | +| `GET /vitals` | CPU, memory, disk and uptime, and whether they describe the **host** or the container. | +| `GET /containers` | Docker containers and their state. | +| `GET /processes` · `GET /services` | Processes, and listening services (metadata only). | +| `GET /packages` · `GET /logs/errors` | Installed packages and recent errors. | +| `GET /backups` | Snapshots whose manifest, hashes and archives verify. [Details](docs/backups.md) | +| `GET /runtime` | Processes, containers and ports in one bounded inventory. [Details](docs/runtime-inventory.md) | -Read-only local system telemetry API. +Every route except `/health` needs `X-SystemGate-Key`. Every response is bounded. -SystemGate exposes host, container, package, log, and backup status without adding any write, exec, or mutation endpoint. It is a standalone gate: run it by itself, or let a dashboard such as Conker consume it through a server-side proxy. +## Quick start -## Run +Requires Docker with Compose. ```bash -cp .env.example .env +cp .env.example .env # then set SYSTEMGATE_ADMIN_KEY to a long random value docker compose up -d --build +curl -H "X-SystemGate-Key: $KEY" http://127.0.0.1:8040/vitals ``` -API: `http://127.0.0.1:8040` - -Authenticated endpoints require `X-SystemGate-Key: `. +The API listens on `127.0.0.1:8040` only. Backups are read from `~/systemgate-backups` by default; +set `SYSTEMGATE_BACKUP_ROOT` to use another host directory. -By default the standalone compose file creates `systemgate_net` and mounts `~/systemgate-backups` read-only. Larger stacks can mount a different backup directory with `SYSTEMGATE_BACKUP_ROOT`. +## Security model, briefly -In standalone Compose, that variable selects the **host** directory. Inside the container, -Compose explicitly sets `SYSTEMGATE_BACKUP_ROOT: /backups`, matching the mount. Other Compose -stacks must set the same container environment entry alongside their `/backups:ro` mount. -Without an environment override, Python also defaults to `/backups`; direct host runs can set -`SYSTEMGATE_BACKUP_ROOT` to their local backup directory. +- **Read-only by construction.** Package and log collection use a fixed command set; no caller input + reaches a shell. +- **Read-only mounts.** The Docker socket, `/proc`, the host root and backups are mounted `:ro`. +- **Stated trade-off.** Host disk figures need the host root visible inside the container. Drop the + mount for container-only figures, and `/vitals` will say so. +- **Keys** are stored as PBKDF2 hashes. Use it from a server-side proxy so browsers never hold one. -### Backup integrity telemetry +Full model: [security](docs/security.md). -`GET /backups` verifies the `conker-snapshot-1` format from Companion's recovery script. It checks -the complete manifest, required stores, image identities, exact file inventory, sizes, SHA-256 -hashes, PostgreSQL dump header and archive structure/required SQLite headers. Archives are read, -never extracted. A snapshot changing during verification is rejected. +## Development -`results` and `latest` contain only verified snapshots, ordered by the manifest's timezone-aware -`created_at`, never filesystem modification time. Every candidate is checked before the newest -20 verified entries are selected, so incomplete directories cannot hide a valid snapshot. -`rejected` separately lists up to 20 incomplete, invalid, unsupported or unverifiable directories; -`verified_count` and `rejected_count` give the full counts. Missing/unreadable roots report -`unavailable`, an empty root reports `empty`, and rejected candidates make the response `degraded`. -An incomplete directory does not become a recovery point merely because it has a recent name. - -Verification reads the snapshot files in full on each request; large model archives can make -this endpoint slow. Poll it deliberately rather than at dashboard animation frequency. Responses -include `checked_at` and per-snapshot `verified_at`; there is no stale success cache. Hashes establish -consistency with the manifest, not authenticity against an attacker who can rewrite both. -`restore_status: not_tested` is explicit: this telemetry does not test decryption, restore a stack, -replay later deletions or reconcile actions. Manifest deletion-ledger and execution-journal states -are preserved instead of interpreting a complete snapshot as permission to resume. +```bash +pip install -r requirements.txt pytest httpx +python -m pytest tests -q +``` -## Endpoints +## License -- `GET /health` -- `GET /vitals` -- `GET /containers` -- `GET /processes` -- `GET /logs/errors` -- `GET /packages` -- `GET /backups` -- `GET /services` — listening processes, metadata only (no pid, cmdline, env or user) - -All non-health endpoints are read-only, bounded, and require the admin key. - -`GET /health` takes no key and runs real dependency probes — procfs, the Docker socket and the -admin key store. It reports `degraded` with the failing dependency named, never a hardcoded `ok`. -Probe detail is deliberately coarse because the endpoint is unauthenticated. - -## Security Layers - -SystemGate is intentionally narrow. It exists so a local dashboard or agent harness can show operational state without receiving host write access. - -- **Read-only API**: there are no write, exec, restart, package-install, file-edit, or Docker mutation endpoints. Package and log collection shell out to a fixed, non-injectable command set; no caller input reaches a shell. -- **Admin-key auth**: every endpoint except `/health` requires `X-SystemGate-Key`. -- **PBKDF2 key storage**: the first configured `SYSTEMGATE_ADMIN_KEY` is stored as a PBKDF2 hash under `data/admin-key.pbkdf2`; the raw key is not stored by SystemGate. -- **Server-side proxy friendly**: a dashboard can call SystemGate from its backend so the browser never receives the SystemGate key. -- **Loopback binding**: compose publishes the API on `127.0.0.1:8040` only. -- **Private Docker network**: the standalone compose file uses `systemgate_net` for local service-to-service traffic. -- **Read-only host mounts**: the Docker socket, `/proc`, the host root and the backup directory are all mounted `:ro`. -- **The host root mount is a deliberate trade-off, stated plainly**: reporting the host's disk usage requires the host filesystem to be visible, so `/` is mounted at `/host/root` read-only. This is what host telemetry exporters do, and it means anything able to read files from inside this container can read any file on the host. SystemGate exposes no file-read endpoint, and adding one would turn this mount into a serious hole. Drop the mount if you would rather have container-scoped disk figures; `/vitals` will say `scope: container` and remain truthful. -- **Says which machine it measured**: `/vitals` reports `source.scope` as `host` or `container`, with the procfs and disk paths actually in use - read back from psutil, not from the configuration, so it reports where the numbers came from rather than where they were asked to come from. Host figures require `SYSTEMGATE_PROCFS_PATH=/host/proc`, the host root mount and `uts: host`, all set by the bundled compose file. psutil honours no environment variable for procfs redirection, so SystemGate assigns `psutil.PROCFS_PATH` itself at import; setting an environment variable alone silently does nothing. -- **Bounded outputs**: process, log, package, backup, and container responses are capped so host telemetry cannot become an unbounded data leak. -- **No secret logging**: endpoints return system telemetry only; admin keys and upstream service secrets are not returned in responses. -- **Telemetry only**: SystemGate can observe Docker/container state, process lists, packages, logs, vitals, and backup timestamps, but it cannot change them. - -In short: SystemGate is a read-only window, not a remote control. - -## Runtime inventory - -`GET /runtime?limit=100` requires the existing `X-SystemGate-Key`; it returns -`Cache-Control: no-store`. It is an observation endpoint, not a control surface. -`systemgate/runtime.py` projects injected psutil/Docker collectors; no command, -restart, kill, container mutation, port edit, terminal, or file API is added. - -The response includes `mode: observed`, `sampledAt` (UTC collection start), -`ageSeconds`, `collectionSeconds`, and source scopes. Process scope identifies -configured procfs versus the collector namespace; network scope is reported as -the collector namespace, and Docker describes its configured daemon. These do not -assert that all three inventories describe the same host or namespace. - -`processes`, `containers`, and `ports` each contain `results`, `status` -(`ok`, `partial`, `unavailable`), `truncated`, and bounded static error codes. -Failures retain successfully collected rows and never expose raw exceptions. -Top-level status is partial if any section fails or truncates. Empty successful -inventory remains distinguishable from unavailable collection. - -Process IDs combine PID with exact floating-point creation time, checked again -against a fresh Process object to reject reused PIDs. Listener associations also -recheck that identity. Process RSS is bytes; CPU interval measurements, usernames, -and command lines are not collected. Their fields are null. Unmanaged processes -have no claimed start/restart definition or restart count. Container identity is -the full Docker ID; sparse list metadata avoids per-container inspect/stats calls. -Container-to-process association and restart count remain null. - -Port rows distinguish observed TCP listeners/UDP bindings from Docker's published -port mappings. IPv6 addresses are preserved. A Docker mapping never establishes -reachability: `listening` and `bound` are null. UDP `bound: true` does not invent a -TCP listening state. Docker exposed-only ports without host publication are not -presented as host mappings. No claim is made that stopped configured mappings are -fully discoverable from sparse Docker inventory. - -Result limits clamp to 1–200 rows **per section**, and process/listener scanning -stops after 2,000 entries. Ports share a row budget (listeners first, then Docker -bindings); truncation is explicit. Strings are bounded; Docker transport timeout -is five seconds. Collector internals may materialize full OS socket/container -lists before projection: these limits bound scanning/output, not OS collector -allocation or an overall wall-clock deadline. Results are fresh per request and -not persisted. Capabilities explicitly advertise all mutation, terminal, and file -operations as unavailable. The frontend's fixture-only schema still requires a -separate adapter; this endpoint does not claim P14 completion. - -`python -m pytest tests/test_runtime.py -q` uses injected fake collectors and a -temporary authentication directory to verify identities/PID reuse, auth, no -mutation routes, partial failures, bounded scanning/output, and truthful bindings. -No live host inventory, Docker daemon, or external service is needed by these tests. +[MIT](LICENSE) diff --git a/docs/backups.md b/docs/backups.md new file mode 100644 index 0000000..91839a1 --- /dev/null +++ b/docs/backups.md @@ -0,0 +1,24 @@ +# Backup integrity telemetry + +What `GET /backups` verifies, and what it deliberately does not claim. + +`GET /backups` verifies the `conker-snapshot-1` format from Companion's recovery script. It checks +the complete manifest, required stores, image identities, exact file inventory, sizes, SHA-256 +hashes, PostgreSQL dump header and archive structure/required SQLite headers. Archives are read, +never extracted. A snapshot changing during verification is rejected. + +`results` and `latest` contain only verified snapshots, ordered by the manifest's timezone-aware +`created_at`, never filesystem modification time. Every candidate is checked before the newest +20 verified entries are selected, so incomplete directories cannot hide a valid snapshot. +`rejected` separately lists up to 20 incomplete, invalid, unsupported or unverifiable directories; +`verified_count` and `rejected_count` give the full counts. Missing/unreadable roots report +`unavailable`, an empty root reports `empty`, and rejected candidates make the response `degraded`. +An incomplete directory does not become a recovery point merely because it has a recent name. + +Verification reads the snapshot files in full on each request; large model archives can make +this endpoint slow. Poll it deliberately rather than at dashboard animation frequency. Responses +include `checked_at` and per-snapshot `verified_at`; there is no stale success cache. Hashes establish +consistency with the manifest, not authenticity against an attacker who can rewrite both. +`restore_status: not_tested` is explicit: this telemetry does not test decryption, restore a stack, +replay later deletions or reconcile actions. Manifest deletion-ledger and execution-journal states +are preserved instead of interpreting a complete snapshot as permission to resume. diff --git a/docs/runtime-inventory.md b/docs/runtime-inventory.md new file mode 100644 index 0000000..9b4c5f0 --- /dev/null +++ b/docs/runtime-inventory.md @@ -0,0 +1,48 @@ +# Runtime inventory + +`GET /runtime?limit=100` requires the existing `X-SystemGate-Key`; it returns +`Cache-Control: no-store`. It is an observation endpoint, not a control surface. +`systemgate/runtime.py` projects injected psutil/Docker collectors; no command, +restart, kill, container mutation, port edit, terminal, or file API is added. + +The response includes `mode: observed`, `sampledAt` (UTC collection start), +`ageSeconds`, `collectionSeconds`, and source scopes. Process scope identifies +configured procfs versus the collector namespace; network scope is reported as +the collector namespace, and Docker describes its configured daemon. These do not +assert that all three inventories describe the same host or namespace. + +`processes`, `containers`, and `ports` each contain `results`, `status` +(`ok`, `partial`, `unavailable`), `truncated`, and bounded static error codes. +Failures retain successfully collected rows and never expose raw exceptions. +Top-level status is partial if any section fails or truncates. Empty successful +inventory remains distinguishable from unavailable collection. + +Process IDs combine PID with exact floating-point creation time, checked again +against a fresh Process object to reject reused PIDs. Listener associations also +recheck that identity. Process RSS is bytes; CPU interval measurements, usernames, +and command lines are not collected. Their fields are null. Unmanaged processes +have no claimed start/restart definition or restart count. Container identity is +the full Docker ID; sparse list metadata avoids per-container inspect/stats calls. +Container-to-process association and restart count remain null. + +Port rows distinguish observed TCP listeners/UDP bindings from Docker's published +port mappings. IPv6 addresses are preserved. A Docker mapping never establishes +reachability: `listening` and `bound` are null. UDP `bound: true` does not invent a +TCP listening state. Docker exposed-only ports without host publication are not +presented as host mappings. No claim is made that stopped configured mappings are +fully discoverable from sparse Docker inventory. + +Result limits clamp to 1–200 rows **per section**, and process/listener scanning +stops after 2,000 entries. Ports share a row budget (listeners first, then Docker +bindings); truncation is explicit. Strings are bounded; Docker transport timeout +is five seconds. Collector internals may materialize full OS socket/container +lists before projection: these limits bound scanning/output, not OS collector +allocation or an overall wall-clock deadline. Results are fresh per request and +not persisted. Capabilities explicitly advertise all mutation, terminal, and file +operations as unavailable. The frontend's fixture-only schema still requires a +separate adapter; this endpoint does not claim P14 completion. + +`python -m pytest tests/test_runtime.py -q` uses injected fake collectors and a +temporary authentication directory to verify identities/PID reuse, auth, no +mutation routes, partial failures, bounded scanning/output, and truthful bindings. +No live host inventory, Docker daemon, or external service is needed by these tests. diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..5ebc8b9 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,18 @@ +# SystemGate security model + +SystemGate is intentionally narrow. It exists so a local dashboard or agent harness can show operational state without receiving host write access. + +- **Read-only API**: there are no write, exec, restart, package-install, file-edit, or Docker mutation endpoints. Package and log collection shell out to a fixed, non-injectable command set; no caller input reaches a shell. +- **Admin-key auth**: every endpoint except `/health` requires `X-SystemGate-Key`. +- **PBKDF2 key storage**: the first configured `SYSTEMGATE_ADMIN_KEY` is stored as a PBKDF2 hash under `data/admin-key.pbkdf2`; the raw key is not stored by SystemGate. +- **Server-side proxy friendly**: a dashboard can call SystemGate from its backend so the browser never receives the SystemGate key. +- **Loopback binding**: compose publishes the API on `127.0.0.1:8040` only. +- **Private Docker network**: the standalone compose file uses `systemgate_net` for local service-to-service traffic. +- **Read-only host mounts**: the Docker socket, `/proc`, the host root and the backup directory are all mounted `:ro`. +- **The host root mount is a deliberate trade-off, stated plainly**: reporting the host's disk usage requires the host filesystem to be visible, so `/` is mounted at `/host/root` read-only. This is what host telemetry exporters do, and it means anything able to read files from inside this container can read any file on the host. SystemGate exposes no file-read endpoint, and adding one would turn this mount into a serious hole. Drop the mount if you would rather have container-scoped disk figures; `/vitals` will say `scope: container` and remain truthful. +- **Says which machine it measured**: `/vitals` reports `source.scope` as `host` or `container`, with the procfs and disk paths actually in use - read back from psutil, not from the configuration, so it reports where the numbers came from rather than where they were asked to come from. Host figures require `SYSTEMGATE_PROCFS_PATH=/host/proc`, the host root mount and `uts: host`, all set by the bundled compose file. psutil honours no environment variable for procfs redirection, so SystemGate assigns `psutil.PROCFS_PATH` itself at import; setting an environment variable alone silently does nothing. +- **Bounded outputs**: process, log, package, backup, and container responses are capped so host telemetry cannot become an unbounded data leak. +- **No secret logging**: endpoints return system telemetry only; admin keys and upstream service secrets are not returned in responses. +- **Telemetry only**: SystemGate can observe Docker/container state, process lists, packages, logs, vitals, and backup timestamps, but it cannot change them. + +In short: SystemGate is a read-only window, not a remote control.