Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/spinloop/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ func main() {
os.Args = append(os.Args, "")
}
}
remote.SetCLIVersion(version)
if err := run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, "Error:", err)
os.Exit(1)
Expand Down
7 changes: 7 additions & 0 deletions cmd/spinloop/remote_bootstrap.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ var (
preflightFn = checkNodeAndPackageManager
)

// controlPlaneVersionEnv is read by the CDK app (remote/lib/config.ts) to stamp
// the control plane with the version of the CLI that deploys it.
const controlPlaneVersionEnv = "SPINLOOP_CONTROL_PLANE_VERSION"

// packageManagerEnv pins the Node package manager bootstrap drives the CDK
// project with, when the --package-manager flag is not given.
const packageManagerEnv = "SPINLOOP_REMOTE_PACKAGE_MANAGER"
Expand Down Expand Up @@ -278,6 +282,9 @@ func execStep(ctx context.Context, name string, argv []string, workDir string) e
fmt.Fprintf(os.Stderr, "\n$ %s\n", strings.Join(argv, " "))
cmd := exec.CommandContext(ctx, argv[0], argv[1:]...)
cmd.Dir = workDir
// The control plane reports this version in every response, so a CLI at a
// different version can warn about the mismatch.
cmd.Env = append(os.Environ(), controlPlaneVersionEnv+"="+version)
cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
if err := cmd.Run(); err != nil {
if errors.Is(err, exec.ErrNotFound) || errors.Is(err, os.ErrNotExist) {
Expand Down
13 changes: 13 additions & 0 deletions cmd/spinloop/remote_bootstrap_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -381,3 +381,16 @@ func TestBootstrap_NpmOverrideDrivesNpmCommands(t *testing.T) {
t.Errorf("commands = %v, want %v", got, want)
}
}

// Each bootstrap step runs with the CLI's version in its environment, which
// the CDK app stamps onto the control plane.
func TestExecStep_PassesTheControlPlaneVersion(t *testing.T) {
orig := version
t.Cleanup(func() { version = orig })
version = "1.30.0"

check := []string{"sh", "-c", `test "$` + controlPlaneVersionEnv + `" = "1.30.0"`}
if err := execStep(context.Background(), "check-env", check, t.TempDir()); err != nil {
t.Errorf("the step did not see %s=1.30.0: %v", controlPlaneVersionEnv, err)
}
}
7 changes: 7 additions & 0 deletions docs/commands/remote.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ path, logging which one it picked. To pin the choice, pass `--package-manager`
env var. A pinned manager that isn't installed fails the preflight rather than
falling back. `spinloop remote bake` honours the same flags.

Bootstrap stamps the control plane with the version of the `spinloop` that deploys
it, and every control plane response carries that version. When a later `remote`
command sees a version different from its own, it prints one warning to stderr
naming both, and carries on; re-run `spinloop remote bootstrap` to bring the
control plane up to date. No warning appears for a control plane deployed before
this was added, or when either side is a development build.

## Baking the AMIs

Each engine runs from a baked AMI (driver + engine, no model).
Expand Down
4 changes: 4 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ scaled; see [`spinloop serve`](commands/serve.md#parallelism).
an endpoint needs `spinloop remote bootstrap` to have run once per account;
a missing control plane says so. An older control plane that lacks a feature
says to re-run `bootstrap` to add it.
- **A warning says the control plane is at a different version.** The
`spinloop` you are running differs from the one that deployed the control
plane. Commands still run, but re-run `spinloop remote bootstrap` to update
it, or install the matching `spinloop`.
- **The AMI is not baked.** `spinloop remote bake` once per engine, and it
waits until the AMI is available.
- **A cold start takes about ten minutes.** `start` prints its progress on
Expand Down
2 changes: 2 additions & 0 deletions internal/remote/remote.go
Original file line number Diff line number Diff line change
Expand Up @@ -592,6 +592,7 @@ func send(
if err != nil {
return 0, nil, err
}
checkControlPlaneVersion(resp.Header)
return resp.StatusCode, respBody, nil
}

Expand Down Expand Up @@ -838,6 +839,7 @@ func callStats(ctx context.Context, cfg Config) (*StatsResponse, error) {
if err != nil {
return nil, err
}
checkControlPlaneVersion(resp.Header)
out := &StatsResponse{StatusCode: resp.StatusCode}
if err := json.Unmarshal(respBody, out); err != nil {
hint := ""
Expand Down
58 changes: 58 additions & 0 deletions internal/remote/versioncheck.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package remote

import (
"fmt"
"io"
"net/http"
"os"
"strings"
"sync"
)

// ControlPlaneVersionHeader is the response header every control plane Lambda
// sets to the spinloop version it was deployed with.
const ControlPlaneVersionHeader = "X-Spinloop-Control-Plane-Version"

var (
// cliVersion is the running binary's version, set once at startup by
// SetCLIVersion. While empty, no comparison is made.
cliVersion string
// versionWarnWriter receives the mismatch warning. A variable so tests can
// capture it.
versionWarnWriter io.Writer = os.Stderr
versionWarnOnce sync.Once
)

// SetCLIVersion records the running binary's version for the comparison
// against the control plane's.
func SetCLIVersion(v string) {
cliVersion = v
}

// versionsDiffer reports whether the CLI and control plane versions are both
// real release versions and are not the same. An empty or "dev" value on
// either side is not comparable, and a leading "v" is ignored.
func versionsDiffer(cli, controlPlane string) bool {
cli = strings.TrimPrefix(strings.TrimSpace(cli), "v")
controlPlane = strings.TrimPrefix(strings.TrimSpace(controlPlane), "v")
if cli == "" || cli == "dev" || controlPlane == "" || controlPlane == "dev" {
return false
}
return cli != controlPlane
}

// checkControlPlaneVersion writes a warning to stderr, at most once per
// process, when the response headers carry a control plane version that
// differs from the CLI's. A response without the header (a control plane that
// predates it) produces no warning.
func checkControlPlaneVersion(h http.Header) {
got := h.Get(ControlPlaneVersionHeader)
if !versionsDiffer(cliVersion, got) {
return
}
versionWarnOnce.Do(func() {
fmt.Fprintf(versionWarnWriter,
"Warning: the control plane is at version %s but this spinloop is %s. Run `spinloop remote bootstrap` to bring it up to date.\n",
strings.TrimPrefix(got, "v"), strings.TrimPrefix(cliVersion, "v"))
})
}
152 changes: 152 additions & 0 deletions internal/remote/versioncheck_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
package remote

import (
"bytes"
"context"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)

// captureVersionWarning points the warning at a buffer, sets the CLI version,
// and re-arms the once-per-process guard, restoring all three afterwards.
func captureVersionWarning(t *testing.T, cli string) *bytes.Buffer {
t.Helper()
var buf bytes.Buffer
origVersion, origWriter := cliVersion, versionWarnWriter
t.Cleanup(func() {
cliVersion, versionWarnWriter = origVersion, origWriter
versionWarnOnce = sync.Once{}
})
cliVersion, versionWarnWriter = cli, &buf
versionWarnOnce = sync.Once{}
return &buf
}

func headerWith(v string) http.Header {
h := http.Header{}
if v != "" {
h.Set(ControlPlaneVersionHeader, v)
}
return h
}

func TestVersionsDiffer(t *testing.T) {
cases := []struct {
name, cli, controlPlane string
want bool
}{
{"different", "1.30.0", "1.28.0", true},
{"same", "1.30.0", "1.30.0", false},
{"v prefix on the CLI", "v1.30.0", "1.30.0", false},
{"v prefix on the control plane", "1.30.0", "v1.30.0", false},
{"header absent", "1.30.0", "", false},
{"control plane dev", "1.30.0", "dev", false},
{"cli dev", "dev", "1.30.0", false},
{"cli empty", "", "1.30.0", false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
if got := versionsDiffer(c.cli, c.controlPlane); got != c.want {
t.Errorf("versionsDiffer(%q, %q) = %v, want %v", c.cli, c.controlPlane, got, c.want)
}
})
}
}

func TestCheckControlPlaneVersion_WarnsOnceOnMismatch(t *testing.T) {
buf := captureVersionWarning(t, "1.30.0")
checkControlPlaneVersion(headerWith("1.28.0"))
checkControlPlaneVersion(headerWith("1.28.0"))

out := buf.String()
if strings.Count(out, "Warning:") != 1 {
t.Errorf("expected exactly one warning, got %q", out)
}
for _, want := range []string{"1.28.0", "1.30.0", "spinloop remote bootstrap"} {
if !strings.Contains(out, want) {
t.Errorf("warning %q does not mention %q", out, want)
}
}
}

func TestCheckControlPlaneVersion_SilentWhenNotComparable(t *testing.T) {
for _, h := range []string{"1.30.0", "v1.30.0", "", "dev"} {
buf := captureVersionWarning(t, "1.30.0")
checkControlPlaneVersion(headerWith(h))
if buf.Len() != 0 {
t.Errorf("header %q: expected no warning, got %q", h, buf.String())
}
}
}

// Every control plane call passes through send or callStats; each reads the
// header off the response to the call itself.
func TestControlPlaneCalls_WarnFromTheirOwnResponse(t *testing.T) {
stubAWSEnv(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set(ControlPlaneVersionHeader, "1.28.0")
w.Write([]byte(`{"state":"stopped"}`))
}))
defer server.Close()
cfg := Config{StartURL: server.URL, StatsURL: server.URL, Region: "eu-west-1"}

t.Run("status", func(t *testing.T) {
buf := captureVersionWarning(t, "1.30.0")
if _, err := Status(context.Background(), cfg); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "1.28.0") {
t.Errorf("expected a warning naming the control plane version, got %q", buf.String())
}
})

t.Run("stats", func(t *testing.T) {
buf := captureVersionWarning(t, "1.30.0")
if _, err := Stats(context.Background(), cfg); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "1.28.0") {
t.Errorf("expected a warning naming the control plane version, got %q", buf.String())
}
})
}

func TestControlPlaneCalls_WarnOnErrorResponses(t *testing.T) {
stubAWSEnv(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set(ControlPlaneVersionHeader, "1.28.0")
w.WriteHeader(http.StatusBadGateway)
w.Write([]byte(`{"message":"boom"}`))
}))
defer server.Close()

buf := captureVersionWarning(t, "1.30.0")
if _, err := Status(context.Background(), Config{StartURL: server.URL, Region: "eu-west-1"}); err == nil {
t.Fatal("expected the 502 to be an error")
}
if !strings.Contains(buf.String(), "1.28.0") {
t.Errorf("expected a warning on an error response, got %q", buf.String())
}
}

func TestControlPlaneCalls_NoHeaderNoWarning(t *testing.T) {
stubAWSEnv(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"state":"stopped"}`))
}))
defer server.Close()

buf := captureVersionWarning(t, "1.30.0")
if _, err := Status(context.Background(), Config{StartURL: server.URL, Region: "eu-west-1"}); err != nil {
t.Fatal(err)
}
if buf.Len() != 0 {
t.Errorf("expected no warning without the header, got %q", buf.String())
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-05
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## Context

The control plane is a set of Lambdas deployed by CDK from `remote/`, using sources version-matched to the CLI (`ResolveRef`). The CLI calls them through `call`, `send` and `callStats` in `internal/remote/remote.go`. All Lambdas build responses through `jsonResponse` in `remote/lambda/shared/http.ts`.

## Goals / Non-Goals

**Goals:**
- A version mismatch warning on every remote command, with no extra API call.
- One place on each side that needs to change.

**Non-Goals:**
- Blocking commands on a mismatch.
- Comparing semantic versions (older vs newer); any difference warns.
- Changing response bodies.

## Decisions

**Response header, not a body field.** `jsonResponse` is used by every Lambda response, error ones included, so one edit covers all of them. Bodies differ in shape per Lambda (some are not objects), and the Go side would need a field on each reply struct. A header is read in `send`/`callStats`, which every call passes through.

**Version injected at deploy through CDK context.** `config.ts` already reads CDK context values; add `controlPlaneVersion` (context key `controlPlaneVersion`, default `dev`) and set it as `CONTROL_PLANE_VERSION` in `commonEnv` so all Lambdas get it. Bootstrap passes `-c controlPlaneVersion=<version>` to the `deploy` script via an environment variable read by `loadConfig` (`SPINLOOP_CONTROL_PLANE_VERSION`), which avoids changing the package scripts. `jsonResponse` reads `process.env.CONTROL_PLANE_VERSION` and falls back to `dev`.

**Warning once, in the transport layer.** `send` and `callStats` return the header value; a small function in `internal/remote` compares it against the CLI version and calls a warning writer, guarded by `sync.Once`. The CLI version is set at startup from `main.version`. Comparison trims a leading `v`. Absent header, empty, or `dev` on either side produces no warning.

**Stderr.** The warning is written to stderr in the CLI's chrome style per `cli-ux`, so JSON and other piped stdout stay clean.

## Risks / Trade-offs

- A control plane deployed from a dev build reports `dev` and never warns. Accepted; dev builds are not comparable.
- A once-per-process guard means a long-running `daemon` or `gateway` process warns once, not on every poll. Accepted.
- Existing control planes show no warning until re-bootstrapped. Accepted; there is no reliable way to tell them from old ones without a call.
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
## Why

The `spinloop remote` commands call a control plane (Lambdas in the user's AWS account) that was deployed from a particular spinloop release. When the CLI is upgraded and the control plane is not, the two can disagree about request and response shapes, and nothing tells the user. Issue 213 asks for a warning on any cloud command when the versions differ, without an extra API call.

## What Changes

- Every response from a control plane Lambda carries the control plane's version in a response header, `x-spinloop-control-plane-version`.
- `spinloop remote bootstrap` stamps the control plane with the version of the CLI that deploys it, through a CDK context value that becomes a Lambda environment variable.
- The CLI reads that header on every control plane call and, when it differs from the CLI's own version, prints one warning to stderr per process naming both versions and the fix (`spinloop remote bootstrap`).
- No warning when the header is absent (a control plane that predates this change) or when either side is a development build.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `remote-version-reporting`: adds the control plane version header and the CLI's mismatch warning.
- `endpoint-provisioning`: bootstrap records the deploying CLI's version in the control plane.

## Impact

- `remote/lib/llm-stack.ts`, `remote/lib/config.ts`, `remote/lambda/shared/http.ts` (CDK and Lambda response helper).
- `internal/remote/remote.go` (read the header in `send` and `callStats`), `cmd/spinloop/remote*.go` (print the warning, pass the version into bootstrap).
- Docs for `spinloop remote` and the `remote/` README.
- No new API calls and no change to response bodies.
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## ADDED Requirements

### Requirement: Bootstrap records the deploying CLI's version

`spinloop remote bootstrap` SHALL pass the running binary's version to the control plane deploy, so every control plane Lambda reports it in the `x-spinloop-control-plane-version` response header. A binary built without a version SHALL record `dev`.

#### Scenario: A release build bootstraps

- **WHEN** a CLI at version `1.30.0` runs `spinloop remote bootstrap`
- **THEN** the deployed Lambdas report `1.30.0` as the control plane version

#### Scenario: A development build bootstraps

- **WHEN** a CLI built without a version override runs `spinloop remote bootstrap`
- **THEN** the deployed Lambdas report `dev`
Loading
Loading