Add Kubernetes manifests, validated on kind, and fix two bugs they exposed

[ShreeBohara]

StatefulSet + PVC + ConfigMap/Secret + Services for the API, plus a kind cluster config. Everything here was actually run — the pod reaches 1/1 Running, serves traffic on a NodePort, and its volume survives pod deletion.

What this is honestly for

Of the capabilities kubernetes.io lists, this workload can use two: storage orchestration and self-healing restarts.

It does not give horizontal scaling, and replicas: 1 is a correct description of the system rather than a TODO. DATABASE_URL is a SQLite file on a ReadWriteOnce volume, and chroma_store.py holds a process-local persistent client over the same directory — a second pod would be a second independent writer to both.

Writing the manifests is what surfaced that, and two real bugs.

Bug 1: the image ran as root

Confirmed by inspecting the built image (id -u → 0). Under docker-compose that produced root-owned files in your working tree via the ../data bind mount; in Kubernetes it made runAsNonRoot unsatisfiable.

Fixed in Dockerfile.api (uid 10001, owns /app, sets USER). fsGroup: 10001 is what makes the mounted PVC writable by it — without that the pod starts and then fails on the first SQLite write, which is a worse failure than not starting.

Bug 2: the API could not start without a provider key

The pod went into CrashLoopBackOff:

openai.OpenAIError: Missing credentials. Please pass an `api_key` ...
ERROR:    Application startup failed. Exiting.

The lifespan builds the vector store eagerly, and the OpenAI SDK raises at construction, not first use. Three consequences that only became obvious once deployed:

• the pod crash-loops instead of reporting itself unhealthy
• you cannot deploy first and supply credentials afterwards
• /api/health can never report llm_provider unreachable, because the process never boots far enough to serve it

Vector store init is now non-fatal, matching how Neo4j and the progress store already behave. Endpoints needing embeddings fail per-request with a clear error; the rest keep working:

{ "status": "degraded",
  "checks": { "database": "ok",
              "vector_store": "error: Missing credentials...",
              "llm_provider": "error: Missing credentials..." } }

Prerequisite: concurrent startup was unsafe

run_pending_migrations used check-then-act (_column_exists, then ALTER TABLE). Two replicas starting together both saw the column missing, both issued the ALTER, and the loser died on duplicate column name.

SQLite has no ADD COLUMN IF NOT EXISTS, so rather than serialize startup with a lock (unreliable on a shared volume; a DB mutex needs its own release path), the operation is now genuinely idempotent — a duplicate-column error means someone else applied it, which is success. Verified with 6 concurrent runs against one database, 0 failures.

One design note

The readiness probe deliberately does not use /api/health. That reports degraded whenever any dependency is unreachable, including a third-party LLM provider — so gating readiness on it would pull the pod out of service because OpenAI had a bad minute, and on a keyless cluster it would never become ready at all. It hits /api/platform/config, which touches only the local database.

Verified

Check	Result
kubectl kustomize	5 resources render
Rollout	codebaseqa-api-0 1/1 Running 0 restarts
NodePort from host	curl localhost:30080/api/platform/config returns JSON
/health, no key	degraded with a specific reason, process stays up
Migrations at startup	applied, incl. ix_code_dependencies_repo
PVC	Bound, 5Gi, RWO
Non-root write	uid 10001 writes /app/data via fsGroup
Persistence	marker file and codebaseqa.db survive kubectl delete pod
Concurrent startup	6 simultaneous migration runs, 0 failures
Suite	168 passed (incl. 9 live Neo4j), ruff clean

Also worth noting: this was the first ever build of docker/Dockerfile.api, which let me empirically confirm the earlier .dockerignore fix — no .env in the image, no .venv, no baked key, and a filesystem-wide grep for sk- patterns finds nothing. That was previously only an fnmatch simulation.

Not verified

The web frontend is not deployed by these manifests — API only. Nothing has run on a managed cluster; a cloud StorageClass and LoadBalancer would replace kind's standard class and the NodePort.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
codebaseqa-web	Ready	Preview	Aug 11, 2026 8:17pm