feat(llm): native Anthropic Messages API wire protocol "anthropic"

.github/workflows/ci.yml

@@ -75,7 +75,16 @@ jobs:
           # der es selbst gebaut wurde, und v2.11.4 bricht gegen ein Modul mit
           # `go 1.27.0` mit "the Go language version used to build
           # golangci-lint is lower than the targeted Go version" ab.
-          version: v2.13.2
+          #
+          # Bump auf v2.14.0 (2026-10-11, PR #47): der Runner löst GO_VERSION
+          # "1.27" seit dem 10.10. auf go1.27.2 auf, und v2.13.2 (gebaut mit
+          # einem älteren 1.27) kann dessen Export-Daten nicht lesen —
+          # "export data version 5 is greater than maximum supported version
+          # 4", typecheck-Fehler auf dem ersten `import "fmt"`, Nightly auf
+          # root rot seit 38019738113 (10.10.) bei sonst grünen Jobs. v2.14.0
+          # (24.09.) liest sie; lokal gegen go1.27.2 mit demselben
+          # .golangci.yml: 0 issues.
+          version: v2.14.0
           working-directory: go
           args: --timeout=5m
           # Kein Analyse-Cache (2026-08-26): der Cache-Key der Action trägt das

README.md

@@ -73,7 +73,7 @@ Running the server (Go daemon + PostgreSQL 18 + pgvector) is `docker compose up
 
 > **Upgrading from v4? Reach v4.38.0 first.** v5.0.0 removes the `CTX_{CHAT,CHAT_FALLBACK,EMBED,DREAM,DREAM_EMBED,RERANK}_*` connection variables: backends are configured in the pool (`ctx backends`) and nowhere else. **v4.38.0 is the last release that still reads those variables**, and one boot on it carries a `.env`-only configuration into the pool for you, then names every variable and leftover settings row you still have. Going straight from an older v4 to v5 is not fatal down to v4.19.0 — the env bridge is gone by then and you re-enter your topology by hand with `ctx backends seed` — but **below v4.19.0 it is a hard stop**: v5 folds migrations 001–113 into a single baseline and refuses a database that never applied 113. Migration path and rollback notes: [operations](docs/operations.md#migration-133-the-backend-tuple-rows-are-deleted--and-the-upgrade-hop-that-has-to-come-first).
 
-**Models:** the tested reference stack is qwen3.6:27b for chat/synthesis/dream/digest, Qwen3-Embedding-8B (1024-dim, CPU-sized) for embeddings, and a bge-reranker-v2-m3 cross-encoder sidecar for rerank — but ctx is engine-agnostic: any OpenAI-compatible or Ollama API works, models are per-backend configuration. CPU-only setups work; a GPU shortens chat latency. Details: [Models & hardware](docs/operations.md#models--hardware).
+**Models:** the tested reference stack is qwen3.6:27b for chat/synthesis/dream/digest, Qwen3-Embedding-8B (1024-dim, CPU-sized) for embeddings, and a bge-reranker-v2-m3 cross-encoder sidecar for rerank — but ctx is engine-agnostic: any OpenAI-compatible or Ollama API works, and the chat-class roles can run on the native Anthropic Messages API (`protocol: "anthropic"`); models are per-backend configuration. CPU-only setups work; a GPU shortens chat latency. Details: [Models & hardware](docs/operations.md#models--hardware).
 
 ## How LLMs use ctx
 

docker-compose.yml

@@ -362,8 +362,10 @@ services:
       # ── status.* (1) ──
       CTX_STATUS_CHANNEL_PROBE_INTERVAL: ${CTX_STATUS_CHANNEL_PROBE_INTERVAL:-}
 
-      # ── embed_backfill.* (4) ──
+      # ── embed_backfill.* (6) ──
       CTX_EMBED_BACKFILL_SYNC_CAP: ${CTX_EMBED_BACKFILL_SYNC_CAP:-}
+      CTX_EMBED_BACKFILL_SYNC_BUDGET: ${CTX_EMBED_BACKFILL_SYNC_BUDGET:-}
+      CTX_EMBED_BACKFILL_INTERVAL: ${CTX_EMBED_BACKFILL_INTERVAL:-}
       CTX_EMBED_BACKFILL_MAX_TOKENS: ${CTX_EMBED_BACKFILL_MAX_TOKENS:-}
       CTX_EMBED_BACKFILL_BACKOFF_BASE: ${CTX_EMBED_BACKFILL_BACKOFF_BASE:-}
       CTX_EMBED_BACKFILL_BACKOFF_CAP: ${CTX_EMBED_BACKFILL_BACKOFF_CAP:-}

docs/api.md

@@ -441,13 +441,15 @@ message — the verdict lives in the body, like every `backend-test` outcome.
 
 **Validation guards** (create AND update, `422` with field errors): credential-carrier headers in `extra_headers` (`Authorization`, `Cookie`, `*-key`, `*-token`, …) and credential-semantic `extra_body` fields are rejected — provider keys go through `api_key_ref` (the *name* of a sealed secret, resolved in-memory only); a `base_url` carrying userinfo (`user:pass@host`) is rejected for the same reason — the URL is echoed unredacted by every list response, log line and the append-only audit (rows written before this guard load fine and are rejected on their next write); `locality` is cross-validated against `base_url` (a publicly routable host must be `external`); embed roles on external backends are blocked without `metadata.embed_equivalence_verified=true` (foreign quantization corrupts the shared vector space). `metadata.score_domain` (rerank dialect slot) must be one of `auto` | `logit` | `probability` — `auto` (the default for every row that does not set it) keeps the wire-container coupling: `results` ⇒ raw logits, `data` ⇒ calibrated [0,1] probability; explicit values override the coupling for dialect-mixing backends. Changing the *effective* score domain of a rerank-capable backend (create with a non-`auto` value included — the delete+create bypass is closed like the trust confirm's) requires `confirm_score_domain_change:true`; the 400 carries the full chain of change (effective immediately on all future rankings; the access-log score trail is only comparable up to the switch; no corpus reprocessing — no curation artifact persists rerank-derived data) and the confirmed write echoes it as a warning. Raising `trust` requires `confirm_trust_elevation:true`. Every mutation reloads the pool snapshot synchronously — `backend-update {"enabled":false}` is an instant brake, no restart; psql edits converge via the 053 NOTIFY trigger.
 
-**OpenRouter (first external backend, G29).** `provider_class: "openrouter"` refines the openai wire: the request **always** carries `provider.zdr:true` + `provider.data_collection:"deny"`, independent of trust level — trust decides WHICH content may flow, the provider class decides whether the provider may store it. `extra_body.provider` entries merge but can only tighten (the force runs after the merge). The single escape is `metadata.allow_data_collection:true`, requiring `confirm_data_collection:true`. Responses feed telemetry: `usage.cost` → llmlog `cost_usd` (local NULL), the top-level `model` (that actually answered) overwrites the row's model, the response `id` lands in `metadata.provider_request_id`. A "no providers" rejection (zdr/deny filter leaves no provider) classifies as configuration-permanent (1h cooldown, no retry storm). `backend-test` additionally reports `credits_remaining`/`usage_usd` (`GET /v1/key`) and `zdr_endpoints` (`GET /v1/endpoints/zdr`). **`base_url` is the API root WITHOUT the version segment** — the wire paths append `/v1/...` themselves; a `base_url` ending in `/v1` double-segments to a 404, and since v4.25.4 create/update reject such a root with a 422 (all openai- and rerank-protocol backends, not just openrouter).
+**OpenRouter (first external backend, G29).** `provider_class: "openrouter"` refines the openai wire: the request **always** carries `provider.zdr:true` + `provider.data_collection:"deny"`, independent of trust level — trust decides WHICH content may flow, the provider class decides whether the provider may store it. `extra_body.provider` entries merge but can only tighten (the force runs after the merge). The single escape is `metadata.allow_data_collection:true`, requiring `confirm_data_collection:true`. Responses feed telemetry: `usage.cost` → llmlog `cost_usd` (local NULL), the top-level `model` (that actually answered) overwrites the row's model, the response `id` lands in `metadata.provider_request_id`. A "no providers" rejection (zdr/deny filter leaves no provider) classifies as configuration-permanent (1h cooldown, no retry storm). `backend-test` additionally reports `credits_remaining`/`usage_usd` (`GET /v1/key`) and `zdr_endpoints` (`GET /v1/endpoints/zdr`). **`base_url` is the API root WITHOUT the version segment** — the wire paths append `/v1/...` themselves; a `base_url` ending in `/v1` double-segments to a 404, and since v4.25.4 create/update reject such a root with a 422 (all openai-, anthropic- and rerank-protocol backends, not just openrouter).
+
+**Anthropic (native Messages API, Migration 155).** `protocol: "anthropic"` speaks `POST {base_url}/v1/messages` directly (`base_url: "https://api.anthropic.com"`, the key from `api_key_ref` travels as `x-api-key`, `anthropic-version: 2023-06-01`) instead of Anthropic's OpenAI-compatible layer — which rejects every sampling parameter on the Claude 5 models (the first production query failed on `temperature`), and, as observed on the ku-heimdall install, drops `response_format` silently and offers neither prompt caching nor effort control. The request carries **no sampling parameters** (`temperature`/`top_p`/`top_k` are 400s on these models; `num_ctx` stays ctx's own prompt budget), the system prompt as a top-level block with `cache_control: ephemeral` (cache hits need a ≥512-token prefix; verify in `metadata.cache_read_input_tokens` of the `context_llm_log` row), `max_tokens` from the pipeline's `num_predict` (16000 when a caller sets none; **thinking tokens count against it** — the Claude 5 models think adaptively and cannot be told not to, so a model_map `think:false` is ignored on this wire; a role whose rows show `metadata.finish_reason = length` needs a larger `model_map` `params.max_tokens` for that role), and `output_config.effort`: the role's `model_map` `params.effort` beats the row's `extra_body.effort`, which beats the explicit default `medium` (the model-side default of Claude Opus 5.5); both carriers accept only `low|medium|high|xhigh|max` (422 otherwise); the field is always sent, so the row needs a model that takes it (the Claude 5 family; Sonnet 4.5 / Haiku 4.5 reject it, and on Opus 4.7/4.8 an omitted `thinking` field means no thinking — neither is a supported row). Every other `extra_body` key merges top-level (the hatch for `thinking.display`, `stop_sequences`, …; the row wins over model_map params on collision, as on the openai wire). **JSON mode is a no-op** on this wire: `output_config.format` (structured outputs) needs a JSON schema with `additionalProperties:false` on every object, which ctx's schemaless `json_object` mode does not have, so `ChatJSON` sends the same body as `Chat` and the pipelines' local parsers stay the validator — run Dream with `CTX_DREAM_JSON_MODE=off` (see [operations](operations.md#environment-variables)). Responses: text blocks are concatenated (thinking blocks skipped), `prompt_tokens` is the **total** prompt (`input_tokens` + `cache_read_input_tokens` + `cache_creation_input_tokens` — Anthropic's `input_tokens` is the uncached remainder only), the two cache counters land in `metadata`, the message id in `metadata.provider_request_id`, `cost_usd` stays NULL (ctx does not price Anthropic; read the console), and `stop_reason` is mapped onto the vocabulary the consumers read (`end_turn`/`stop_sequence`/`pause_turn`→`stop`, `max_tokens`/`model_context_window_exceeded`→`length`, `tool_use`→`tool_calls`, `refusal`→`content_filter`; unknown values pass through raw — a `refusal` after partial text still ends the turn with that text, marked `content_filter` in the `done` event). Errors classify like every other wire: 429 → rate-limit cooldown (`Retry-After` honoured), 529 `overloaded_error` → gateway (failover), 401 → auth, 400 → request-specific, 500 `api_error` → stop (the pool's HTTP-500 doctrine); the provider's error type prefixes the logged message, and the same types inside a mid-stream `error` event (HTTP 200) classify the same way. Row shape (422 on create and update): `api_key_ref` required, `locality: external`, `num_ctx > 0` (no window discovery on this wire — synthesis would fail closed), `provider_class: generic`, no `embed`/`dream-embed`/`rerank` roles (the Messages API has no such endpoints). `backend-test` probes `GET {base_url}/v1/models` with the key, then the 1-token chat. The web-chat stream path (`ChatStreamBackend`) maps the Messages API SSE events (`content_block_delta` text/`input_json_delta`, `message_delta` stop reason + usage, `error`) onto the same event contract as the openai wire, tools included (`tool_use`/`tool_result` blocks, consecutive tool results folded into one user turn). Three repairs keep a session alive on this wire: the answer's `thinking` blocks, when the model emits any, ride along in memory to the next iteration of the same turn and are replayed verbatim ahead of the `tool_use` they introduced (the reference asks that a turn whose `tool_use` still awaits its result keep its thinking block, and an altered block is rejected; live on 2026-10-11 a short tool turn carried no thinking block and the round trip was accepted — they are not persisted, later turns replay without them), a `tool_use` the history never answered (interrupted turn) gets a synthetic `is_error` `tool_result`, and tool calls that arrived without an id from an openai-wire row earlier in the session get synthetic ids. The system block's breakpoint covers the stable prefix only — the per-call `promptguard` nonce rule goes into a second, uncached block — and the stream path adds the top-level automatic `cache_control` so the growing history is read from cache across iterations and turns. Not available on this wire by construction: logprobs (the dream decide primitive falls back to its generating prompt and memoises the backend as decide-incapable), forced `tool_choice` (a 400 on Claude Opus 5.5; `auto` is what both wires send).
 
 ## Web chat sessions
 
 (F6, migration 056) The persistence layer, server-side tool harness and streaming HTTP endpoint for web chat. `context_chat_sessions` is **scope-owned**: list and delete key on the creating tenant's home scope, so a key never sees a foreign tenant's chats. It snapshots the creating key's `read_scopes` and carries a **monotone `max_sensitivity` high-water-mark**. Reading or continuing a session requires `session.read_scopes ⊆ caller.ReadScopes` (else 404, no oracle), closing the shadow-corpus channel against future least-privilege keys. The HWM rises with **every** appended message (in the same short transaction that assigns the message `seq`), so a credentials-touched session stays full-trust-only for its whole life. `context_chat_messages` records per-message `sensitivity` (fail-closed `credentials` default), tool-call metadata, telemetry and a gapless `seq` (`UNIQUE(session_id, seq)`). A turn claims its session via a short `busy_until` CAS — a second concurrent turn gets 409 without blocking, a crashed turn self-heals on expiry. Retention is off by default (`CTX_WEBCHAT_SESSION_RETENTION`).
 
-The **harness** (`internal/chat`) drives the model loop (model call → tool execution → next call), re-resolving the F3 chat chain each iteration on `max(request, session HWM)` sensitivity (an empty chain ends the turn — never a silent escalation). Four read-only tools run under the session's `read_scopes` snapshot: **ctx_query** (hybrid retrieval, delegated to `/api/query` with `synthesize:false`+`include_content:true`, attributed to the real key), **ctx_search**, **ctx_get** (full block, paged past the window via a resumable `offset`), **ctx_recent**. Each result is annotated with `max(sensitivity)` of the blocks it carried; tools are offered only to a full-trust backend, and the closing call after the tool-budget cap carries no tools array. Tool errors return to the model as `{"error":…}` and never abort the turn. Events flow through a narrow `Sink` interface, so a future headless agent runner can drive the same loop without HTTP.
+The **harness** (`internal/chat`) drives the model loop (model call → tool execution → next call), re-resolving the F3 chat chain each iteration on `max(request, session HWM)` sensitivity (an empty chain ends the turn — never a silent escalation). Each model call streams over the wire of the backend's `protocol` (`ChatStreamBackend`): the OpenAI-compatible SSE for `openai` and `ollama` rows, the native Messages API SSE for `anthropic` rows (tool calls as `tool_use`/`tool_result` blocks, see the Anthropic paragraph under [Backend pool](#backend-pool)) — the harness's event contract (`delta`, `tool_call_start`, `done`) is the same on both. Four read-only tools run under the session's `read_scopes` snapshot: **ctx_query** (hybrid retrieval, delegated to `/api/query` with `synthesize:false`+`include_content:true`, attributed to the real key), **ctx_search**, **ctx_get** (full block, paged past the window via a resumable `offset`), **ctx_recent**. Each result is annotated with `max(sensitivity)` of the blocks it carried; tools are offered only to a full-trust backend, and the closing call after the tool-budget cap carries no tools array. Tool errors return to the model as `{"error":…}` and never abort the turn. Events flow through a narrow `Sink` interface, so a future headless agent runner can drive the same loop without HTTP.
 
 **Endpoints** (auth required; `CTX_WEBCHAT_ENABLED=false` ⇒ 404):