From c90191c7c4c8accad9619f924870cacc170b2609 Mon Sep 17 00:00:00 2001 From: Mariusz Sabath Date: Tue, 17 Nov 2026 15:16:35 -0500 Subject: [PATCH 1/2] Docs: Lead the export guide with what Cortex records Three documents cover exporting, and their names do not say which to open. `session-export.md` and `session-export-guide.md` differ by one word and cover the same endpoint. `session-dump.md` covers a different tool with the opposite privacy posture. A reader could not tell them apart. Rename the reference to `session-export-reference.md`, so the pair reads `-guide` and `-reference` rather than bare against suffixed. Update every inbound link, including `doc_example_test.go`, which reads the file by path and would otherwise fail. Add a "What an export contains" section at the top of the guide. It states what a record carries, as a table of all 36 fields; what a record never carries; which questions an export answers; and which it cannot. The claim is verified against the struct, not written from memory. Reorder the steps to the order a reader works in: list the sessions, export, narrow, analyse. The archive pre-check was step 1. It asked the reader to interpret a `null` before seeing the tool run, and on a laptop the answer is always yes. It now sits under "When the proxy has no archive", where it is the diagnosis for a reader who needs it. Fold "find the id" into the listing step, because listing is how you find an id. Lead the three index rows in `docs/README.md` with content or no content. Both export rows began "Export the...", so the privacy posture, which is the only thing distinguishing them, was invisible. Prose follows ASD-STE100: direct sentences, one action per step, one term per concept. Signed-off-by: Mariusz Sabath Assisted-By: Claude (Anthropic AI) --- cmd/agentop/cmd_export.go | 2 +- core/sessionexport/doc_example_test.go | 10 +- docs/README.md | 6 +- docs/session-api.md | 2 +- docs/session-dump.md | 2 +- docs/session-export-guide.md | 197 +++++++++++------- ...-export.md => session-export-reference.md} | 10 +- 7 files changed, 142 insertions(+), 87 deletions(-) rename docs/{session-export.md => session-export-reference.md} (96%) diff --git a/cmd/agentop/cmd_export.go b/cmd/agentop/cmd_export.go index 65e91af68..a0e64fca8 100644 --- a/cmd/agentop/cmd_export.go +++ b/cmd/agentop/cmd_export.go @@ -70,7 +70,7 @@ which is what makes it shareable where the full events are not. It is not anonym though: hosts, models, session ids and timestamps remain, and together they describe who talked to what and when. A month of traffic is about 12MB. The reading rules that keep a consumer from misinterpreting the fields (zero means "not recorded", never a real zero; -order by "at" then "seq") are in Cortex's docs/session-export.md. +order by "at" then "seq") are in Cortex's docs/session-export-reference.md. The export reads the durable archive, not memory: a session with nothing flushed yet does not appear, and the live tail is /v1/events, not this. diff --git a/core/sessionexport/doc_example_test.go b/core/sessionexport/doc_example_test.go index d0f082128..56c86173b 100644 --- a/core/sessionexport/doc_example_test.go +++ b/core/sessionexport/doc_example_test.go @@ -9,26 +9,26 @@ import ( "testing" ) -// TestDocExampleRecordMatchesWire pins that the record example in docs/session-export.md +// TestDocExampleRecordMatchesWire pins that the record example in docs/session-export-reference.md // strict-decodes into Record. The doc once showed promptTokens and completionTokens — // fields Record deliberately has not, the very double count its own reading rules warn // about — so every jq written against the example came back null (#1830 review). The // strict decode is the pin: a field the projection does not emit fails here before it // ships again, the way sessionapi's TestDocExamplePromptContextMatchesWire holds its doc. func TestDocExampleRecordMatchesWire(t *testing.T) { - raw, err := os.ReadFile(filepath.Join("..", "..", "docs", "session-export.md")) + raw, err := os.ReadFile(filepath.Join("..", "..", "docs", "session-export-reference.md")) if err != nil { - t.Fatalf("read docs/session-export.md: %v", err) + t.Fatalf("read docs/session-export-reference.md: %v", err) } doc := string(raw) start := strings.Index(doc, "```json\n") if start < 0 { - t.Fatal("docs/session-export.md carries no json example") + t.Fatal("docs/session-export-reference.md carries no json example") } scope := doc[start+len("```json\n"):] end := strings.Index(scope, "```") if end < 0 { - t.Fatal("docs/session-export.md's json example is never closed") + t.Fatal("docs/session-export-reference.md's json example is never closed") } example := scope[:end] diff --git a/docs/README.md b/docs/README.md index 81ff42f4e..93a2b449e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,9 +17,9 @@ holds the repo-level pieces; this file is the index. | Run an agent on a local model, or switch inference servers | [`laptop-local-model.md`](laptop-local-model.md) | | Use the `agentop` TUI | [`cmd/agentop/README.md`](../cmd/agentop/README.md) | | Add your agent's sessions from before Cortex | [`agentop import`](../cmd/agentop/README.md#importing-agent-history-agentop-import) | -| Export sessions to files | [`session-dump.md`](session-dump.md) | -| Export the archive for offline analysis (content-free records) | [`session-export.md`](session-export.md) | -| Walk through exporting and analysing it, with the traps | [`session-export-guide.md`](session-export-guide.md) | +| **Without content:** measure traffic — tokens, latency, errors. Start here | [`session-export-guide.md`](session-export-guide.md) | +| **Without content:** look up what a record field means | [`session-export-reference.md`](session-export-reference.md) | +| **With content:** keep prompts, completions and tool results as files | [`session-dump.md`](session-dump.md) | ## Kubernetes and architecture diff --git a/docs/session-api.md b/docs/session-api.md index ce0e28402..715f92685 100644 --- a/docs/session-api.md +++ b/docs/session-api.md @@ -27,7 +27,7 @@ through DNS rebinding. The stats server applies the same check. | `DELETE /v1/sessions` | `application/json` | Clear every session: the store, and the session archive where one runs. The cost ledger is kept | | `DELETE /v1/sessions/{id}` | `application/json` | Erase one session: the store's entry and, where the archive runs, its files (see below) | | `GET /v1/events` | `text/event-stream` | SSE stream of new events; `?session=` filters to one session. Heartbeat every 30s | -| `GET /v1/export` | `application/x-ndjson` | The session archive as content-free records, one per event — the export for offline analysis; `?session=` and `?since=` select. A walk that dies past its first record resets the connection rather than ending the body cleanly, so a truncated export reaches a client as an error ([`session-export.md`](session-export.md)) | +| `GET /v1/export` | `application/x-ndjson` | The session archive as content-free records, one per event — the export for offline analysis; `?session=` and `?since=` select. A walk that dies past its first record resets the connection rather than ending the body cleanly, so a truncated export reaches a client as an error ([`session-export-reference.md`](session-export-reference.md)) | | `GET /v1/pipeline` | `application/json` | Active pipeline composition, per plugin: name, direction, position, capabilities, runtime `config`. `onError` is sent only when it is not the default — in practice `observe`, since an `on_error: off` plugin is not built. A plugin under observe runs and records, but its rejections, body writes and redirects are dropped (header writes still apply), so its `config` says what it would do. agentop renders this as the Pipeline pane (`P`) | | `GET /v1/plugins` | `application/json` | Catalog of every registered plugin, active or not. 404 when the binary built its session API without `WithCatalog`. agentop renders this as the Catalog pane (`C`) | | `GET /v1/usage` | `application/json` | Windowed usage aggregation — the aggregate behind `agentop cost`; parameters and semantics in [`pricing.md`](pricing.md#reading-usage-over-http) | diff --git a/docs/session-dump.md b/docs/session-dump.md index 769115631..cb0b34d61 100644 --- a/docs/session-dump.md +++ b/docs/session-dump.md @@ -23,7 +23,7 @@ holds at the moment it runs. You must run it before the thing you want is gone. laptop the session archive closes that gap. In a cluster the gap is still open. A dump is the *content* — prompts, completions, tool results. When what you want is the -*measurements* over a month of traffic, [`GET /v1/export`](session-export.md) writes the +*measurements* over a month of traffic, [`GET /v1/export`](session-export-reference.md) writes the archive as content-free records instead: no prompts, completions or tool results, which is what makes it shareable where a dump is not. It is not anonymous, though — hosts, models, session ids and timestamps remain, and together they describe who talked to what diff --git a/docs/session-export-guide.md b/docs/session-export-guide.md index 08fc66117..5d7e7aab4 100644 --- a/docs/session-export-guide.md +++ b/docs/session-export-guide.md @@ -1,7 +1,8 @@ # Export your session data: a walkthrough -[`session-export.md`](session-export.md) is the reference. It tells you what each field -means. This page tells you how to get an export and how to read it correctly. +This page tells you how to get an export and how to read it correctly. +[`session-export-reference.md`](session-export-reference.md) is the reference. It tells +you what each field means. Every command and every output on this page was run against a served archive. @@ -9,56 +10,106 @@ Every command and every output on this page was run against a served archive. request or response that Cortex captured. One event becomes one record. The **store** is what Cortex holds in memory. The **archive** is what Cortex has written to disk. -- [Step 1: confirm the proxy has an archive](#step-1-confirm-the-proxy-has-an-archive) -- [Step 2: list what the archive holds](#step-2-list-what-the-archive-holds) - - [Find the id of the session you want](#find-the-id-of-the-session-you-want) -- [Step 3: export it](#step-3-export-it) -- [Step 4: select less than everything](#step-4-select-less-than-everything) -- [Answer a question with the export](#answer-a-question-with-the-export) +- [What an export contains](#what-an-export-contains) +- [Step 1: list the sessions you can export](#step-1-list-the-sessions-you-can-export) +- [Step 2: export](#step-2-export) +- [Step 3: select less than everything](#step-3-select-less-than-everything) +- [Step 4: answer a question with the export](#step-4-answer-a-question-with-the-export) - [Three traps](#three-traps) -- [Export from a cluster sidecar](#export-from-a-cluster-sidecar) +- [When the proxy has no archive](#when-the-proxy-has-no-archive) - [Use the export in a script](#use-the-export-in-a-script) - [Read the refusals](#read-the-refusals) -## Step 1: confirm the proxy has an archive +## What an export contains -The export reads the archive on disk. It does not read the store in memory. A laptop -install has an archive. Most other deployments do not. +Cortex records the **shape** of your traffic. It does not record the content. An export +tells you how much, how fast, how often, and to where. It cannot tell you what was said. -Ask the proxy what its archive holds: +Read this section before you export. It decides whether this is the right tool. -```sh -curl -s 'http://127.0.0.1:47601/v1/sessions?archived=true' | jq '.archive' -``` +### What a record carries -```json -{ - "bytes": 493, - "maxBytes": 2147483648, - "retentionDays": 30 -} -``` +One record is one event. A record holds 36 fields. Every field is a number or a short +label. -`retentionDays` is how long the archive keeps a session. `maxBytes` is its size limit. +| Group | Fields | +|---|---| +| Identity | `sessionId`, `seq`, `requestId`, `at` | +| Destination | `host`, `requestedHost`, `httpMethod` | +| Outcome | `statusCode`, `errorKind`, `errorCode`, `finishReason` | +| Timing | `durationMs`, `ttfbMs` | +| Volume | `bytesUp`, `bytesDown` | +| Tokens | `inputTokens`, `cacheReadTokens`, `cacheWriteTokens`, `outputTokens`, `reasoningTokens` | +| Model | `model`, `requestedModel`, `servedModel`, `stream`, `streamedResponse` | +| Counts | `messageCount`, `toolCount`, `toolCallCount`, `presentKinds` | +| Caller | `client`, `agentRole` | +| Transport | `direction`, `phase`, `tunnel`, `tunnelReason`, `schema` | -If the reply is `null`, this proxy has no archive. Go to -[Export from a cluster sidecar](#export-from-a-cluster-sidecar). +### What a record never carries -## Step 2: list what the archive holds +A record never contains the items below. Cortex does not remove them. Cortex never copies +them in. A record is an allowlist: a field must be added on purpose to appear. + +- Prompts and user queries +- Model completions and reasoning text +- Tool arguments, such as a `Bash` command line +- Tool results, and tool names +- MCP parameters and results, and A2A message parts and artifacts +- Request and response bodies, and request headers +- Authenticated identity: subject, client id, agent id, scopes, SPIFFE id +- URL paths + +A test holds this contract. `TestRecord_CarriesNoContent` puts 24 distinct secrets in the +content-bearing fields of an event. It then encodes the record. The build fails if any +secret reaches the encoded bytes. + +### Questions an export answers + +- Token spend per model +- Latency percentiles, including time to first byte +- Cache hit rates +- Which hosts an agent reached +- Error rates and refusal rates +- Agent-version adoption +- How traffic changed across a release + +### Questions an export cannot answer + +- What a user asked +- What the model replied +- Which file a tool touched +- Why a request was made +- Who the user was +- Which tools ran. A record counts them in `toolCallCount`. It never names them. + +An export is **content-free**. It is **not anonymous**. It keeps hosts, models, session +ids and timestamps. Together these show who called what, and when. Read an export before +you send it somewhere. + +If you need any of the content above, use [`session-dump.md`](session-dump.md) instead. + +## Step 1: list the sessions you can export + +Ask the proxy which sessions the archive holds: ```sh curl -s 'http://127.0.0.1:47601/v1/sessions?archived=true' \ - | jq -r '.sessions[] | [.id, .eventCount, (.resident != false)] | @tsv' + | jq -r '.sessions[] + | [.id, .eventCount, (.resident != false), (.title // "(untitled)")[0:50]] + | @tsv' ``` ``` -old-session 3 false -today-session 4 false +849102c5-e59f-404c-8b2e-3dde35e052bb 412 true please address the review comments +59046505-b6f3-43f5-a2ef-9071c9a847a2 188 false https://github.com/rossoctl/cortex/pull/1854 +0e2f9a41-7c3d-4b18-9f52-6a1e4d7b2c90 24 false (untitled) ``` -The third column is `resident`. Test it with `!= false`, not with `// true`: in `jq` the -`//` operator treats `false` as absent, so `// true` would report every session as -resident. +Each row is one session: its id, its event count, whether the store still holds it, and +its title. + +The third column is `resident`. Test it with `!= false`, not with `// true`. In `jq` the +`//` operator treats `false` as absent, so `// true` reports every session as resident. The values mean: @@ -71,47 +122,33 @@ Cortex drops a session from the store when the store reaches its session limit, the session expires. The archive keeps the session after that. This is the reason to export: the archive holds history that memory has lost. -### Find the id of the session you want +If the command returns no sessions, see +[When the proxy has no archive](#when-the-proxy-has-no-archive). -`--session` takes a **session id**, and it matches that id exactly. It does not match a -title, and it does not match a prefix. +### Copy the id you need + +`--session` takes a **session id**. It matches that id exactly. It does not match a title. +It does not match a prefix. A session id is the value a client asserted. It is often a UUID. It can also be a plain word, such as `default`. -An id alone does not tell you which session it is. List the ids with their titles: - -```sh -curl -s 'http://localhost:47601/v1/sessions?archived=true' \ - | jq -r '.sessions[] | "\(.id) \((.title // "(untitled)")[0:60])"' -``` - -``` -849102c5-e59f-404c-8b2e-3dde35e052bb please address the review comments -59046505-b6f3-43f5-a2ef-9071c9a847a2 https://github.com/rossoctl/cortex/pull/1854 -24e624cb-b578-42c4-9c8b-9bfbe29341ab please address Alek's comments -17051bcc-5ae8-40eb-a975-48488242b93d Our rossoctl org has several repos -0e2f9a41-7c3d-4b18-9f52-6a1e4d7b2c90 (untitled) -``` - A session has no title until its events carry one: a `/rename`, a user query, or ordinary -prose. Tunnel-only traffic and utility clients therefore stay untitled. Keep those rows — -they are the ones you most need an id for. - -Copy the id from the first column. +prose. Tunnel-only traffic and utility clients therefore stay untitled. Keep those rows. +They are the ones you most need an id for. > **A title is user content. An export is not.** > > Cortex derives a title from the session's own events: a `/rename`, else a user query, > else ordinary conversation. A title therefore contains prompt text. > -> The export excludes prompt text by design. The command above does not. Use it in your -> own terminal. Do not paste its output where you would paste an export. +> The export excludes prompt text. The command above does not. Run it in your own +> terminal. Do not paste its output where you would paste an export. -The export carries no title. A record identifies its session by `sessionId` only. To -label a report, join the ids back to the titles yourself. +The export carries no title. A record identifies its session by `sessionId` only. To label +a report, join the ids back to the titles yourself. -## Step 3: export it +## Step 2: export ```sh agentop export --out usage.ndjson @@ -128,7 +165,7 @@ can be the export itself: agentop export | jq -r '.model // empty' | sort | uniq -c ``` -Each record is one event, reduced to about 30 numbers and short strings: +Each record is one event, reduced to numbers and short labels: ```json { @@ -149,12 +186,7 @@ Each record is one event, reduced to about 30 numbers and short strings: ``` Look at what the record does not have. The source event held this message content: -`"SECRET-PROMPT-DO-NOT-EXPORT"`. The record has `"messageCount": 1` in its place. The -export carries no prompts, no completions and no tool arguments. This is why you can -share an export where you cannot share the full events. - -An export is not anonymous. It keeps hosts, models, session ids and timestamps. Together -these show who called what, and when. Read an export before you send it somewhere. +`"SECRET-PROMPT-DO-NOT-EXPORT"`. The record has `"messageCount": 1` in its place. The command refuses to overwrite a file: @@ -165,7 +197,7 @@ agentop export: usage.ndjson already exists; --force overwrites it Add `--force` to overwrite it. -## Step 4: select less than everything +## Step 3: select less than everything Select by time: @@ -174,7 +206,7 @@ agentop export --since 7d ``` Select by session id. Get the id from -[Step 2](#find-the-id-of-the-session-you-want): +[Step 1](#copy-the-id-you-need): ```sh agentop export --session 849102c5-e59f-404c-8b2e-3dde35e052bb @@ -206,7 +238,7 @@ A session can appear in an export with fewer events than it has. Example: a sess started a month ago, and it ran again this morning. `--since 24h` carries this morning's events from it. `--since 24h` excludes its older events. -## Answer a question with the export +## Step 4: answer a question with the export The export is NDJSON. One record per line. `jq` reads it directly. @@ -432,7 +464,28 @@ agentop export --session default \ 2026-11-16T23:08:45.84114Z gap=191h ``` -## Export from a cluster sidecar +## When the proxy has no archive + +The export reads the archive on disk. It does not read the store in memory. A laptop +install has an archive. Most other deployments do not. + +Ask the proxy what its archive holds: + +```sh +curl -s 'http://127.0.0.1:47601/v1/sessions?archived=true' | jq '.archive' +``` + +```json +{ + "bytes": 493, + "maxBytes": 2147483648, + "retentionDays": 30 +} +``` + +`retentionDays` is how long the archive keeps a session. `maxBytes` is its size limit. + +A reply of `null` means this proxy has no archive. An injected sidecar normally runs no archive. The endpoint then answers 404: @@ -588,7 +641,7 @@ this error, because it escapes its own query. A hand-written `curl` can. ## See also -- [`session-export.md`](session-export.md) — the fields of a record, and the full reading +- [`session-export-reference.md`](session-export-reference.md) — the fields of a record, and the full reading rules - [`session-api.md`](session-api.md) — every endpoint, `GET /v1/export` included - [`session-dump.md`](session-dump.md) — use this when you want the content: prompts, diff --git a/docs/session-export.md b/docs/session-export-reference.md similarity index 96% rename from docs/session-export.md rename to docs/session-export-reference.md index 5fe0262b9..ff2567634 100644 --- a/docs/session-export.md +++ b/docs/session-export-reference.md @@ -1,4 +1,9 @@ -# Export sessions for analysis +# Export sessions for analysis: reference + +> **New to the export?** Start with +> [`session-export-guide.md`](session-export-guide.md). It walks through one export, and +> it lists the three traps that give a wrong answer with no error. This page is the field +> reference. Use it to look something up. `GET /v1/export` writes the session archive as NDJSON. One line per archived event. It is on the [Session Events API](session-api.md). @@ -21,9 +26,6 @@ rates, tool failure rates. A live TUI cannot answer these. [`agentop cost`](pricing.md) answers a few of them inside the tool. This endpoint is the layer behind anything else. -For a step-by-step walkthrough, see -[`session-export-guide.md`](session-export-guide.md). This page is the reference. - ## What a record is One line is one event. A record is content-free by construction. It is an allowlist of From 1b8a1b9a64606f7c10420cb4c11ca45f0ee0692e Mon Sep 17 00:00:00 2001 From: Mariusz Sabath Date: Tue, 17 Nov 2026 17:59:47 -0500 Subject: [PATCH 2/2] Docs: Fix the no-archive signal, and pin the counts this guide states Review of #1931 found the step-1 pointer names a symptom that does not happen. `handleList` (core/sessionapi/server.go:439) merges archived rows only when `s.archive != nil`. Otherwise it ignores `archived=true` and answers with the store's sessions, every one of them resident. So a reader with no archive sees a full table, never follows the pointer, and learns the truth at step 2 from a 404. Trigger on the `archive` key instead, which is the signal the linked section already uses. Qualify the content claim. Cortex does record content, in the store and in the archive; the export is the path that leaves it behind. The previous wording read as a claim about Cortex. Qualify the allowlist promise for `sessionId`. It is client-asserted and copied through uncapped to 256 bytes (core/session/store.go:193), unlike every neighbouring label, which `exportLabel` caps at 96. A client may use a sentence as its session id. Both reviewers asked for this, and it is the one field where client text reaches a record. Separate `client` from authenticated identity. The never-carries list excludes "client id" while the Caller group carries `client`; the two read as contradicting each other. `client` is the User-Agent product label. Document the comma split in `--session`. The flag takes a comma-separated list (cmd/agentop/cmd_export.go:254), so an id containing a comma selects two sessions. `?session=` takes the whole value, verified against the handler. Say that a resident session may not be exportable yet. The archive is at most a tick behind the store. Pin the stated counts, rather than restating them. Three places state the field count, and before this PR the guide said 36 while the reference and `--help` said "about 30". `TestDocGuideFieldCountMatchesRecord` now reads the number out of all three and compares it with `reflect`, and `TestDocGuideSecretCountMatchesAllSecrets` does the same for `allSecrets()`. Both were mutation-tested: each fails with an actionable message when the number is wrong. Correct two further errors found while verifying: - `presentKinds` is not "which content kinds the event held". It names which TOKEN COUNTS the provider reported, and is the one field that escapes the zero-means-absent rule (record.go:201). Moved from Counts to Tokens. - The listing sample was typeset by hand, which `@tsv` cannot produce. Added `column -t -s $'\t'`, which a plain `column -t` cannot replace: it splits on any whitespace, so a title with spaces becomes several columns. Also add `presentKinds` and `reasoningTokens` to the reference, and make its pointer to `record.go` a real link. A reader looking either one up found neither. Signed-off-by: Mariusz Sabath Assisted-By: Claude (Anthropic AI) --- cmd/agentop/cmd_export.go | 2 +- core/sessionexport/doc_example_test.go | 81 ++++++++++++++++++++++++++ docs/session-export-guide.md | 80 ++++++++++++++++++++----- docs/session-export-reference.md | 22 +++++-- 4 files changed, 165 insertions(+), 20 deletions(-) diff --git a/cmd/agentop/cmd_export.go b/cmd/agentop/cmd_export.go index a0e64fca8..48eb028e3 100644 --- a/cmd/agentop/cmd_export.go +++ b/cmd/agentop/cmd_export.go @@ -64,7 +64,7 @@ Usage: agentop export --session ID one session, repeatable or comma-separated agentop export --endpoint URL ask a specific proxy rather than the local one -Each line is one archived event reduced to ~30 content-free scalars — timings, token +Each line is one archived event reduced to 36 content-free scalars — timings, token counts, hosts, models, outcomes — never message content, completions or tool arguments, which is what makes it shareable where the full events are not. It is not anonymous, though: hosts, models, session ids and timestamps remain, and together they describe who diff --git a/core/sessionexport/doc_example_test.go b/core/sessionexport/doc_example_test.go index 56c86173b..9de12fef9 100644 --- a/core/sessionexport/doc_example_test.go +++ b/core/sessionexport/doc_example_test.go @@ -5,6 +5,9 @@ import ( "encoding/json" "os" "path/filepath" + "reflect" + "regexp" + "strconv" "strings" "testing" ) @@ -50,3 +53,81 @@ func TestDocExampleRecordMatchesWire(t *testing.T) { t.Errorf("example carries no real token figures: %+v", r) } } + +// readGuide returns docs/session-export-guide.md, which states two counts about this +// package as prose. +func readGuide(t *testing.T) string { + t.Helper() + raw, err := os.ReadFile(filepath.Join("..", "..", "docs", "session-export-guide.md")) + if err != nil { + t.Fatalf("read docs/session-export-guide.md: %v", err) + } + return string(raw) +} + +// docCount pulls the one number a pattern captures out of a document. +func docCount(t *testing.T, doc, pattern string) int { + t.Helper() + m := regexp.MustCompile(pattern).FindStringSubmatch(doc) + if m == nil { + t.Fatalf("no count matches %q any more — if the sentence was reworded, "+ + "reword this pattern with it", pattern) + } + n, err := strconv.Atoi(m[1]) + if err != nil { + t.Fatalf("count %q is not a number: %v", m[1], err) + } + return n +} + +// TestDocGuideFieldCountMatchesRecord pins the guide's field count to Record. +// +// The guide's "What an export contains" section tables every field, so a reader can decide +// whether an export is shareable before running one. That table is a hand-maintained copy +// of this struct, and the prose above it states the count. A field added to Record made the +// sentence quietly wrong, in the direction that matters: a reader would trust a list that +// no longer described the export (#1931 review). +// +// Neither existing pin covers this. TestDocExampleRecordMatchesWire pins the REFERENCE +// page's json example, and TestRecord_EveryFieldIsPopulated pins that FromEvent sets every +// field. Neither counts them. +// Three places state the field count: the guide's table, the reference's one-line +// summary, and `agentop export --help`. Before #1931 the guide said 36 while the other +// two said "about 30", which is how a hand-maintained count drifts. +func TestDocGuideFieldCountMatchesRecord(t *testing.T) { + want := reflect.TypeOf(Record{}).NumField() + for _, c := range []struct { + where, path, pattern string + }{ + {"docs/session-export-guide.md", filepath.Join("..", "..", "docs", "session-export-guide.md"), + `A record holds (\d+) fields`}, + {"docs/session-export-reference.md", filepath.Join("..", "..", "docs", "session-export-reference.md"), + `an allowlist of (\d+)\s+scalars`}, + {"cmd/agentop/cmd_export.go", filepath.Join("..", "..", "cmd", "agentop", "cmd_export.go"), + `reduced to (\d+) content-free scalars`}, + } { + raw, err := os.ReadFile(c.path) + if err != nil { + t.Fatalf("read %s: %v", c.where, err) + } + if got := docCount(t, string(raw), c.pattern); got != want { + t.Errorf("%s says %d fields, Record has %d — add the new field to the guide's "+ + "\"What a record carries\" table, or to \"What a record never carries\" if it "+ + "must not be exported, and fix all three counts", c.where, got, want) + } + } +} + +// TestDocGuideSecretCountMatchesAllSecrets pins the guide's planted-secret count. +// +// The guide cites the count as the evidence for the privacy contract. Understating it is +// the harmful direction: a 25th secret planted here without a doc edit leaves the guide +// claiming a weaker guarantee than the one the test holds, and the number is the reason a +// reader believes the claim at all (#1931 review). +func TestDocGuideSecretCountMatchesAllSecrets(t *testing.T) { + want := len(allSecrets()) + if got := docCount(t, readGuide(t), `puts (\d+) distinct secrets`); got != want { + t.Errorf("docs/session-export-guide.md says the test plants %d secrets, allSecrets() has %d — "+ + "update the count in \"What a record never carries\"", got, want) + } +} diff --git a/docs/session-export-guide.md b/docs/session-export-guide.md index 5d7e7aab4..1dabeb696 100644 --- a/docs/session-export-guide.md +++ b/docs/session-export-guide.md @@ -11,7 +11,7 @@ request or response that Cortex captured. One event becomes one record. The **st what Cortex holds in memory. The **archive** is what Cortex has written to disk. - [What an export contains](#what-an-export-contains) -- [Step 1: list the sessions you can export](#step-1-list-the-sessions-you-can-export) +- [Step 1: list the sessions](#step-1-list-the-sessions) - [Step 2: export](#step-2-export) - [Step 3: select less than everything](#step-3-select-less-than-everything) - [Step 4: answer a question with the export](#step-4-answer-a-question-with-the-export) @@ -22,15 +22,20 @@ what Cortex holds in memory. The **archive** is what Cortex has written to disk. ## What an export contains -Cortex records the **shape** of your traffic. It does not record the content. An export -tells you how much, how fast, how often, and to where. It cannot tell you what was said. +An export carries the **shape** of your traffic. It omits the content. An export tells you +how much, how fast, how often, and to where. It cannot tell you what was said. + +Cortex does record content. It holds it in the store and in the archive, and +[`session-dump.md`](session-dump.md) is how you get it. The export is the path that leaves +the content behind. Read this section before you export. It decides whether this is the right tool. ### What a record carries -One record is one event. A record holds 36 fields. Every field is a number or a short -label. +One record is one event. A record holds 36 fields. Cortex caps each label at 96 bytes, so +every field is a number or a short label. `sessionId` is the exception: see +[One field is not short](#one-field-is-not-short). | Group | Fields | |---|---| @@ -39,16 +44,31 @@ label. | Outcome | `statusCode`, `errorKind`, `errorCode`, `finishReason` | | Timing | `durationMs`, `ttfbMs` | | Volume | `bytesUp`, `bytesDown` | -| Tokens | `inputTokens`, `cacheReadTokens`, `cacheWriteTokens`, `outputTokens`, `reasoningTokens` | +| Tokens | `inputTokens`, `cacheReadTokens`, `cacheWriteTokens`, `outputTokens`, `reasoningTokens`, `presentKinds` | | Model | `model`, `requestedModel`, `servedModel`, `stream`, `streamedResponse` | -| Counts | `messageCount`, `toolCount`, `toolCallCount`, `presentKinds` | +| Counts | `messageCount`, `toolCount`, `toolCallCount` | | Caller | `client`, `agentRole` | | Transport | `direction`, `phase`, `tunnel`, `tunnelReason`, `schema` | +`client` is the product label from the User-Agent header, such as `claude-cli`. It is not +an authenticated client id. `agentRole` comes from the inference metadata. Neither one +names an authenticated principal. + +### One field is not short + +`sessionId` is the value a client asserted. Cortex copies it through unchanged, up to 256 +bytes. The 96-byte label cap does not apply to it. + +A client is therefore free to use a sentence as its session id. If a client does that, the +export carries that sentence. + +**Read the `sessionId` values before you share an export.** This is the one field where +client text can reach a record. + ### What a record never carries -A record never contains the items below. Cortex does not remove them. Cortex never copies -them in. A record is an allowlist: a field must be added on purpose to appear. +A record never contains the content below. Cortex does not remove it. Cortex never copies +it in. A record is an allowlist: a field must be added deliberately to appear. - Prompts and user queries - Model completions and reasoning text @@ -60,8 +80,11 @@ them in. A record is an allowlist: a field must be added on purpose to appear. - URL paths A test holds this contract. `TestRecord_CarriesNoContent` puts 24 distinct secrets in the -content-bearing fields of an event. It then encodes the record. The build fails if any -secret reaches the encoded bytes. +content-bearing fields of an event. It then encodes the record. The test fails if any +secret reaches the encoded bytes, which fails CI. + +The count above is pinned. `TestDocGuideSecretCountMatchesAllSecrets` fails if this page +and the test disagree. ### Questions an export answers @@ -88,7 +111,7 @@ you send it somewhere. If you need any of the content above, use [`session-dump.md`](session-dump.md) instead. -## Step 1: list the sessions you can export +## Step 1: list the sessions Ask the proxy which sessions the archive holds: @@ -96,15 +119,22 @@ Ask the proxy which sessions the archive holds: curl -s 'http://127.0.0.1:47601/v1/sessions?archived=true' \ | jq -r '.sessions[] | [.id, .eventCount, (.resident != false), (.title // "(untitled)")[0:50]] - | @tsv' + | @tsv' \ + | column -t -s $'\t' ``` ``` 849102c5-e59f-404c-8b2e-3dde35e052bb 412 true please address the review comments 59046505-b6f3-43f5-a2ef-9071c9a847a2 188 false https://github.com/rossoctl/cortex/pull/1854 -0e2f9a41-7c3d-4b18-9f52-6a1e4d7b2c90 24 false (untitled) +0e2f9a41-7c3d-4b18-9f52-6a1e4d7b2c90 24 false (untitled) ``` +`column -t` aligns the columns. Without it `@tsv` separates the fields with tabs, and a +short event count moves the later columns out of line. + +`-s $'\t'` is not optional. `column` splits on any whitespace by default, so a title with +spaces becomes several columns. + Each row is one session: its id, its event count, whether the store still holds it, and its title. @@ -118,11 +148,23 @@ The values mean: - **`false`**: the store has dropped this session, and the archive supplied this row. The viewer no longer shows it. The export still carries it. +A `resident` session is not always in the archive yet. Cortex writes to the archive a tick +behind the store, so a session that started seconds ago can be listed here and still be +absent from the export. Give it a moment and export again. + Cortex drops a session from the store when the store reaches its session limit, or when the session expires. The archive keeps the session after that. This is the reason to export: the archive holds history that memory has lost. -If the command returns no sessions, see +A proxy with no archive still answers this command. It ignores `archived=true` and returns +the store's sessions, each one `resident`. Check the `archive` key to tell the two apart: + +```sh +curl -s 'http://127.0.0.1:47601/v1/sessions?archived=true' | jq '.archive' +``` + +A reply of `null` means this proxy has no archive. The rows above then come from memory +only, and the export cannot read them. See [When the proxy has no archive](#when-the-proxy-has-no-archive). ### Copy the id you need @@ -130,6 +172,14 @@ If the command returns no sessions, see `--session` takes a **session id**. It matches that id exactly. It does not match a title. It does not match a prefix. +One exception: the flag splits its value at commas, because it accepts a comma-separated +list. So `--session task,one` asks for two sessions, `task` and `one`. To export an id that +contains a comma, call the endpoint directly. `?session=` reads the whole value as one id: + +```sh +curl -s 'http://127.0.0.1:47601/v1/export?session=task%2Cone' > one.ndjson +``` + A session id is the value a client asserted. It is often a UUID. It can also be a plain word, such as `default`. diff --git a/docs/session-export-reference.md b/docs/session-export-reference.md index ff2567634..4e8faa9d8 100644 --- a/docs/session-export-reference.md +++ b/docs/session-export-reference.md @@ -28,8 +28,10 @@ layer behind anything else. ## What a record is -One line is one event. A record is content-free by construction. It is an allowlist of -about 30 scalars projected from the event. It is never the event itself. +One line is one event. A record is content-free by construction. It is an allowlist of 36 +scalars projected from the event. It is never the event itself. +[The guide tables them](session-export-guide.md#what-a-record-carries), and a test pins +that count. ```json {"schema":1,"sessionId":"45fa31b2-…","seq":2077,"requestId":"a0-9e37cd", @@ -77,8 +79,20 @@ external collector is where someone will treat it as identity. raw tool results therefore overcounts by about 272x on real traffic. `toolCallCount` is the count of calls the model emitted. It is exported, and this problem does not affect it. -The doc comments on `core/sessionexport/record.go` are the full field list. They carry -these rules per field. This page is the summary. +Two fields need their own note: + +- **`presentKinds`** is a bitfield. It names which of the token counts the provider + actually reported. It is the one field that escapes the zero-means-absent rule above: a + set bit with a zero count means the provider reported zero, and an unset bit means the + provider does not expose that count. Bits are Input=1, CacheRead=2, CacheWrite=4, + Output=8, Reasoning=16. +- **`reasoningTokens`** is a **subset** of `outputTokens`, not an addition to it. Do not add + the two together. + +[`record.go`](../core/sessionexport/record.go) is the full field list. Its doc comments +carry the reading rules for each field, and +[the guide tables every field by group](session-export-guide.md#what-a-record-carries). +This page is the summary. ## Selection