Skip to content

docs: Add the new-user path: getting started, troubleshooting and a link checker #1938

Closed

Description

@huang195

Part 1 of the docs overhaul (parent: #1937). Do this one first. The other sub-issues use the link checker and the pages that this one adds.

Problem

A new laptop user has a good README, but nothing tells them what to do after agentop observe. Four things are missing:

  • No "check it works" step. No doc in the repo says what the first row in agentop observe looks like.
  • No page shows how to read the screen. The screen mock-up is at line 989 of cmd/agentop/README.md, and the key table starts at line 1558.
  • No troubleshooting page. Every fix is an ### section under "Three ways to turn it off" in docs/laptop-service.md. In the outline, those sections look like ways to turn Cortex off.
  • The CLI and the README disagree.
    • The last message of setup tells a user without --claude-code to run agentop exec -- <cmd>, not agentop configure claude-code enable (cmd/agentop/cmd_setup.go, the ui.Next("Open a new terminal, then:", ...) call).
    • For a --claude-code user, it prints bare agentop, which is deprecated (cmd/agentop/main.go). The README says agentop observe.
    • scripts/install.sh --help says "The session store is memory-only". This is false on a laptop: history is kept in ~/.cortex/sessions.

docs/README.md is also hard to scan:

  • Its "Getting started" table mixes first-day tasks with praxis policy enforcement and offline export analysis.
  • It holds the full configuration-reference table.
  • It has a typo: "group sessions into sessions".

Tasks

1. Add the link checker: scripts/dev/check-doc-links.py

It uses the Python standard library only, and it exits 1 when it finds a problem. It must check:

  • every relative Markdown link ](path) and ](path#anchor) in every *.md file, except under docs/superpowers/ and any testdata/ directory
  • that the path is a file or a directory
  • that each #anchor on a .md target matches a heading slug, using GitHub's rules:
    • lowercase
    • remove HTML tags
    • remove every character that is not a word character, - or a space
    • change each space to -
    • give the second identical slug -1, the third -2, and so on
    • also accept <a name=...> and <a id=...>
    • ignore headings inside fenced code blocks
  • that each path and anchor in the "Keep these paths valid" list below still exists, as a hard-coded list. A comment names the Go or shell file that prints each one.

Add a local pre-commit hook in .pre-commit-config.yaml that runs it on \.md$ changes (pass_filenames: false).

Add --selftest. It builds a temporary tree with one good link, one missing file, one missing anchor and one duplicate heading, and it asserts the result for each.

Reference implementation. It passes on main at 02e489e with no findings. Extend it with the file check, the hard-coded list and --selftest:

import os, re, sys
root = sys.argv[1]
def slugs(path):
    s, seen, fence = set(), {}, False
    for line in open(path):
        if line.lstrip().startswith('```'): fence = not fence; continue
        if fence: continue
        m = re.match(r'#{1,6}\s+(.*)', line)
        if not m: continue
        t = re.sub(r'<[^>]+>', '', m.group(1).strip()).lower()
        t = re.sub(r'[^\w\- ]', '', t).replace(' ', '-')
        n = seen.get(t, 0); seen[t] = n + 1
        s.add(t if n == 0 else f'{t}-{n}')
    for a in re.findall(r'<a (?:name|id)="([^"]+)"', open(path).read()): s.add(a)
    return s
cache = {}
for d, _, fs in os.walk(root):
    if 'superpowers' in d or 'testdata' in d: continue
    for f in fs:
        if not f.endswith('.md') or f == 'CLAUDE.md': continue
        p = os.path.join(d, f)
        for m in re.findall(r'\]\(([^)\s]+)\)', open(p).read()):
            if m.startswith(('http', 'mailto')) or '#' not in m: continue
            path, a = m.split('#', 1)
            t = os.path.normpath(os.path.join(d, path)) if path else p
            if not t.endswith('.md') or not os.path.exists(t): continue
            if t not in cache: cache[t] = slugs(t)
            if a not in cache[t]: print(os.path.relpath(p, root), '->', m)

Default root to the repository root (the parent of scripts/), so python3 scripts/dev/check-doc-links.py works with no argument. Skip .claude/ and node_modules/.

2. Add docs/getting-started.md (about 150 lines)

This is the page the README sends a new user to after the install. Keep this order and these headings:

  1. Install. The one-liner from the README. Say what setup asks, and that it asks once.
  2. Connect your agent. A table with one row per agent: the agentop configure <agent> enable command, the command to start the agent, and whether a running agent must restart. Get the facts from each docs/agents/<agent>.md, section "Enable and revert" or equal. Use agentop exec -- <command> for any other agent.
  3. Check that it works.
    • Run agentop observe and send one prompt.
    • Say what a new row shows: the agent, the model, tokens and cost.
    • Use the columns that cmd/agentop/README.md lists under "Panes" → Sessions.
    • When no row appears, run agentop doctor, then read docs/troubleshooting.md.
  4. Read the screen.
    • The Sessions screen mock-up from cmd/agentop/README.md (the fenced block of the band, table and footer, near the CTX text).
    • One line for each column.
    • A table of the 10 keys a new user needs: ↑/↓, Enter, Esc, /, ?, $, u, T, A, q.
    • Check each key in cmd/agentop/README.md → "Keybindings" and in the code under cmd/agentop/tui/.
    • End with: press ? for every key.
  5. Next steps. One line and one command each, with a link for more detail:
    • add history from before Cortex (agentop import --dry-run)
    • switch model server (agentop server, link laptop-local-model.md)
    • see cost (agentop cost --window month --by agent)
  6. Undo it. agentop configure <agent> disable, then restart the agent. agentop uninstall, and --purge to also delete ~/.cortex. Link laptop-service.md#remove-it.

3. Add docs/troubleshooting.md

Start with one section: "Run agentop doctor first". It checks the install and prints the fix for each problem.

Then move these sections from docs/laptop-service.md, as ## headings in this order. Keep each heading's text the same, so the existing anchors stay valid:

  1. Install:
    • "agentop: command not found"
    • "Restricted environments (sandboxes, no launchd session)"
  2. My agent does not show in agentop:
    • "Why traffic disappears from agentop"
    • "Everything shows as tunnel and no plugin ever runs"
    • "Developer tooling is not intercepted at all"
    • "Go tools on macOS are passed through unread"
  3. Errors:
    • "502 Cortex proxy is not serving"
    • "A request gets 421 Misdirected Request"
    • "Failing closed"

Use ## group headings for the three groups. End with "Problems with one agent": one link per agent to docs/agents/<agent>.md#known-issues, or to the equal section where a guide has no "Known issues".

Update every link to the moved anchors. Today these files link to them: docs/agents/claude-code.md, docs/agents/opencode.md, and comments in cmd/agentop/cmd_claudecode.go. Use grep to find the complete list.

4. Add docs/configuration.md

Move the section "Configuration reference" from docs/README.md without changes, together with the two paragraphs below its table. Fix the links in it for the new location.

5. Rewrite docs/README.md

Use three paths, in this order:

  • New here. Getting started, your agent's guide (one link per agent), Using agentop, Troubleshooting.
  • Daily use.
    • local models and servers
    • script plugins
    • remote machine
    • importing history
    • pricing and cost
    • running the service (laptop-service.md)
    • praxis policy
    • exporting sessions
  • Reference. Four groups:
    • Kubernetes: kubernetes.md, architecture.md, cmd/README.md, the demos
    • Plugins: the catalog, then the author docs in this order: plugin-tutorial.md, plugin-reference.md, script-plugin.md. Then the per-plugin list, which must include script.
    • Configuration (configuration.md), the session API, the proxy contract
    • Contributor and integrator contracts: idp-plugin-contract.md, lineage-wire-contract.md

Keep "In this directory". Remove "group sessions into sessions". Each table row is one short line.

6. Update README.md

  • After the Quick start commands, add one line: Next: Getting started. It shows what you should see, how to read the screen, and how to undo it.
  • In "Learn more", put "Getting started", "Using agentop" and "Troubleshooting" first. Then "All docs".
  • Remove the "full install guide" link to rossoctl.dev. That page is stale (see the parent issue). The README is the one full guide.
  • docs/using-agentop.md is added by sub-issue docs: Add an agentop user guide and reorder the agentop reference #1939. Until it exists, link cmd/agentop/README.md#panes with the text "Using agentop".

6a. Remove the cost-reduction pitch from README.md

Rule 11 has the details.

  • Change the tagline "Watch every model call, tool call and dollar your agent spends, live — then cut the waste." Keep the first part, and remove "then cut the waste".
  • Remove the section "Trim 4–20% off Claude Code's prompts" and its agentop tools scan --write command.
  • Remove "token savings" from the "Guides:" line.
  • Keep "Read-only until you turn on pruning, server routing or an enforcing script". It is a fact about data, not a pitch.
  • docs/README.md: put the token-savings row under "Reference" → Plugins, not under "New here" or "Daily use".

7. Fix the CLI text

  • cmd/agentop/cmd_setup.go, the last message of setup:
    • Without --claude-code: print agentop configure claude-code enable with the comment connect your agent (or opencode, codex, bob, pi). Then print agentop observe with the comment watch your agent traffic live.
    • With --claude-code: print agentop observe, not bare agentop.
    • Update the tests that assert this output. Find them with grep -rn 'send an agent through Cortex\|watch your agent traffic live' cmd/agentop.
  • scripts/install.sh usage():
    • Remove the block "After installing, to cut Claude Code's token cost:", the agentop tools scan --write line and "Then watch the $ saved on every prompt…". Keep one line: Then watch your agent live: agentop observe.
    • Check the setup output (cmd/agentop/cmd_setup.go and the setup steps) for the same pitch, and remove it.
    • Change bare agentop to agentop observe.
    • Replace the two lines "The session store is memory-only…" and cortex-session-dump --out ./cortex-dump with: History is kept in ~/.cortex/sessions. agentop import adds sessions from before Cortex.
    • Check that install.sh installs cortex-session-dump today. If it does, keep the tool and only change the help text.
  • cmd/agentop/main.go usage text: the configure <agent> line lists claude-code, bob, codex, opencode. Add pi, because cmd_configure.go dispatches it.

8. Fix stale facts in the new-user path

  • docs/agents/claude-code.md:
    • "Seven keys" (two places) disagrees with the table and with "six keys" (two places). Count the keys that cmd/agentop/cmd_claudecode.go writes, and use the code's number everywhere.
    • Do the same check for "six keys" in docs/laptop-service.md.
  • docs/agents/bob.md: remove the two references to configure bobshell. That command was removed. The shell half is now part of agentop configure bob.
  • docs/laptop-service.md:
    • "Bob … our internal coding agent (not the bob demo user)": describe IBM Bob as the README does.
    • "run agentop with no arguments": change it to agentop observe.
  • docs/laptop-token-savings.md: "run agentop" → agentop observe.
  • docs/laptop-service.md → "Remove it": it lists the agents that agentop uninstall unroutes. Check the list against the uninstall code in cmd/agentop, which includes Codex and Pi, and fix the list.

Acceptance

  • python3 scripts/dev/check-doc-links.py and --selftest pass. The pre-commit hook runs it.
  • docs/laptop-service.md no longer has the moved sections. Each section appears once, in docs/troubleshooting.md.
  • A reader can follow README.md → docs/getting-started.md from install to undo and open no other file.
  • agentop setup without --claude-code ends with configure … enable and agentop observe.
  • The cmd/agentop tests pass with -short.

Out of scope

For a human after review

None.

Rules for every PR in this overhaul

These rules are the same in every sub-issue of the docs overhaul, so each sub-issue is complete without the others.

  1. Language. Write in ASD-STE100 Simplified Technical English, in the prose mode:
    • one idea per sentence, at most 25 words
    • active voice
    • no semicolons
    • no phrasal verbs
    • no marketing adjectives
  2. Naming. Use Cortex for the product and its binaries. Use AuthBridge only for a frozen artifact name: an image, the authbridge-proxy container, a ConfigMap, a header, an env var, or the mode field. The full table is in the root CLAUDE.md, section "Naming".
  3. Move, do not copy. When a section moves to a new file:
    • Delete it from the old file.
    • Update every inbound link and anchor. Use grep -rn '<file>#<anchor>' --include='*.md' --include='*.go' --include='*.sh' . to find them.
    • In the old place, leave at most one line that points to the new place.
  4. Keep these paths valid. Released binaries and install.sh print them to users:
    • docs/pricing.md (cmd/agentop/cmd_config_migrate_pricing.go, cmd/agentop/cmd_service.go, cmd/cortex/local.go)
    • docs/praxis-policy.md and its anchor #move-from-the-cpex-plugin (cmd/agentop/cmd_update.go, cmd/agentop/setup_step_binaries.go, cmd/cortex/praxis.go, scripts/install.sh)
    • docs/session-export.md (cmd/agentop/cmd_export.go)
    • docs/laptop-local-model.md, as a full GitHub URL (cmd/agentop/cmd_server.go)
    • docs/laptop-service.md (code comments in cmd/agentop/cmd_service.go and cmd/agentop/cmd_claudecode.go. They name sections by title. If a section title changes or the section moves, update the comment.)
  5. New-user pages are complete. A reader of README.md, docs/getting-started.md, docs/using-agentop.md or docs/troubleshooting.md must finish each step without opening a reference doc. Links to reference docs are for "more detail" only.
  6. Facts come from the code. When a doc states a command, flag, key, port, count or file path, check it in the code on main before you write it. When the code and a doc disagree, change the doc to match the code.
  7. Do not state counts that drift. Do not write "all sixteen plugins" or "seven keys". Link to the list instead, or show the list.
  8. Size. One PR, at most 2,000 diff lines (git diff --stat main). Moved text counts twice.
  9. Gate. Run these before you push:
    • python3 scripts/dev/check-doc-links.py. The first sub-issue adds this script. Until it merges, use the reference implementation in that sub-issue.
    • pre-commit run --files <changed files>
    • When Go files change: go vet ./... and gofmt -l . in each module you touched. For cmd/agentop, also run scripts/dev/go-test-shards.sh -short cmd/agentop.
    • Add any new word that the spellcheck job rejects to the word list that .github/spellcheck/.spellcheck.yml names.
  10. PR title. Use docs: <Subject starting with uppercase>. End the PR body with Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>. Sign off every commit (git commit -s).
  11. No cost-reduction pitch for new users. A new user is not interested in it. Tool pruning comes back later, together with the context-guru integration. In README.md, docs/getting-started.md, docs/using-agentop.md, docs/troubleshooting.md, the "New here" part of docs/README.md, install.sh help and setup output, the first paragraph and "Use it" of each agent guide, and the first screen of docs/plugin-catalog.md:
    • do not mention cost reduction, token savings or "cut the waste"
    • do not mention agentop tools scan, --write, or turning on tool-prune
    • showing what an agent spends (tokens, cost) is fine: that is observation, not reduction
    • reference docs (laptop-token-savings.md, tool-prune-plugin.md, the agentop reference, "How it works" sections) keep their content without changes

Activity

  1. self-assigned this
    on Nov 17, 2026
  2. huang195 commented on Nov 18, 2026

    @huang195
    MemberAuthor

    Done in #1945 and #1947. The acceptance items pass on main: the link checker and its --selftest pass, the pre-commit hook runs it, the moved sections are only in docs/troubleshooting.md, and setup ends with configure … enable and agentop observe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentation

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions