From 1cedc81345714ca6dbb29502f396617af206c978 Mon Sep 17 00:00:00 2001 From: Matt Godbolt Date: Sat, 10 Oct 2026 23:16:14 -0500 Subject: [PATCH] Cap explanation length by bullets and sentences, not words Output length is nearly all of the latency: a Sonnet 5.5 call takes about 0.4s plus 7.7ms per output token. The prompt asked for "under roughly 250 words" but explanations averaged 423 words on the hard corpus, so the limit was ignored. Replace it with a bullet and sentence cap, which Anthropic's latency guide recommends over word limits. On 2 runs of the full suite with the Opus 5 reviewer, mean latency fell 8.2s to 6.5s (p95 12.2s to 9.0s, max 17.7s to 9.9s) and output tokens 1002 to 791, with no detectable accuracy change (87.8% vs 88.3% correct over 188 reviews each). Recorded in CLAUDE.md. Co-Authored-By: Claude Sonnet 5.5 --- CLAUDE.md | 7 +++++++ app/prompt.yaml | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index c36d4cb..e5868d5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -95,6 +95,13 @@ Pre-commit hooks may modify files (e.g. ruff format); re-`git add` if a hook rep the CE frontend can render. - **`build_api_payload` is the single source of truth for API kwargs.** Production (`app/explain.py`) and the prompt-test runner both call it; don't reconstruct thinking/temperature/output_config logic elsewhere. +- **Latency is output length.** A Sonnet 5.5 call takes about 0.4s plus 7.7ms per output token (~130 tok/s), so + shorter answers are the main speed lever; input size barely matters (cutting input tokens 35% did not move + latency). The old "under roughly 250 words" limit was ignored (mean 423 words on the hard corpus). A + sentence/bullet cap ("at most 6 bullets or short paragraphs, none longer than two sentences, no headings") cut + mean latency 8.2s to 6.5s (p95 12.2s to 9.0s, max 17.7s to 9.9s) with no detectable change in accuracy (87.8% + vs 88.3% correct over 188 reviews each, across both corpora). Anthropic's latency guide says the same: word + limits work worse than sentence or paragraph limits. ## Code Style Guidelines diff --git a/app/prompt.yaml b/app/prompt.yaml index 1858225..e9f1777 100644 --- a/app/prompt.yaml +++ b/app/prompt.yaml @@ -57,7 +57,7 @@ system_prompt: | - **Be accurate.** Before explaining any instruction, trace its inputs and outputs step-by-step. Be especially careful with multi-operand instructions (e.g., verify whether `lea` performs address calculation vs memory access, check `imul` operand order). - **Be definitive about what you can observe** (instruction behaviour, register usage, memory operations). Be appropriately cautious about inferred purposes or design decisions. - - **Be concise.** Don't explain what the source code does — the user wrote it. Reference source only to clarify the assembly mapping. Skip non-essential context, keep examples minimal, and stop once the essential points are covered; when in doubt, cut. Cover the few points that matter rather than every instruction. Keep the whole explanation under roughly 250 words; only exceed that when the assembly is unusually long or complex, and even then stay brief. + - **Be concise.** Don't explain what the source code does — the user wrote it. Reference source only to clarify the assembly mapping. Skip non-essential context, keep examples minimal, and stop once the essential points are covered; when in doubt, cut. Cover the few points that matter rather than every instruction. Hard limit: at most 6 bullets or short paragraphs in total, none longer than two sentences, and no headings. Only when the assembly is unusually long or complex may you use up to 10. Lead with the single most important insight, and omit anything the reader could infer from the instruction names alone. - **Characterise optimisation levels accurately.** If compilation options are empty or contain no optimisation flags, this is definitively unoptimised code — state it confidently, never say "likely -O0" or "appears to be". When explicit flags are present, reference them directly. - **Handle undefined behaviour.** When code contains UB, explain that the compiler was free to choose any implementation. Describe the result as "one possible implementation" and explain why the code is problematic. - **Use qualified language for performance claims** ('typically', 'on most modern processors') — avoid absolute claims about branch prediction, cache performance, or pipelining unless verifiable for the specific architecture.