feat(client): preserve paired routing weights for Total Router Recall

[faresobeid]

Summary

• Preserve paired expert IDs and optional FP32 coefficients as raw-response-backed memoryviews in compact /inference/v1/generate responses. Keep metadata and legacy IDs-only payloads unchanged.
• Preserve JSON decoder behavior for other layouts, escaped strings, duplicate keys, unrelated fields, and malformed input. Validate buffer placement before returning the fast path.
• Add exact-byte, ownership, fallback, and client passthrough tests. Preserve current main's usage metadata, including nested usage details alongside paired routing.

Scope and companions

This is the renderer transport portion of the experimental Total Router Recall MVP, not router capture or trainer replay. Semantic routing validation belongs downstream. Full-coefficient replay remains single-turn and non-forked; the companion verifier path rejects reused-prefix continuation/forks. This parser does not enforce that graph guard or add multi-turn support. Legacy IDs-only behavior remains supported.

• Companion verifiers draft: feat(v1): carry paired routing weights in single-turn traces verifiers#2568
• Companion prime-rl draft: feat(rl): add experimental Total Router Recall prime-rl#3509
• vLLM coefficient capture is prepared separately; upstream submission is held pending the required human review, testing and DCO sign-off.

Related #79 transports Dynamo expert IDs, not routing coefficients. This change does not add Dynamo support.

Validation

Fresh main: 06bcf635f4b582216fbb225eb4ed2e5c84fad7fe.

• uv run --frozen pytest tests/test_client_routing.py tests/test_client.py -q -rs: 78 passed, 5 skipped. The skips are existing optional vLLM feature-serialization tests; vLLM is not installed in this renderer environment. Normal conftest loading succeeded.
• Changed-file uv run --frozen ruff check and uv run --frozen ruff format --check: passed. git diff --cached --check: passed.
• Prior implementation evidence was 80 passed in a different shared environment with --noconftest; it was not rerun here. This revision collects 83 cases, including current main's extra usage case and two added paired-usage cases.

These are focused CPU transport tests. No full renderer suite, live server, GPU, end-to-end numerical parity, performance, or training-quality qualification is claimed.

AI assistance

AI assistance was used to prepare and review this patch, add regression coverage, run checks, and draft this description.

Note

Preserve paired routing weights as raw-backed memoryviews in parse_generate_response

• The response parser now locates compact routed-expert ID and optional weight streams by prefix, validates unescaped base64 content, and returns read-only memoryviews over the original response bytes without copying.
• Recognized streams are temporarily replaced with numeric markers and a custom float parser binds those markers to the views; the parser verifies the views belong to the first choice's routed-experts object.
• Falls back to ordinary JSON decoding when the compact prefix matches an unrelated field, duplicate keys survive, or the structure is incompatible; escaped base64, empty streams, and reserved marker collisions in metadata are all handled by this fallback.
• Adds fixture helpers and extensive parameterized tests covering fast-path correctness, fallback layouts, malformed/truncated responses, duplicate-key semantics, buffer-association safety, and zero-copy allocation.
• Risk: parse_generate_response now returns memoryview objects (not bytes or str) for compact routing ID and weight fields in client.py; in-tree consumers are updated, but out-of-tree callers expecting concrete bytes will need to adapt.

^{Macroscope summarized fa5624c.}