[pull] main from danny-avila:main - #209
Merged
Merged
Conversation
#497) * 🧊 fix: Keep Historical Tool Result Truncation Byte-Stable Across Turns Pre-flight truncation and observation masking recomputed every historical tool result's character cap on every call from quantities that move from turn to turn: the number of tool results (`0.2 + 0.8 * t / (n - 1)` recency weighting and per-count budget shares), the calibration ratio (`effectiveMaxTokens / calibrationRatio`) and the instruction overhead. Once a conversation crossed the fading threshold — or, with summarization enabled, from the first oversized result onward — the same stored tool output was truncated to a slightly different length on nearly every request. Anthropic's prompt cache is prefix-based, so each such change invalidated the cached prefix for everything after it and forced a full cache write per turn for the rest of the conversation's life. - Derive every fading cap from the fixed context window scaled by the discrete pressure band (`resolveFadingBudgetTokens`), independent of calibration, instruction overhead and tool-result count - Apply one flat cap per pre-flight pass instead of position weighting - Mask consumed results to a fixed fraction of the fresh-result cap (`calculateMaskedResultMaxChars`) instead of count-weighted budget shares - Cap fit-to-budget truncation (summarization on) at the window-derived ingestion cap rather than the calibrated effective budget - Add stability tests covering position drift, count drift and calibration drift through `createPruneMessages`, within a run and across runs Within a pressure band a historical tool result now maps to identical bytes on every call; only band transitions (at most three per conversation) and summarization rewrite the prefix. Fixes danny-avila/LibreChat#15499 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 refactor: Latch Context Fading on a Quantized Cap Ladder Review of the first pass found the window-derived flat cap dropped the fit guarantee: with summarization on, a tool result under the ingestion cap but over the effective budget was no longer pre-truncated, so a small window with large instruction overhead yielded an empty first context and sent everything to the summarizer after one tool call. The fallback and emergency paths also still derived caps from `effectiveMaxTokens`, and band selection had no hysteresis, so pressure near a threshold flipped bands between runs and rewrote every historical result each flip. Replace the per-call cap derivations with one latched tier: - Add `messages/fading.ts`: a pure cap ladder that halves the window per rung. `resolveFadingTier` deepens the rung to the shallowest one whose fresh-result cap fits half the effective budget (the fit guarantee), adds pressure-band rungs when summarization is off, and latches masking at 80 %. The tier only ever deepens, which is the hysteresis - Derive every cap (`resolveFadingCaps`) from `(window, rung, masked)` alone, so a historical tool result maps to identical bytes on every call; only escalation rewrites the prefix - Apply caps in one pass (`applyFadingCaps`) with watermarks, so an unchanged tier only visits messages that arrived since the last call; masking and pre-flight truncation become thin wrappers over it - Drop fallback fading, subsumed by the fit rung, and make emergency truncation deepen the latched tier on the live messages instead of re-deriving count-proportional caps on a clone each call - Thread `fadingTier` through `createPruneMessages`, `AgentContext`, `StandardGraph` and `Run` (`RunConfig.fadingTier`, `Run.getFadingTier()`) so hosts can persist it beside `calibrationRatio` and seed the next run; an invalid or foreign-window tier starts fresh - Move `calculateMaxToolCallInputChars` next to its sibling in `utils/truncation.ts` and re-export it from `messages/prune` Tests cover the ladder, band hysteresis, tier validation and seeding, the fit guarantee on a 32k window with 21k of instructions, byte stability across runs with and without a seeded tier, and within a run under recalibration. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Persist Fading Budget, Keep Emergency Truncation Transient Address the second review pass on the latched fading tier: - Persist an absolute `budgetTokens` instead of `(window, rung)`. A tier keyed by window was discarded by a mid-run budget correction and again on the next run's return to the normal window, rewriting the prefix twice; the budget is now only clamped to the current window and never grows, so both transitions keep the same bytes - Make emergency truncation transient again. It derives a temporary deeper tier from a per-message share of the effective budget and applies it to a clone, restoring token counts afterwards; latching that count-dependent share pinned every future result to one transient event - Reset `messagesToRefine` to the retry outcome after an emergency recovery, so messages in the recovered context are not also summarized - Share `hasNonEmptyTextContent` from `messages/core` between the consumed boundary scan and the preempt gate instead of a second copy - Update `docs/summarization-behavior.md` for the tier, the fit rung, the band rungs, the removed fallback stage and the transient emergency path Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Expose Only Informative Fading Tiers to Hosts `Run.getFadingTier()` returned the fresh tier too, so a host comparing it against its own context window (which can differ from the pruner's after the reserve ratio is applied) misclassified an untouched conversation as faded and persisted a tier that would pin later runs to a smaller budget. Decide it where the pruner's window is known: `Graph.getFadingTier()` now returns the tier only once masking has activated or the budget sits below `maxContextTokens`, via `isInformativeFadingTier`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * fix: Preserve fading state across agent boundaries * fix: Rebuild fading inputs from canonical calls * 🧊 test: Derive Emergency Caps From Canonical Stores, Cover Summarizer Snapshot The emergency pass now reads the same canonical stores as the tier pass so its clone derives from original bytes, and a test proves the summarizer snapshot captures the true original when a fresh-capped result is masked on a later call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * fix: Bound canonical fading state * fix: Clone fading tier snapshots * fix: Preserve fading state across sessions * fix: Bound fading state across histories * refactor: project fading from canonical graph history * fix: preserve projected fading state * perf: keep projection sync append-only * fix: invalidate rewritten provider projections * fix: invalidate projections from reducer updates * fix: preserve conservative counts on projection rebuild * fix: honor result caps during fading selection * fix: fit complete tool exchanges within fading budget * fix: bound parallel fading and session restore * fix: preserve fading across graph lifecycle edges * fix: retain canonical replacements and failed-run tiers * test: verify Anthropic fading across fresh runs * perf: update tool width sources incrementally * fix: close fading integration gaps * fix: align fading boundary state * fix: invalidate rewritten fading history * fix: normalize reducer update typing * fix: bound routing and preserve replay caches * fix: respect scoped fading budgets * fix: reset fading state with compacted history * test: measure scoped routing budget * fix: close fading reset lifecycle gaps * fix: preserve routing and tier edge cases * fix: reject stale fading captures * 🧊 fix: Keep Re-seeded Fresh Tiers Fresh, Validate Restored Fading State - `seedFadingTier` latches a seed only when it was tightened, masked, already latched, or clamped, so a fresh tier re-seeded through a pruner rebuild is never reported as informative and cannot pin later runs to the current window; a sub-floor budget clamps up to the ladder floor so every cap stays positive. - Restored tiers are validated at every boundary (`Run`, `StandardGraph`, the `AgentSession` merge and its store restore), per-agent tiers are read as own properties so an agent ID such as `constructor` persists, and a `session_state` line carrying null data no longer breaks `JsonlSessionStore`. - The provider projection's fallback rebuild discards the index-keyed original-content sidecar, so a shifted prefix cannot restore one tool's bytes onto another message before summarization. - Emergency truncation derives its rung from the whole tool exchange, so a small configured result cap cannot satisfy the target while call inputs stay uncapped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Bound Handoff Instructions, Isolate Fading Reset Generations - Handoff instructions are bounded by the receiving agent's routing cap before they become a HumanMessage, through the same destination-derived budget that direct-edge routing prompts use, since fading can never shrink a human message later. - `AgentSession` advances a rewrite epoch on branch, compact and restore, so a run that started before an explicit history rewrite cannot re-persist tiers learned on the pre-rewrite history when it completes. - Reset generations are tracked per scope and per tier (the default tier and each agent), so one agent's compaction rejects only that agent's stale captures and a concurrent, valid escalation of another agent still lands. - `hasNonEmptyTextContent` scans content blocks by index through own data descriptors and requires an array, so a hostile iterator or accessor-backed element can no longer abort pruning. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 📝 docs: Describe the Fit Rung, Canonical History and Emergency Exchange The fading section still described the fit rung as a single result fitting half the effective budget and the emergency tier as a per-message result share. The fit rung now sizes the widest observed parallel exchange (inputs plus results) against the effective budget, the emergency tier is the rung at which one complete exchange fits the per-message share, the tier shape carries its version, and hosts read compaction resets through `Run.didResetFadingTier()`. Also notes that graph history stays canonical behind a per-Run provider projection and that restored tiers are validated at every boundary.
* 🛡️ fix: Require Context Pressure for Numberless Overflow Recovery * fix: narrow ambiguous overflow guard * fix: carry completion pressure through fallbacks * fix: bound blind recovery by output reserve * fix: scope completion pressure to request semantics * fix: centralize provider window semantics * fix: prefer provider facts over ambiguous wrappers * fix: preserve overflow recovery evidence across retries * fix: keep overflow budgets provider scoped * fix: keep overflow pressure in provider token space * fix: reject ambiguous overflow recovery * fix: reject ambiguous loose overflow matches
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )