Skip to content

Feature offers from a consumer quota layer: verdict ladder, multi-account recovery, depletion projection, history — disposition wanted #2

Description

@iceteaSA

Feature offers from our quota layer — want any of these upstream?

Follow-up to #1 (gap report). That issue covers bugs/parity; this one covers quota features we carry consumer-side that may belong in insula. Before entrenching them further in our stack, we want upstream disposition. Each item gives what it does, why it exists, and its possible insula shape. Disposition: insula / elsewhere in the fleet / keep it consumer-side.

F-A · Verdict ladder (allow / downgrade / block per window)

We derive a three-state routing verdict from each window: allow (reserve or on-pace), downgrade (past pacing thresholds — route heavy work elsewhere), block (window effectively unusable now). A pre-dispatch gate in our harness denies dispatches onto fresh block lanes and suggests healthy alternatives.

  • Why: raw used% pushes every consumer to reinvent thresholds; ours took three iterations to stop misfiring (pacing-vs-reset math on regen windows was the last bug — see Linux gap report: credential discovery misses + provider behavior (12 triage items, live daemon evidence) #1 FEAT-1).
  • In insula this would be: an optional derived field per window (or a separate usage.verdict op) so routers share one interpretation. Fleet-map says routing is Alfonso's job — if verdicts belong there or in astrocyte's admission model instead, that answers this item too.
  • We keep local either way: our roster-policy bans (model X may not fill role Y) — those are ours, not quota.

F-B · Multi-account recovery semantics

When one account's window blocks, we soften the verdict to downgrade if another enabled account of the same provider has headroom, or if the active account has reset-credits available to spend (codex). The router then rotates instead of abandoning the provider.

  • Why: with 2-3 accounts per provider, per-account verdicts without a provider-level rollup made the router abandon providers that still had capacity.
  • In insula: a provider-level aggregate entry (or a documented convention for consumers to fold accounts). Depends on F-A landing somewhere.

F-C · Time-to-depletion projection

Per window: burn rate over the recent window vs remaining capacity → projected depletion time vs reset (or vs regen inflow for continuous-regen providers). Ours renders as deficit warnings ("Gemini 7d out in 1d12h at current pace").

  • Why: used% alone flags nothing until it's too late to shift a batch; the projection is what actually changes routing decisions mid-day.
  • In insula: needs short-horizon history (see F-E) or at least a used%-delta between refresher ticks. Could also live consumer-side forever — cheap to compute if F-E exists.

F-D · Credit-pool backstop semantics

For providers with prepaid credit pools behind plan windows (MiniMax today): a depleted plan window with a non-expired credit balance ≥ threshold is downgrade, not block — the pool is real headroom the provider debits automatically.

F-E · Short-horizon window history

Last-N samples per (provider, account, window) — even a small in-memory ring buffer served alongside usage.get — enables F-C, burn-vs-window regression (quota-weighting of cache tokens is unpublished and empirically fittable), and post-hoc "which task burned this window" attribution.

  • Why: we currently reconstruct history from prose snapshots persisted in our harness DB; it works but is parsing-hostile and harness-bound.
  • Placement question: DURABLE history belongs in astrocyte (its refresher already polls usage.get; we planned to propose a window_sample table there), and insula stays current-state. If you agree, F-E collapses to "astrocyte proposal, insula unchanged" — flagging it here so the two issues don't fork the same design.

F-F · Operator kill/enable flags respected in served entries

Consumer-side we let the operator mark an account killed/disabled and our tooling stops routing to it even when its window is healthy. #1 FEAT-5 asks for the flags to be surfaced; this asks the follow-up: should insula filter or annotate entries for operator-disabled accounts, or is that consumer policy?


Ready to PR any of these where wanted (TS reference implementations exist for all six, with tests; the verdict ladder + pacing math have ~6 weeks of production use across ~9 providers). Equally fine keeping them consumer-side — the ask is just a disposition per item so we invest in the right place.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions