Repository navigation
perf(observability): cache the complete-batch endpoint tail #274
Description
Activity
Implemented: endpoint reads merge completed-batch minute histograms, exact partial-minute projections, and uncached active files. Removed the endpoint watermark/readiness/disabled pipeline; service/edge safety lag is unchanged. Contributions and markers commit atomically and are read in one pinned transaction. Tests cover cold/warm parity, late publication, rollback, compaction, retention, and prompt background draining. On identical 1.2M-row files, broad endpoint median improves 47.84 to 31.01ms; narrow windows still pay extra planning cost (14.64 to 23.76ms), explicitly documented.
Implemented in ea13675; tracked in #274. Measurement report and reproduction.
just check, the full Go race suite, and native Linux ARM64 full Go tests pass. Issue closure is linked to merge of #272.
PR #272 review follow-up for
performance.go: #272 (comment)Stop recomputing the five-minute raw endpoint histogram for every dashboard read. Preserve the existing publication safety lag. Cache complete immutable batch aggregates by minute and combine them with exact partial-minute boundaries and uncached batches; invalidate correctly for compaction, retention, and late arrivals.
Validation: endpoint calls, errors, histogram buckets/quantiles, namespace/service filters, partial minutes, startup, new and late batches, replacement, and cancellation must match authoritative scans. Measure repeated queries at a fixed ingest rate; cached reads must not scan the entire safety-lag region.
Implementation is authorized in PR #272; this issue tracks the fix and its validation.