Repository navigation
Measure Homepage probes after the chart health-check fix #5349
Description
🤖 Generated by the Agentic Engineer
Objective
Decide which Homepage probe overrides remain necessary after chart 4.14.0 switches liveness and readiness from the rendered page to /api/healthcheck.
Details
Release review in #5341 found that chart 4.14.0 supplies a dedicated health endpoint. Our HelmRelease still carries startup timing and relaxed probe settings introduced around slow page rendering; it also records a subsequently corrected CPU bottleneck. The endpoint change may remove some need for those overrides, but it does not prove cold-start behavior or justify removing safeguards without measurements.
Timebox: two engineering hours, including the existing protected deployment and verification path. Start after #5341 is delivered. Compare the current configuration with a candidate that uses the chart's health endpoint consistently, preserving the readiness signal and rollback path.
Acceptance criteria
- Record baseline and candidate startup latency, probe failure count, restart count and public availability through cold start and steady state.
- Demonstrate that an unhealthy application fails its probe; a faster response must not mask application failure.
- Record a decision on each local startup, timeout and failure-threshold override using before/after evidence.
- File any resulting implementation as a separate linked issue, or record why the overrides should remain. A decision completes this Spike; it does not require a code PR.
Planning rationale
Low urgency: availability is currently protected and the chart update can proceed with the existing safeguards. The outcome is simpler health checks with evidence that failures remain visible. The investigation is bounded to two hours; this is Backlog refinement, not an incident or an already measured improvement.
Parent: #4981. Evidence: the exact old/new published chart archives and rendered manifests reviewed in #5341.
Activity
devantler commentedon Nov 4, 2026
Status 2026-11-04: still open. A read-only pass can record the baseline, but the candidate measurements and the failing-probe demonstration require deploying a candidate and injecting a fault. Both are changes to the cluster, so they were not made here, and no candidate number has been estimated.
Baseline (current production configuration, chart 5.9.0, Homepage v2.7.0):
//api/healthcheck/api/healthcheckA finding for the second criterion: in v2.7.0,
/api/healthcheckanswersupwithout checking anything beyond the allowed host. It catches a hung or dead server process, but not a broken config or a page that fails to render. That is the "faster response masking failure" risk the criterion names, and the fault-injection step should test it specifically.What remains: deploy a candidate that uses the health endpoint for startup as well, through the protected path. Then record the same four measures through a cold start and steady state, inject a failure to show the probe fails, and decide on each override. Also note #5876: every replica restart changes Homepage's config hash, so rollouts during the measurement will trigger dashboard reloads that are unrelated to the probes.
🤖 Generated with Claude Code