Stop logging phantom 500s and cap the catalog refresh retries - #3
Merged
Conversation
Two leftovers from the traffic rate-limit incident.
The 123 "500" responses on /u/{login}/all were never server errors. Inertia
executes the root template straight into the response writer, so when a
client aborts mid-page the write fails with the status already committed.
The handler then called http.Error, which net/http rejects as a superfluous
WriteHeader, logged a phantom 500, and appended error text to the partial
HTML. Rendering cannot fail on data here: the template only interpolates two
pre-rendered template.HTML values, so every one of those failures was a
write error. The handler now tracks whether the response started and logs
the failure instead of rewriting the status when it did.
PerformRefreshCatalog still had the old shape: it walked the entire watch
list after an error and let a rate limit fail the job outright, so the
worker burned its 3 attempts in seconds against an hour-long window. It now
stops at the first error and reuses the shared rate-limit policy.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problema 1 — os 123 "500" em
/u/{login}/allnunca foram erro de servidorOs logs de produção mostravam 123 respostas
500(todas nessa rota, a mais pesada), sempre acompanhadas de:Investigando o caminho de render:
gonertiaexecuta o root template direto noResponseWriter(i.rootTemplate.Execute(w, templateData)), esetHTMLResponsesó seta oContent-Type— o status é gravado implicitamente no primeiro write.Executeretorna erro com o status já comprometido.http.Error, que onet/httprejeita comoWriteHeadersupérfluo: loga um 500 fantasma e ainda concatena o texto do erro no HTML parcial.Não é erro de dado. O
app.htmlsó interpola{{ .inertia }}e{{ .inertiaHead }}, ambostemplate.HTMLpré-renderizados — o template não tem como falhar na execução. Logo, todo erro ali é erro de escrita. Corroborando: os mesmos IPs aparecem com 200 e 500 na mesma rota, e os 500 terminam antes (mediana 3,97s) que os 200 (6,23s) — assinatura de cliente que desiste no meio.Correção: o handler rastreia se a resposta já começou; se começou, loga o erro em vez de reescrever o status.
Problema 2 —
PerformRefreshCatalogcom o mesmo padrão de retryAgora para no primeiro erro e reusa a política compartilhada de rate limit (
skipRateLimited, extraída parainternal/jobs/rate_limit.go).Testes (TDD, escritos antes)
TestHomeHandler_AbortedClientDoesNotRewriteHeader— com o código antigo falhava comstatuses = [200 500], reproduzindo exatamente osuperfluousde produção.TestPerformRefreshCatalog_StopsOnFirstError— 3 logins falhando geravam 3 chamadas; agora 1.TestPerformRefreshCatalog_RateLimitedIsSkipped— rate limit não marca o job como failed.make cilocal: race tests, golangci-lint (0 issues) e Prettier verdes.