Skip to content

feat(strategy): ecr request metrics - #39

Merged
ravi-pplx merged 2 commits into
pplx/mainfrom
ravi/ecr-metrics
Nov 7, 2026
Merged

ravi-pplx merged 2 commits into
pplx/mainfrom
ravi/ecr-metrics

Conversation

@ravi-pplx

@ravi-pplx ravi-pplx commented Nov 7, 2026 •

Copy link
Copy Markdown

The ecr strategy has no metrics, so a rollout can't show how often manifests come from cache, how much blob traffic is redirected to S3, or why requests fail. Some failures are invisible without them: a manifest that fails digest verification has already answered 200, and the client only sees a short body.

This adds three counters, documented in the README:

  • cachew.ecr.requests_total{kind, method, result}: kind is manifest, blob, or unknown for a path that names neither. result is hit or miss for manifests, redirect for blob GETs, metadata for blob HEADs, not_found, rate_limited, not_served (paths or repositories the proxy refuses), or failure.
  • cachew.ecr.failures_total{kind, reason}: digest_mismatch, no_length, partial, rejected (ECR refused the proxy's token), authorization (no token could be minted or refreshed), bad_redirect, upstream_status, internal, or client_canceled when the client left first. Connection failures use the transport_* statuses the CodeArtifact metrics already report; the classifier moves to origin_transport.go as originTransportStatus so both strategies share it.
  • cachew.ecr.auth_events_total{event}: reuse, refresh, forced_refresh and failure, the same events the codeartifact strategy records.

Every label value comes from a fixed set. Repository names and digests are never labels.

Each request carries a small observation in its context. Whichever layer finds the cause records it, and the first one recorded wins: serveBlob, the error writer, the transport, or the digest verifier in the handler's fill goroutine. When the request finishes, the outcome is classified from that cause plus the response status and the X-Cachew-Result header. The response wrapper that already hid the cache's expiry header (renamed to ecrResponse) now also keeps the status. The /v2/ version check isn't counted. Metrics are recorded with an uncancelable context, as in the CodeArtifact strategy.

Tests use a recording fake, as the codeartifact metrics tests do. They cover each result and every failure reason except internal through the real handler and transport.

@ravi-pplx
ravi-pplx force-pushed the ravi/origin-token-rename branch from cf74283 to 46d68b1 Compare November 7, 2026 07:48
@ravi-pplx
ravi-pplx deleted the branch ravi/origin-token-rename November 7, 2026 14:29
@ravi-pplx ravi-pplx closed this Nov 7, 2026
Count ECR requests by kind, method and result (cache hit or miss, blob
redirect, blob metadata, not found, rate limited, not served, failure),
failures by reason, and authorization token events. Labels come from fixed
sets; repository names and digests are never labels.
@ravi-pplx ravi-pplx reopened this Nov 7, 2026
@ravi-pplx
ravi-pplx changed the base branch from ravi/origin-token-rename to pplx/main November 7, 2026 14:30
Label transport failures with the statuses the codeartifact metrics use,
through a shared originTransportStatus, and count a client that left first
as client_canceled rather than a proxy failure. Cache errors on the manifest
GET and HEAD paths both report internal. Record with an uncancelable context,
as the codeartifact metrics do.
@ravi-pplx
ravi-pplx marked this pull request as ready for review November 7, 2026 18:37
@ravi-pplx ravi-pplx assigned Copilot and unassigned Copilot Nov 7, 2026
@ravi-pplx
ravi-pplx merged commit ba36053 into pplx/main Nov 7, 2026
5 checks passed
@ravi-pplx
ravi-pplx deleted the ravi/ecr-metrics branch November 7, 2026 18:38
ravi-pplx added a commit that referenced this pull request Nov 9, 2026
The `ecr` strategy has no metrics, so a rollout can't show how often manifests come from cache, how much blob traffic is redirected to S3, or why requests fail. Some failures are invisible without them: a manifest that fails digest verification has already answered `200`, and the client only sees a short body.

This adds three counters, documented in the README:

- `cachew.ecr.requests_total{kind, method, result}`: `kind` is `manifest`, `blob`, or `unknown` for a path that names neither. `result` is `hit` or `miss` for manifests, `redirect` for blob `GET`s, `metadata` for blob `HEAD`s, `not_found`, `rate_limited`, `not_served` (paths or repositories the proxy refuses), or `failure`.
- `cachew.ecr.failures_total{kind, reason}`: `digest_mismatch`, `no_length`, `partial`, `rejected` (ECR refused the proxy's token), `authorization` (no token could be minted or refreshed), `bad_redirect`, `upstream_status`, `internal`, or `client_canceled` when the client left first. Connection failures use the `transport_*` statuses the CodeArtifact metrics already report; the classifier moves to `origin_transport.go` as `originTransportStatus` so both strategies share it.
- `cachew.ecr.auth_events_total{event}`: `reuse`, `refresh`, `forced_refresh` and `failure`, the same events the `codeartifact` strategy records.

Every label value comes from a fixed set. Repository names and digests are never labels.

Each request carries a small observation in its context. Whichever layer finds the cause records it, and the first one recorded wins: `serveBlob`, the error writer, the transport, or the digest verifier in the handler's fill goroutine. When the request finishes, the outcome is classified from that cause plus the response status and the `X-Cachew-Result` header. The response wrapper that already hid the cache's expiry header (renamed to `ecrResponse`) now also keeps the status. The `/v2/` version check isn't counted. Metrics are recorded with an uncancelable context, as in the CodeArtifact strategy.

Tests use a recording fake, as the `codeartifact` metrics tests do. They cover each result and every failure reason except `internal` through the real handler and transport.
ravi-pplx added a commit that referenced this pull request Nov 12, 2026
The `ecr` strategy has no metrics, so a rollout can't show how often manifests come from cache, how much blob traffic is redirected to S3, or why requests fail. Some failures are invisible without them: a manifest that fails digest verification has already answered `200`, and the client only sees a short body.

This adds three counters, documented in the README:

- `cachew.ecr.requests_total{kind, method, result}`: `kind` is `manifest`, `blob`, or `unknown` for a path that names neither. `result` is `hit` or `miss` for manifests, `redirect` for blob `GET`s, `metadata` for blob `HEAD`s, `not_found`, `rate_limited`, `not_served` (paths or repositories the proxy refuses), or `failure`.
- `cachew.ecr.failures_total{kind, reason}`: `digest_mismatch`, `no_length`, `partial`, `rejected` (ECR refused the proxy's token), `authorization` (no token could be minted or refreshed), `bad_redirect`, `upstream_status`, `internal`, or `client_canceled` when the client left first. Connection failures use the `transport_*` statuses the CodeArtifact metrics already report; the classifier moves to `origin_transport.go` as `originTransportStatus` so both strategies share it.
- `cachew.ecr.auth_events_total{event}`: `reuse`, `refresh`, `forced_refresh` and `failure`, the same events the `codeartifact` strategy records.

Every label value comes from a fixed set. Repository names and digests are never labels.

Each request carries a small observation in its context. Whichever layer finds the cause records it, and the first one recorded wins: `serveBlob`, the error writer, the transport, or the digest verifier in the handler's fill goroutine. When the request finishes, the outcome is classified from that cause plus the response status and the `X-Cachew-Result` header. The response wrapper that already hid the cache's expiry header (renamed to `ecrResponse`) now also keeps the status. The `/v2/` version check isn't counted. Metrics are recorded with an uncancelable context, as in the CodeArtifact strategy.

Tests use a recording fake, as the `codeartifact` metrics tests do. They cover each result and every failure reason except `internal` through the real handler and transport.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants