Seed zero-value metrics for empty buckets in count-items#378
Conversation
Hello delthas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## development/1 #378 +/- ##
=================================================
+ Coverage 45.47% 45.62% +0.15%
=================================================
Files 88 88
Lines 6529 6544 +15
Branches 1372 1376 +4
=================================================
+ Hits 2969 2986 +17
+ Misses 3510 3508 -2
Partials 50 50 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
c954e09 to
ef5a150
Compare
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
|
|
LGTM |
ef5a150 to
6442f99
Compare
cf6258d to
61c1dca
Compare
count-items only produced metric documents for buckets that hold objects. An empty bucket yielded no document in __infostore, which makes Scuba's deep health check fail and disables the quota service cluster-wide (ARTESCA-17063). Seed zero-value bucket, account and location entries in getObjectMDStats() when a bucket scan yields no objects. The bucket key is derived from the __usersbucket creation date, exactly like populated buckets (S3UTILS-131), so the seeded document matches the key quota lookups use; if that date is missing the bucket is skipped with a warning rather than written under an unmatchable key. Issue: S3UTILS-224
61c1dca to
9d957f3
Compare
| * @returns{boolean} true when zero metrics were seeded, false otherwise | ||
| */ | ||
| _seedEmptyBucketMetrics(collRes, bucketResource, bucketInfo, locationConfig, log) { | ||
| if (Object.keys(collRes.bucket).length > 0) { |
There was a problem hiding this comment.
object.keys(collRes.bucket).length === 0 does not necessarily mean the bucket is empty. It also happens when every entry failed processing, for example because of invalid content length or missing location configuration. We would then publish zero capacity for a non-empty bucket and potentially allow quota violations. can we track processing errors and only seed when the scan completed successfully without producing entries? we should also add a test for this case...
count-items only produced metric documents for buckets that hold objects. An empty bucket yielded no document in
__infostore, so Scuba's deep health check (getAnyMetric) found nothing and returned 500, causing CloudServer to disable the quota service cluster-wide (ARTESCA-17063).This seeds zero-value entries for the empty bucket, its account and its location in
getObjectMDStats()when a bucket scan yields no objects, so every known resource always has a document in__infostore.The bucket key is derived from the
__usersbucketcreation date, exactly like populated buckets (S3UTILS-131), so the seeded document matches the key quota lookups use. If that date is missing, the bucket is skipped with a warning rather than written under a key that lookups could never match.Note: this addresses only the empty-bucket case. The new-bucket window (before the first count-items run) and the Scuba deep-health-check / alert behavior are tracked separately.
Issue: S3UTILS-224