Backport(v1.19): plugin base: bound the number of worker lock files with hashed buckets (#5470)

[github-actions]

Which issue(s) this PR fixes:
Backport #5470
Fixes #

What this PR does / why we need it:
When workers > 1, out_file (with append) and out_secondary_file take an inter-worker lock per output path, and get_lock_path derives one lock file per path: /tmp/fluentd-lock-*/fluentd-<sanitized path>.lock. A lock file is never removed while fluentd is running; the only cleanup is cleanup_lock_dir at a graceful shutdown.

So the number of lock files grows with the number of unique output paths, and with a date or a tag placeholder in path that set is effectively unbounded over time.

This PR folds the paths into a fixed pool of buckets:

LOCK_FILE_BUCKETS = 65536

def get_lock_path(name)
  bucket = Zlib.crc32(name.to_s) % LOCK_FILE_BUCKETS
  File.join(@fluentd_lock_dir, "fluentd-bucket-#{bucket}.lock")
end

The lock is a flock, which is bound to the open file description and released when the file is closed or the process dies. The lock file is not state; it is only an anchor that the workers rendezvous on by name. Mutual exclusion therefore does not need a 1:1 correspondence between a path and a file. It needs exactly one property:

the same path always maps to the same file, in every worker process.

crc32 % 65536 preserves that property, so any two workers writing the same path still contend on the same file and still serialize. acquire_worker_lock itself (open / flock / touch) is untouched.

What changes is that the number of lock files is now bounded by a constant instead of by the number of unique paths. The accumulation, the mass deletion at shutdown, and the dependence on an external tmp cleaner all disappear structurally rather than being mitigated.

Collision semantics

Two unrelated paths can land in the same bucket. The effect on correctness is zero: they are serialized against each other, which is exclusion that is wider than necessary, never exclusion that is lost.

The effect on throughput is a rare latency coupling, and it is worth being explicit about it because flock is held for the whole chunk write, not just for opening the file. With N = 65536 buckets and k locks held concurrently across all workers, the probability that some pair of them collides is about 1 - exp(-k(k-1)/2N):

concurrent locks k	some pair collides	a given flush collides with another
8	0.043%	0.011%
32	0.75%	0.047%
64	3.0%	0.096%
120	10.3%	0.18%

The right-hand column is the one a single flush actually experiences, and it stays below 0.2% even at 120 concurrent locks. A collision costs the duration of one chunk write, and only for the two paths involved.

Docs Changes:
N/A

Release Note:

• plugin base: bound the number of worker lock files by hashing the path into a fixed set of buckets