Repository navigation
event-exporter-rs (3/3): add streaming K8s WatchList, binary entrypoint, Dockerfile, and E2E tests - #1305
Draft
zxqlxy wants to merge 3 commits into
Draft
event-exporter-rs (3/3): add streaming K8s WatchList, binary entrypoint, Dockerfile, and E2E tests#1305zxqlxy wants to merge 3 commits into
zxqlxy wants to merge 3 commits into
Conversation
…nricher, and metrics Part 1 of 3 introducing `event-exporter-rs`, a `#![forbid(unsafe_code)]` pure-Rust Kubernetes Event exporter for GKE: - `src/cli.rs` & `src/shard.rs`: Drop-in parser for GKE single-dash Go flags (`-sink-opts`, `-universeDomain`, `-system-namespaces`, `-event-label-selector`) plus KEP-5866 `--shard-mode=server` and Jump Consistent Hash `--shard-mode=client` Pod/Event co-partitioning. - `src/pod_cache.rs`: Lock-sharded open-addressed 24-byte/slot `PodOwnerTable` (`16B PodKey` + `8B CompactOwner`) resolving `Deployment`, `StatefulSet`, `DaemonSet`, `CronJob`, `Job`, and `JobSet` (`restart-attempt` + `jobset-uid`) owners while holding 1,000,000 active Pods in ~50.4 MiB RSS delta (~58.8 MiB total process RSS vs. >1.2 GiB in Go `client-go`). - `src/enricher.rs` & `src/metrics.rs`: Cloud Logging `k8s_pod`/`k8s_node`/`k8s_cluster` routing, cross-namespace RBAC spoofing guard, `fieldBlacklist` filtering, Gregorian RFC3339 validation, and 100% LOrIS Prometheus metric compatibility. - Formal state-machine refinement, sharding bijection, and 1,000,000-pod memory benchmark suites, wired into GitHub Actions (`ci-rust-event-exporter`).
… pipeline, and dual sinks Part 2 of 3 adding networking, GKE credential rotation, bounded batching, and Cloud Logging v2 + OpenTelemetry OTLP export sinks: - `src/transport.rs` & `src/otlp_proto.rs`: Pure-Rust `reqwest` + `rustls` HTTP/2 client (honoring GKE private-node `HTTP_PROXY`/`NO_PROXY` exemptions for `169.254.0.0/16` and `metadata.google.internal`) and `tonic`/`prost` OTLP `LogsService/Export` gRPC client. - `src/auth.rs`: GKE `NodeTokenBroker` external-account STS exchange (`GOOGLE_APPLICATION_CREDENTIALS`), direct token-broker URL support, GCE Metadata Server fallback, K8s SA bearer rotation, and `--universe-domain` endpoint resolution. - `src/sink.rs` & `src/pipeline.rs`: Byte-bounded (`16 MiB` default) micro-batch export queue with retry/jitter backoff, gzipped Cloud Logging `/v2/entries:write` JSON sink, and OTLP gRPC/HTTP sink preserving LOrIS status & freshness metrics.
…ckerfile, and E2E tests Part 3 of 3 completing the standalone `event-exporter` binary, `distroless/static` container build, and live multi-shard E2E test suites: - `src/k8s_stream.rs`: Streaming `PartialObjectMetadata` Pod reflector with explicit `k8s.io/initial-events-end` bookmark gate, automatic HTTP 422 paginated `List` fallback, Go `"items": null` empty-list handling, and line-by-line Event watcher with `410 Gone` watermark recovery. - `src/main.rs`, `Dockerfile`, `Makefile`: Multi-arch (`linux/amd64`, `linux/arm64`) statically linked musl build (`+crt-static`) onto `gcr.io/distroless/static:latest` (`USER 65532:65532`), wired into `ci-container-build` and Dependabot. - Subprocess CLI, WatchList premature-EOF regression, `410 Gone` recovery, and 3-shard OTLP + HTTP 429 backpressure E2E test suites.
zxqlxy
force-pushed
the
event-exporter-rs-pr3
branch
from
November 11, 2026 02:35
01e2385 to
424434f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary (Part 3 of 3)
Completes
event-exporter-rswith the streaming Kubernetes WatchList Pod reflector and410 Gone-resilient Event watcher, theevent-exporterbinary entrypoint, multi-arch static musl container build ontogcr.io/distroless/static:latest, and live multi-shard subprocess E2E tests.Included in Part 3
src/k8s_stream.rs: StreamingPartialObjectMetadataPod reflector with explicitk8s.io/initial-events-endbookmark gating, automatic HTTP422paginatedListfallback, Go"items": nullempty-list handling, and line-by-line Event watcher with410 Gonewatermark recovery (zeroDeltaFIFOpayload retention).src/main.rs,Dockerfile,Makefile: Multi-arch (linux/amd64,linux/arm64) statically linked musl build (rust:1-alpine+crt-static) ontogcr.io/distroless/static:latest(USER 65532:65532), wired into.github/workflows/ci.yaml(ci-container-build) and.github/dependabot.yaml(docker).tests/: WatchList premature-EOF regression test,422fallback +410 Gonerecovery test, subprocess CLI +SIGTERMdrain test, and 3-shard live subprocess OTLP + HTTP429backpressure benchmark (1,500/1,500unique enriched events at10.9-12.3 MiBRSS/shard).Stack
PodOwnerTable, enricher, and metrics410 Gonerecovery, binary entrypoint,Dockerfile, and multi-shard E2E suite