Skip to content

event-exporter-rs (1/3): add core crate, CLI/sharding, 24B PodOwnerTable, enricher, and metrics - #1303

Open
zxqlxy wants to merge 1 commit into
GoogleCloudPlatform:masterfrom
zxqlxy:event-exporter-rs-pr1
Open

zxqlxy wants to merge 1 commit into
GoogleCloudPlatform:masterfrom
zxqlxy:event-exporter-rs-pr1

Conversation

@zxqlxy

@zxqlxy zxqlxy commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary (Part 1 of 3)

Adds the core crate, CLI/sharding model, 24-byte/slot PodOwnerTable, Cloud Logging event enricher, and LOrIS Prometheus metrics for event-exporter-rs, a standalone #![forbid(unsafe_code)] Rust implementation of GKE's Kubernetes Event exporter alongside the existing Go event-exporter/.

On 100k to 1M Pod clusters, the Go exporter repeatedly OOMKills inside its 100Mi memory budget (0.8 GiB to 1.6 GiB peak RSS during initial WatchList bursts under GOGC=100, plus unpopped DeltaFIFO.Replace() entries on List fallback). PodOwnerTable stores interned Pod controller owners in flat 24-byte open-addressed slots (16B PodKey + 8B CompactOwner), holding 1,000,000 active Pods in ~50.4 MiB RSS delta (~58.8 MiB total process RSS).

Included in Part 1

  • src/cli.rs & src/shard.rs: Drop-in parser for GKE single-dash Go flags (-sink-opts, -universeDomain, -system-namespaces, -event-label-selector) plus KEP-5866 --shard-mode=server and Jump Consistent Hash --shard-mode=client Pod/Event co-partitioning.
  • src/pod_cache.rs: Lock-sharded 24-byte/slot PodOwnerTable resolving Deployment, StatefulSet, DaemonSet, CronJob, Job, and JobSet (restart-attempt + jobset-uid) owners.
  • src/enricher.rs & src/metrics.rs: k8s_pod / k8s_node / k8s_cluster routing, cross-namespace RBAC spoofing guard (event.metadata.namespace == involvedObject.namespace), fieldBlacklist filtering, Gregorian RFC3339 validation, and :8080/metrics + /healthz LOrIS compatibility.
  • tests/ & .github/: Exhaustive state-machine refinement test, sharding bijection test, 1,000,000-pod memory benchmark, GitHub Actions ci-rust-event-exporter workflow, and Dependabot cargo updates.

Stack

  1. Part 1 (event-exporter-rs (1/3): add core crate, CLI/sharding, 24B PodOwnerTable, enricher, and metrics #1303, this PR): Core crate, CLI/sharding, PodOwnerTable, enricher, and metrics
  2. Part 2 (event-exporter-rs (2/3): add pure-Rust HTTP/gRPC transport, GKE auth, batch pipeline, and dual sinks #1304): Pure-Rust HTTP/gRPC transport, GKE NodeTokenBroker / Metadata auth, bounded batch pipeline, and dual Cloud Logging v2 + OTLP sinks
  3. Part 3 (event-exporter-rs (3/3): add streaming K8s WatchList, binary entrypoint, Dockerfile, and E2E tests #1305): Streaming K8s WatchList reflector + 410 Gone recovery, binary entrypoint, gcr.io/distroless/static Dockerfile, and multi-shard E2E suite

…nricher, and metrics

Part 1 of 3 introducing `event-exporter-rs`, a `#![forbid(unsafe_code)]` pure-Rust
Kubernetes Event exporter for GKE:

- `src/cli.rs` & `src/shard.rs`: Drop-in parser for GKE single-dash Go flags
  (`-sink-opts`, `-universeDomain`, `-system-namespaces`, `-event-label-selector`)
  plus KEP-5866 `--shard-mode=server` and Jump Consistent Hash `--shard-mode=client`
  Pod/Event co-partitioning.
- `src/pod_cache.rs`: Lock-sharded open-addressed 24-byte/slot `PodOwnerTable`
  (`16B PodKey` + `8B CompactOwner`) resolving `Deployment`, `StatefulSet`,
  `DaemonSet`, `CronJob`, `Job`, and `JobSet` (`restart-attempt` + `jobset-uid`)
  owners while holding 1,000,000 active Pods in ~50.4 MiB RSS delta (~58.8 MiB
  total process RSS vs. >1.2 GiB in Go `client-go`).
- `src/enricher.rs` & `src/metrics.rs`: Cloud Logging `k8s_pod`/`k8s_node`/`k8s_cluster`
  routing, cross-namespace RBAC spoofing guard, `fieldBlacklist` filtering,
  Gregorian RFC3339 validation, and 100% LOrIS Prometheus metric compatibility.
- Formal state-machine refinement, sharding bijection, and 1,000,000-pod memory
  benchmark suites, wired into GitHub Actions (`ci-rust-event-exporter`).
@zxqlxy
zxqlxy force-pushed the event-exporter-rs-pr1 branch from 4f6f290 to 39df98a Compare November 11, 2026 02:35
@zxqlxy
zxqlxy marked this pull request as ready for review November 11, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant