Repository navigation
event-exporter-rs (2/3): add pure-Rust HTTP/gRPC transport, GKE auth, batch pipeline, and dual sinks - #1304
Draft
zxqlxy wants to merge 2 commits into
Draft
event-exporter-rs (2/3): add pure-Rust HTTP/gRPC transport, GKE auth, batch pipeline, and dual sinks#1304zxqlxy wants to merge 2 commits into
zxqlxy wants to merge 2 commits into
Conversation
…nricher, and metrics Part 1 of 3 introducing `event-exporter-rs`, a `#![forbid(unsafe_code)]` pure-Rust Kubernetes Event exporter for GKE: - `src/cli.rs` & `src/shard.rs`: Drop-in parser for GKE single-dash Go flags (`-sink-opts`, `-universeDomain`, `-system-namespaces`, `-event-label-selector`) plus KEP-5866 `--shard-mode=server` and Jump Consistent Hash `--shard-mode=client` Pod/Event co-partitioning. - `src/pod_cache.rs`: Lock-sharded open-addressed 24-byte/slot `PodOwnerTable` (`16B PodKey` + `8B CompactOwner`) resolving `Deployment`, `StatefulSet`, `DaemonSet`, `CronJob`, `Job`, and `JobSet` (`restart-attempt` + `jobset-uid`) owners while holding 1,000,000 active Pods in ~50.4 MiB RSS delta (~58.8 MiB total process RSS vs. >1.2 GiB in Go `client-go`). - `src/enricher.rs` & `src/metrics.rs`: Cloud Logging `k8s_pod`/`k8s_node`/`k8s_cluster` routing, cross-namespace RBAC spoofing guard, `fieldBlacklist` filtering, Gregorian RFC3339 validation, and 100% LOrIS Prometheus metric compatibility. - Formal state-machine refinement, sharding bijection, and 1,000,000-pod memory benchmark suites, wired into GitHub Actions (`ci-rust-event-exporter`).
… pipeline, and dual sinks Part 2 of 3 adding networking, GKE credential rotation, bounded batching, and Cloud Logging v2 + OpenTelemetry OTLP export sinks: - `src/transport.rs` & `src/otlp_proto.rs`: Pure-Rust `reqwest` + `rustls` HTTP/2 client (honoring GKE private-node `HTTP_PROXY`/`NO_PROXY` exemptions for `169.254.0.0/16` and `metadata.google.internal`) and `tonic`/`prost` OTLP `LogsService/Export` gRPC client. - `src/auth.rs`: GKE `NodeTokenBroker` external-account STS exchange (`GOOGLE_APPLICATION_CREDENTIALS`), direct token-broker URL support, GCE Metadata Server fallback, K8s SA bearer rotation, and `--universe-domain` endpoint resolution. - `src/sink.rs` & `src/pipeline.rs`: Byte-bounded (`16 MiB` default) micro-batch export queue with retry/jitter backoff, gzipped Cloud Logging `/v2/entries:write` JSON sink, and OTLP gRPC/HTTP sink preserving LOrIS status & freshness metrics.
zxqlxy
force-pushed
the
event-exporter-rs-pr2
branch
from
November 11, 2026 02:35
27f5c95 to
3274185
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary (Part 2 of 3)
Adds pure-Rust HTTP/2 + gRPC networking, GKE credential rotation, the byte-bounded micro-batching pipeline, and dual Cloud Logging v2 (
/v2/entries:write) + OpenTelemetry OTLP (LogsService/Export) export sinks toevent-exporter-rs.Included in Part 2
src/transport.rs&src/otlp_proto.rs: Pure-Rustreqwest+rustlsHTTP/2 client (preservinglocalhost,169.254.0.0/16, andmetadata.google.internalNO_PROXYbypasses on private GKE nodes) andtonic/prostOTLPLogsService/ExportgRPC client.src/auth.rs: GKENodeTokenBrokerexternal-account STS token exchange (GOOGLE_APPLICATION_CREDENTIALS), direct token-broker URL support, GCE Metadata Server fallback, K8s ServiceAccount bearer rotation, and--universe-domainendpoint resolution.src/sink.rs&src/pipeline.rs: Byte-bounded (16 MiBdefault) batch export queue with jittered backoff, gzipped Cloud Logging/v2/entries:writeJSON sink, and OTLP gRPC/HTTP sink mapping gRPC statuses (0 -> 200,7 -> 403,8 -> 429) ontostackdriver_sink_request_count.tests/: OTLP protobuf round-trip tests, livetonicgRPC server +partial_successintegration test,NodeTokenBrokerSTS + Metadata discovery tests, and event-to-sink freshness latency tests.Stack
PodOwnerTable, enricher, and metrics410 Gonerecovery, binary entrypoint,Dockerfile, and multi-shard E2E suite