Repository navigation
feat(gateway): expose workspace service route snapshots and watches - #4941
Merged
Merged
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
8 tasks done
|
🌿 Preview your docs: https://nvidia-preview-pr-4941.docs.buildwithfern.com/openshell |
This was referenced Oct 27, 2026
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 03:15
c724036 to
932643d
Compare
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 05:42
932643d to
6c83b66
Compare
This was referenced Oct 27, 2026
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 06:24
6c83b66 to
755e4d1
Compare
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 06:25
755e4d1 to
0944ea2
Compare
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 06:46
0944ea2 to
8504067
Compare
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
October 27, 2026 21:06
8504067 to
feac6b4
Compare
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
…nsion Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
TaylorMutch
force-pushed
the
feat/4667-service-discovery/TaylorMutch
branch
from
November 16, 2026 19:55
fd8af36 to
29c0d05
Compare
pimlock
approved these changes
Nov 16, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 16, 2026
jhjaggars
added a commit
to jhjaggars/OpenShell
that referenced
this pull request
Nov 17, 2026
Notes and live-trial evidence for the workspace service-route discovery API (PR NVIDIA#4941): snapshot/watch behavior on a k3d cluster, the TLS data plane to discovered supervisor endpoints, and a shared-certificate edge PoC (Envoy Gateway + HTTPRoute + BackendTLSPolicy) re-encrypting to a discovered endpoint. Includes design options and upstream feedback for shared-cert service exposure. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expose accepted supervisor installations through the separate
openshell.v1.ServiceDiscoveryservice:GetServiceRouteSnapshotandWatchServiceRoutesfor one explicit authorized workspace. Independent proxies and infrastructure controllers consume exact hostnames and trusted supervisor endpoints through the same API.Based on main at
e8ba30fde.Related Issue
Closes #4667. Part of RFC 0017.
Changes
Keep discovery separate from the management
OpenShellservice. Register it with gateway auth, configured interceptors, and reflection. Existing management server implementations need no discovery stubs; Rust SDK consumers useraw_service_discovery()/raw_service_discovery_fresh().Reuse collection-local provider catalog, global settings, and bounded bulk service declarations while retaining per-sandbox validation. Recheck shared dependencies before publication; unsuitable bulk inputs fall back to individual reads.
Reread accepted installation receipts before publication, recheck live ownership, and revalidate renewed configuration. Return the latest accepted expiry for unchanged identities; omit withdrawn or replaced installations. Reads never extend leases.
Share collection results for one second per workspace, cache at most 16 workspaces, and allow four simultaneous collections. Never return expired cached routes.
Bound unary requests to 16 total, four per principal, eight per workspace. Bound watches to 32 total, four per principal, 16 per workspace; release all budgets on cancellation/error/expiry.
Recheck bearer authentication, use refreshed identity roles, and reauthorize workspace access before publishing watch replacements. Offline JWT revocation follows verifier semantics.
Gate snapshot and watch RPCs on
supervisor_app_listener.enabled; disabled listeners returnFAILED_PRECONDITIONbefore collecting routes or allocating a watch.Publish accepted installation records from feat(supervisor): advertise accepted application route installations #4940, which resolves trusted supervisor endpoints through the private
ResolveSupervisorAppEndpointcompute RPC. External consumers use the public discovery RPCs; ordinaryGetSandboxobservations do not resolve routes.Return complete replacements with installation versions, declaration revisions, expiry, and current owner endpoints.
Recheck session, runtime, credential epoch, and shared pool authority before publishing routes.
Bound collection, watches, buffering, send time, and lifetime; periodically reauthorize watches and fail closed on storage errors.
Document client cache expiry and stable installation versions across lease-only renewal.
Release Notes
Discovery uses
/openshell.v1.ServiceDiscovery/GetServiceRouteSnapshotand/openshell.v1.ServiceDiscovery/WatchServiceRoutes. Message shapes are unchanged.Discovery and Kubernetes request flow
Discovery returns an application
hostnameand a trusted supervisorendpoint(host and port), together with installation identity and expiry.
nameis thesandbox route's service name; an empty value identifies the unnamed/default
service. It is not a Kubernetes Service resource name.
The endpoint is the owning supervisor Pod IP and app-listener port. Consumers
preserve the application hostname as TLS SNI and HTTP Host/authority. Multiple
routes can share one supervisor endpoint; the hostname selects the sandbox
instance and service. The authoritative public application URL is separate and
can use a different public port.
flowchart TB subgraph Control["Control plane: build and publish discovery"] SUP["Supervisor<br/>Reports installed routes against a lease"] GW["OpenShell gateway<br/>Validates installation and ownership"] KD["Kubernetes compute driver<br/>ResolveSupervisorAppEndpoint"] KUBE["Kubernetes objects + supervisor ownership<br/>Find and validate owning supervisor Pod"] API["ServiceDiscovery service<br/>Snapshot / watch<br/>hostname + Pod IP:8443 + expiry"] CONSUMER["Sandbox proxy or<br/>example Kubernetes HTTPRoute controller"] SUP -->|"Installed-route advertisement"| GW GW -->|"Resolve trusted endpoint"| KD KD --> KUBE KUBE -->|"Validated Pod IP and listener port"| KD KD --> GW GW --> API API -->|"Snapshot / watch"| CONSUMER end subgraph Data["Application data plane"] CLIENT["Client<br/>https://team--demo--web.apps.example.com"] EDGE["Configured proxy or Kubernetes Gateway<br/>Public listener, e.g. port 443"] LISTENER["Owning supervisor Pod<br/>10.42.1.27:8443<br/>TLS app listener"] DISPATCH["Select sandbox instance + named service<br/>using SNI / Host"] CONNECTOR["Selected instance's sandbox connector"] APP["Application inside sandbox<br/>127.0.0.1:8080"] CLIENT --> EDGE EDGE -->|"Connect to discovered endpoint<br/>preserve application SNI / Host"| LISTENER LISTENER --> DISPATCH DISPATCH --> CONNECTOR CONNECTOR --> APP end CONSUMER -.->|"Configure forwarding"| EDGEAddresses, domain names, and ports above are illustrative. Discovery describes
accepted installations, not application health or external ingress readiness.
The Kubernetes driver resolves trusted endpoints; a separate consumer configures
HTTPRoutes or proxy forwarding. Application traffic bypasses the OpenShell
gateway. The proxy and example controller are separate consumers, not part of
this PR's implementation.
Testing
e8ba30fde: standalone discovery passed 2,725 server tests (42 PostgreSQL-dependent tests skipped locally).Earlier local k3d validation covered dedicated/shared owner changes, HTTP/WebSockets, network-policy isolation, discovery withdrawal, and two proxy replicas. This revision has no new live-cluster or production-load qualification. New-head CI is pending.
Qualification limits
Live HA validation exercised supervisor process/Pod loss on a local k3d cluster. Whole-node failure, real network partitions, metric-driven HPA, sustained production load, and published release artifacts remain unqualified. Grant-file projection contributes to revocation propagation time. Main now includes #4842 for gateway-mediated hashed hostnames. Direct application-listener/discovery hostnames remain readable; extending hashed naming to that path and installation-status presentation (#4670) remain separate work.
The example Kubernetes HTTPRoute controller is a deferred compatibility POC outside this stack.
Checklist
PR dependency order: #4941 → #4956 → #4963 → #4964 → #4957.