Skip to content

feat(services): publish authoritative application origins - #4956

Open
TaylorMutch wants to merge 8 commits into
mainfrom
feat/4953-authoritative-app-urls/TaylorMutch
Open

TaylorMutch wants to merge 8 commits into
mainfrom
feat/4953-authoritative-app-urls/TaylorMutch

Conversation

@TaylorMutch

@TaylorMutch TaylorMutch commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Publish authoritative application URLs from the configured supervisor application origin, independently of the gateway API origin. Return explicit application and gateway URLs so clients can choose an access path. Allow installations to disable gateway-mediated application routing independently.

Based on main at 9f3c9560d.

Related Issue

Closes #4953. Part of RFC 0017.

Changes

  • Use supervisor.app_listener.base_url as the single authored application origin and derive its service domain. Enabled listeners require an HTTPS DNS base URL; remove the previous domain/public_url fields. The supervisor listener port stays independent of the public URL port.
  • Log enabled gateway/app-listener routing at startup and warn when neither mode is enabled. Runtime defaults remain unchanged.
  • Add [openshell.gateway].service_routing_enabled (default true). Setting it to false omits gateway URLs and returns 404 for gateway application HTTP/WebSocket traffic, while keeping control-plane APIs, app listeners, and discovery available.
  • Share one readable/hashed hostname policy with gateway-mediated URLs, including the existing feat(gateway): add hashed service-routing hostnames for multi-gateway clusters #4842 hash encoding. URL responses, authenticated supervisor configuration, discovery, and SNI/Host routing use the same canonical hostname.
  • Validate the HTTPS public origin and preserve its independent public port.
  • Populate service and sandbox responses, mutation replays, suspend/resume responses, CLI, and Rust/TypeScript clients.
  • Preserve recorded url and service_urls values on idempotent retries; refresh only explicit application/gateway fields. Keep the new curated TypeScript URL maps optional.
  • Document DNS/TLS requirements and distinct ingress authentication boundaries. Installation-status presentation remains in feat(services): publish authoritative supervisor ingress URLs and status #4670.

Configuration

Gateway routing is enabled by default. Disable it independently of the app listener using any of these equivalent settings:

Configuration surface Setting
Gateway TOML [openshell.gateway] → service_routing_enabled = false
Gateway CLI --service-routing-enabled=false
Environment OPENSHELL_SERVICE_ROUTING_ENABLED=false
Helm values gatewayConfig."openshell.gateway".service_routing_enabled: false

Precedence is CLI > environment > TOML > default. Explicit true and false overrides are supported.

Example: publish only application-ingress URLs, with hashed hostnames:

[openshell.gateway]
name = "gateway-a"
service_hostname_format = "hashed"
service_routing_enabled = false

[openshell.supervisor.app_listener]
enabled = true
base_url = "https://apps.example.com"

base_url supplies the hostname suffix under which service DNS records are created. The listener derives its domain from that URL.

name supplies the stable installation identity for hashed names. service_hostname_format defaults to "readable" and applies to both ingress paths. The application base_url is an HTTPS origin whose hostname determines the service domain; its public port can differ from the listener port. DNS, certificates, and ingress are provisioned separately.

For the routing opt-out alone in Helm:

gatewayConfig:
  openshell.gateway:
    service_routing_enabled: false

Integrator perspective

The service API tells clients which public URL to use; discovery tells routers where to send requests.

  • Configure the application base_url for your public ingress. Responses expose application_url, gateway_url when gateway routing is enabled, and a preferred url that selects the application URL when configured.
  • A proxy or controller consumes discovery snapshots/watches mapping each exact hostname to the current supervisor Pod IP and app-listener port. Follow ownership changes and honor withdrawal and expiry; consume the returned hostname without reconstructing it.
  • The sandbox proxy routes directly to those endpoints. An example Kubernetes HTTPRoute controller can instead configure HTTPRoutes and their backend/TLS resources. Application traffic bypasses the OpenShell gateway.

Publishing a URL does not create DNS or ingress resources, or establish reachability. Both ingress paths use [openshell.gateway].service_hostname_format; the app listener has no separate naming mode. Keeping the installation name and public HTTPS domain/port unchanged preserves the service URL when switching ingress.

Public URL selection

This PR publishes the configured application origin alongside the gateway-mediated URL when enabled. The example uses gateway installation gateway-a, workspace default, sandbox my-sandbox, and service web.

flowchart TD
    I["Service identity<br/>gateway-a / default / my-sandbox / web"]
    M["One gateway setting: service_hostname_format<br/>readable or hashed"]
    I --> L["Shared service label generator<br/>Hashed example: enh47uqyj7bpyttz"]
    M --> L
    E["service_routing_enabled<br/>Default: true"] --> G
    L --> G["gateway_url<br/>https://enh47uqyj7bpyttz.gateway.example.com/"]
    L --> A["application_url<br/>https://enh47uqyj7bpyttz.apps.example.com/"]
    O["Configured application public origin<br/>https://apps.example.com"] --> A
    A -->|"Preferred when configured"| U["url returned to clients"]
    G -->|"Fallback when application_url is absent"| U
Loading

Set service_routing_enabled = false to use application ingress without advertising or serving gateway-mediated application URLs. The CLI/env equivalents are --service-routing-enabled=false and OPENSHELL_SERVICE_ROUTING_ENABLED=false; Helm uses gatewayConfig."openshell.gateway".service_routing_enabled: false.

Gateway routing Application origin Preferred url gateway_url
Enabled Configured Application URL Gateway URL
Enabled Absent Gateway URL Gateway URL
Disabled Configured Application URL Empty
Disabled Absent Empty Empty

Disabling routing also makes gateway_service_urls empty. Recorded preferred URLs stay unchanged on idempotent retries; explicit URL fields reflect current configuration.

Hashed mode is shown above. Readable mode uses <workspace>--<sandbox>[--<service>] for both paths. The two access paths are alternatives; when their public origins match, their complete URLs match. Integrators consume discovery hostnames as returned.

Connecting the published URL to a sandbox

The proxy and example controller below are alternative discovery consumers, shown for context. This PR publishes URLs; it does not deploy either integration. Dashed arrows configure routing; solid arrows carry application traffic.

flowchart TD
    API["Service API<br/>Returns public application URL"] -.->|"URL used by client"| Client["Application client"]
    D["Discovery API<br/>Exact hostname → current supervisor Pod IP:port<br/>Installation identity and expiry"]

    D -.->|"Snapshots / watches"| P["Sandbox proxy<br/>TLS passthrough"]
    D -.->|"Snapshots / watches"| C["Example Kubernetes HTTPRoute controller<br/>Separate POC"]
    C -.->|"Reconciles HTTPRoutes, backend resources<br/>and backend TLS configuration"| K["Kubernetes Gateway<br/>Edge TLS termination"]

    Client -->|"Option A: HTTPS to published hostname"| P
    Client -->|"Option B: HTTPS to published hostname"| K
    P -->|"Application TLS forwarded unchanged"| S["Owning supervisor<br/>Application TLS listener"]
    K -->|"HTTPS to discovered endpoint<br/>Preserve application Host and backend SNI"| S
    S --> W["Declared service inside sandbox"]
Loading

Both paths keep application traffic out of the OpenShell gateway. The integrator provides DNS, certificates, and ingress configuration; discovery supplies backend changes and withdrawal/expiry information.

Release Notes

Compatibility context

Enabled application listeners now require base_url in place of domain and optional public_url. Remove those old keys when upgrading and deploy matching gateway/supervisor images. The domain is derived from the HTTPS base URL; its public port remains separate from the supervisor listener port.

Supervisor route declarations are explicitly an internal, version-locked deployment protocol. Discovery and canonical hostname publication are being introduced in this stack; no released discovery hostname API needs a migration. The existing readable listener aliases remain supported.

Protobuf additions remain additive on the wire. Curated SDK callers retain their existing construction paths; new TypeScript URL maps are optional. Hand-written exhaustive generated Rust/TypeScript message literals must include new fields or use generated/default initialization; preserving those literal shapes is outside this change's migration scope.

Python SDK changes are deferred to the external SDK repository; this PR leaves the frozen core SDK files unchanged.

Testing

  • Rebased onto main 9f3c9560d; standalone URL runtime passed 2,772 server tests (45 PostgreSQL-dependent tests skipped), 652 core tests, 336 Kubernetes-driver tests, and 300 supervisor tests.
  • Combined auth/discovery/proxy stack passed 2,803 server tests (46 PostgreSQL-dependent tests skipped), all nine proxy tests, and the additional server test targets. Strict all-target Clippy passed for core, server, Kubernetes driver, supervisor, and proxy.
  • TypeScript CI passed: generation/proto lint, lint, type checks, 207 tests, coverage, and build. Full Helm unit suites, rendered-TOML validation, resource coherence/ownership/scaling checks, Markdown/Mermaid lint, generated docs, and Rust formatting passed.
  • All eight app-listener/gateway-routing/service-auth startup combinations verified with the real gateway and base_url.
  • Live isolated k3d app-listener E2E passed in dedicated and shared modes, including HTTP/WebSockets, Host/SNI validation, network-policy isolation, withdrawal/re-exposure, dedicated gateway-outage expiry, graceful shared owner takeover preserving workload identity/process/memory, and explicit gateway fallback with the listener disabled. Matching gateway/supervisor/sandbox images were built locally; temporary releases were cleaned up.

Live tests exercised supervisor Pod-IP routing with Host/SNI, rather than public DNS or an external ingress controller. New-head CI is pending.

Qualification limits

Live Kubernetes qualification covers dedicated/shared supervisors, graceful shared-owner replacement, gateway outage, and policy isolation. Hashed URL/public-port behavior is covered by unit tests. Whole-node failure, real network partitions, metric-driven HPA, sustained production load, external ingress/DNS automation, and published release artifacts remain unqualified. Installation-status presentation (#4670) remains separate work. The example Kubernetes HTTPRoute controller remains a separate compatibility POC.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Relevant implementation, configuration, and API documentation updated

PR dependency order: #4956 → #4963 → #4964 → #4957.

@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

@TaylorMutch
TaylorMutch added this pull request to stack #4942 October 27, 2026 02:54
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from a6d0833 to e6e1927 Compare October 27, 2026 03:15
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch 2 times, most recently from 932643d to 6c83b66 Compare October 27, 2026 05:42
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from e6e1927 to 2574b9a Compare October 27, 2026 05:42
@TaylorMutch
TaylorMutch removed this pull request from stack #4942 October 27, 2026 05:45
@TaylorMutch
TaylorMutch added this pull request to stack #4965 October 27, 2026 05:46
@TaylorMutch
TaylorMutch removed this pull request from stack #4965 October 27, 2026 05:47
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch from 6c83b66 to 755e4d1 Compare October 27, 2026 06:24
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch 2 times, most recently from 70e365e to 30cf9ce Compare October 27, 2026 06:25
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch 2 times, most recently from 0944ea2 to 8504067 Compare October 27, 2026 06:46
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from 30cf9ce to 6b2fe9c Compare October 27, 2026 06:46
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch from 8504067 to feac6b4 Compare October 27, 2026 21:06
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch 2 times, most recently from dae4a9d to 66220a7 Compare October 28, 2026 17:05
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch 2 times, most recently from 0939a70 to cf5448a Compare October 28, 2026 22:52
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from 66220a7 to 3dc01d7 Compare October 28, 2026 22:52
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch 3 times, most recently from 15f0afb to 6903eaa Compare November 12, 2026 20:17
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch from 316dd66 to fd8af36 Compare November 13, 2026 21:15
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from 6903eaa to 433b9b9 Compare November 13, 2026 21:15
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

3. api-behavior — No explicit risk acceptance found

ExposeService.url and CreateSandbox.service_urls replays now recompute current origins instead of returning their recorded URL values.

  • Impact: Automation retrying the same request_id after a domain or port change receives different URLs, breaking reconciliation against the original receipt.
  • Action: Preserve recorded url and service_urls on v0.1.9 replays and expose current addresses only through the new application and gateway fields.
  • PR: feat(services): publish authoritative application origins #4956
Review details
  • Execution: complete
  • Candidate: PR #4956 (ea8560731cfaf18cc8e02fcc6ac1a19816012080)
  • Baseline: fd8af365e7bb22c5ce57fb2a8e900d4624491d93 (fd8af365e7bb22c5ce57fb2a8e900d4624491d93)
  • Reviewed: rust-sdk, python-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@purp purp added area:platform SDKs, CLIs, APIs, gateway infrastructure, persistence, configuration, and session management. topic:api Public APIs, gRPC/protobuf, SDK interfaces, versioning, compatibility, and API design standards. topic:documentation Architecture docs, READMEs, project website updates, and automated documentation generation. topic:networking General network transport, connectivity, and proxy behavior. topic:sdk:python Python SDK implementation, maintenance, interfaces, and distribution. topic:sdk:rust Rust SDK implementation, maintenance, interfaces, and distribution. topic:sdk:typescript TypeScript SDK implementation, maintenance, interfaces, and distribution. topic:ux CLI commands/flags, help, errors/exit codes, human-readable output, TUI, and first-run experience. labels Nov 15, 2026
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from 433b9b9 to 35e2d05 Compare November 16, 2026 19:55
@TaylorMutch
TaylorMutch force-pushed the feat/4667-service-discovery/TaylorMutch branch from fd8af36 to 29c0d05 Compare November 16, 2026 19:55
Base automatically changed from feat/4667-service-discovery/TaylorMutch to main November 16, 2026 21:36
@TaylorMutch
TaylorMutch force-pushed the feat/4953-authoritative-app-urls/TaylorMutch branch from 35e2d05 to c503a94 Compare November 16, 2026 21:48
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

3. typescript-sdk — No explicit risk acceptance found

Generated raw SandboxResponse, ServiceEndpointResponse, and SandboxServiceRoute types acquire required URL-map, URL, and applicationHostname properties absent from previously valid complete objects.

  • Impact: Typed raw-message fixtures and adapters fail assignment checks, while schema-based initialization and the now-optional curated SandboxRef properties remain compatible.
  • Action: Preserve existing raw TypeScript object shapes for v0.1.9, or defer incompatible additions to a permitted minor train with create(schema, init) migration guidance.
  • PR: feat(services): publish authoritative application origins #4956
Review details
  • Execution: complete
  • Candidate: PR #4956 (8c669ef9f62daea9114916c9482f5b2f05573d2e)
  • Baseline: bf9f55cdf009eaf9b84f486d5c15ff89c9fcf2ca (bf9f55cdf009eaf9b84f486d5c15ff89c9fcf2ca)
  • Reviewed: rust-sdk, python-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. configuration — No explicit risk acceptance found

App-listener TOML domain and supervisor environment variable OPENSHELL_APP_SERVICE_DOMAIN are replaced by base_url and OPENSHELL_APP_BASE_URL without legacy aliases.

  • Impact: Existing gateway TOML fails unknown-field validation, while enabled openshell-supervisor launches using the old environment fail with a missing-base_url error.
  • Action: Retain the legacy TOML key and supervisor environment variable as compatibility inputs, deriving an HTTPS base URL when necessary.
  • PR: feat(services): publish authoritative application origins #4956

2. api-behavior — No explicit risk acceptance found

ServiceEndpointResponse.url and create-time service_urls switch from gateway URLs to application URLs when the application listener is configured.

  • Impact: Older openshell service expose, service list, and sandbox-create clients replace the application port with the gateway connection port, producing unusable URLs when those ports differ.
  • Action: Preserve gateway semantics in the existing URL fields for legacy clients and expose application origins through the new fields or negotiated behavior.
  • PR: feat(services): publish authoritative application origins #4956
Review details
  • Execution: complete
  • Candidate: PR #4956 (e0f79051c0f8c4439881086cb828d34bae487951)
  • Baseline: 4b433e6e3284574866ed0a837a180dbc00ecc800 (4b433e6e3284574866ed0a837a180dbc00ecc800)
  • Reviewed: rust-sdk, python-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:platform SDKs, CLIs, APIs, gateway infrastructure, persistence, configuration, and session management. topic:api Public APIs, gRPC/protobuf, SDK interfaces, versioning, compatibility, and API design standards. topic:documentation Architecture docs, READMEs, project website updates, and automated documentation generation. topic:networking General network transport, connectivity, and proxy behavior. topic:sdk:python Python SDK implementation, maintenance, interfaces, and distribution. topic:sdk:rust Rust SDK implementation, maintenance, interfaces, and distribution. topic:sdk:typescript TypeScript SDK implementation, maintenance, interfaces, and distribution. topic:ux CLI commands/flags, help, errors/exit codes, human-readable output, TUI, and first-run experience.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(services): publish authoritative application origins independently of API transport

3 participants