Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a5b08ad
feat(gateway): broker stable sandbox SPIFFE provider tokens
drew Oct 22, 2026
7410286
refactor(gateway): rename SPIFFE delegation configuration
drew Oct 22, 2026
22456ba
fix(provider): reuse broker channels and attribute grant failures
drew Oct 22, 2026
4a8efe1
test(provider): satisfy broker logging lint checks
drew Oct 22, 2026
4686ac0
fix(provider): harden broker caching and deployment prerequisites
drew Oct 23, 2026
de6dd89
refactor(identity): extract SPIRE sandbox identity adapter
drew Oct 23, 2026
e33dd11
refactor(identity): move shared SPIFFE types out of core
drew Oct 23, 2026
16f61f5
fix(supervisor): follow the sandbox reconciler leader (#4853)
drew Oct 23, 2026
3593be0
chore(sdk): drop regenerated frozen Go bindings
TaylorMutch Nov 10, 2026
626feae
chore: integrate main into SPIFFE token broker
TaylorMutch Nov 10, 2026
100986e
chore: merge latest main into SPIFFE token broker
TaylorMutch Nov 12, 2026
7f88a02
test(spiffe): verify provider tokens across supervisor failover
TaylorMutch Nov 13, 2026
0d71cfd
chore: merge main into SPIFFE token broker
TaylorMutch Nov 13, 2026
3f939a0
feat(crypto): verify TLS peers by SPIFFE identity
TaylorMutch Nov 13, 2026
89aad0f
feat(spiffe): use Broker API for sandbox identity issuance
TaylorMutch Nov 13, 2026
17c8b7b
test(spiffe): verify broker authorization and credential rotation
TaylorMutch Nov 13, 2026
4532524
docs(spiffe): clean up broker troubleshooting and test wording
TaylorMutch Nov 16, 2026
8a2ef16
chore: merge main into SPIFFE token broker
TaylorMutch Nov 16, 2026
d7b0c95
chore: merge main into SPIFFE token broker
TaylorMutch Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .agents/skills/helm-dev-environment/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,22 @@ OPENSHELL_E2E_KUBE_IMAGE_PULL_SECRET=e2e-regcred \
mise run e2e:kubernetes:shared-supervisor
```

`mise run e2e:kubernetes:shared-supervisor-spiffe` installs pinned SPIRE charts
(`ci/values-spire-broker-stack.yaml`) and the `openshell.gateway.spiffe_broker`
overlay (`ci/values-supervisor-spiffe.yaml`). It exercises provider exchange and
abrupt supervisor takeover over TLS, using Sandbox CR UID registrations and
enforced Kubernetes authorization. Workload generation, Pod, process, and stable
SPIFFE identity survive; the new owner acquires a fresh provider token and the
broker rejects the old bearer. The fixture's 30-second workload X.509-SVIDs
exercise acquisition after credential rotation. The gateway Pod must pass
Restricted Pod Security admission. The task owns its SPIRE install and cluster.

The fixture supplies the single agent's exact SPIFFE ID. Multi-agent endpoints
need matching peer identity configuration and registrations for every selected
agent. The installed SPIRE version and operator must expose the experimental
Broker API. `e2e/kubernetes/spiffe-broker/README.md` describes independent
protocol and negative authorization checks.

### Shared supervisor high availability

The shared supervisor is always a pool of `supervisor.replicas` members
Expand Down
2 changes: 1 addition & 1 deletion .config/nextest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ test-group = "kubernetes-ha"
# they kill members or gateways, or scale the pool. Never run them at once.
# Test names, not binary(), for the reason above.
[[profile.e2e-kubernetes.overrides]]
filter = "test(/^(losing_a_supervisor_pod_moves_its_sandboxes_without_restarting_them|scaling_a_supervisor_pool_up_and_down_keeps_its_sandboxes_running|pool_sessions_follow_their_supervisors_gateway)$/)"
filter = "test(/^(losing_a_supervisor_pod_moves_its_sandboxes_without_restarting_them|scaling_a_supervisor_pool_up_and_down_keeps_its_sandboxes_running|pool_sessions_follow_their_supervisors_gateway|shared_supervisor_acts_as_each_sandboxs_stable_spiffe_identity)$/)"
test-group = "kubernetes-shared-supervisor-pool"

# Scaling the pool runs two bounded Helm upgrades (up to 6 minutes each), waits
Expand Down
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Do not rely on this file for a full inventory. The detailed public and contribut
| `crates/openshell-gateway/` | Gateway binary composition | Links selected first-party compute drivers into the backend-agnostic server registry |
| `crates/openshell-sdk/` | Shared client SDK | Async Rust gateway client (gRPC transport, TLS, OIDC refresh, edge tunnel); consumed by CLI, TUI, and `@openshell/sdk` |
| `crates/openshell-providers/` | Provider management | Credential provider backends |
| `crates/openshell-spiffe/` | SPIFFE identity types | Dependency-light SPIFFE ID mapping, claim parsing, and the sandbox identity provider interface |
| `crates/openshell-spire/` | SPIRE adapter | Gateway-owned SPIFFE Broker API client that issues and verifies stable sandbox JWT-SVIDs |
| `crates/openshell-tui/` | Terminal UI | Ratatui-based dashboard for monitoring |
| `crates/openshell-driver-kubernetes-secrets/` | Kubernetes Secrets credential driver | In-process `CredentialDriver` backend for OpenShell-managed K8s Secret storage |
| `crates/openshell-driver-vault/` | Vault credential driver | In-process `CredentialDriver` backend for Vault-compatible KV storage |
Expand Down
39 changes: 39 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions crates/openshell-cli/tests/ensure_providers_integration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,14 @@ impl TestOpenShell {

#[tonic::async_trait]
impl OpenShell for TestOpenShell {
async fn get_sandbox_provider_access_token(
&self,
_request: tonic::Request<openshell_core::proto::GetSandboxProviderAccessTokenRequest>,
) -> Result<Response<openshell_core::proto::GetSandboxProviderAccessTokenResponse>, Status>
{
Err(Status::unimplemented("not used by this test server"))
}

async fn peer_notify_config_update(
&self,
_request: tonic::Request<openshell_core::proto::PeerNotifyConfigUpdateRequest>,
Expand Down
8 changes: 8 additions & 0 deletions crates/openshell-cli/tests/mtls_integration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ struct TestOpenShell;

#[tonic::async_trait]
impl OpenShell for TestOpenShell {
async fn get_sandbox_provider_access_token(
&self,
_request: tonic::Request<openshell_core::proto::GetSandboxProviderAccessTokenRequest>,
) -> Result<Response<openshell_core::proto::GetSandboxProviderAccessTokenResponse>, Status>
{
Err(Status::unimplemented("not used by this test server"))
}

async fn peer_notify_config_update(
&self,
_request: tonic::Request<openshell_core::proto::PeerNotifyConfigUpdateRequest>,
Expand Down
8 changes: 8 additions & 0 deletions crates/openshell-cli/tests/provider_commands_integration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,14 @@ impl TestOpenShell {

#[tonic::async_trait]
impl OpenShell for TestOpenShell {
async fn get_sandbox_provider_access_token(
&self,
_request: tonic::Request<openshell_core::proto::GetSandboxProviderAccessTokenRequest>,
) -> Result<Response<openshell_core::proto::GetSandboxProviderAccessTokenResponse>, Status>
{
Err(Status::unimplemented("not used by this test server"))
}

async fn peer_notify_config_update(
&self,
_request: tonic::Request<openshell_core::proto::PeerNotifyConfigUpdateRequest>,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,14 @@ struct TestOpenShell {

#[tonic::async_trait]
impl OpenShell for TestOpenShell {
async fn get_sandbox_provider_access_token(
&self,
_request: tonic::Request<openshell_core::proto::GetSandboxProviderAccessTokenRequest>,
) -> Result<Response<openshell_core::proto::GetSandboxProviderAccessTokenResponse>, Status>
{
Err(Status::unimplemented("not used by this test server"))
}

async fn peer_notify_config_update(
&self,
_request: tonic::Request<openshell_core::proto::PeerNotifyConfigUpdateRequest>,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,6 @@ macro_rules! mock_gateway {
) => {
#[tonic::async_trait]
impl OpenShell for MockGateway {

$(async fn $method(&self, _: Request<$request>) -> Result<Response<$response>, Status> {
Err(Status::unimplemented("unused test RPC"))
})*
Expand Down Expand Up @@ -272,6 +271,7 @@ mock_gateway! {
report_sandbox_configuration(proto::ReportSandboxConfigurationRequest) -> proto::ReportSandboxConfigurationResponse;
get_sandbox_provider_environment(proto::GetSandboxProviderEnvironmentRequest) -> proto::GetSandboxProviderEnvironmentResponse;
exchange_provider_subject_token(proto::ExchangeProviderSubjectTokenRequest) -> proto::ExchangeProviderSubjectTokenResponse;
get_sandbox_provider_access_token(proto::GetSandboxProviderAccessTokenRequest) -> proto::GetSandboxProviderAccessTokenResponse;
get_sandbox_logs(proto::GetSandboxLogsRequest) -> proto::GetSandboxLogsResponse;
report_main_process_exit(proto::ReportMainProcessExitRequest) -> proto::ReportMainProcessExitResponse;
finalize_main_process_exit(proto::FinalizeMainProcessExitRequest) -> proto::FinalizeMainProcessExitResponse;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ struct TestOpenShell {

#[tonic::async_trait]
impl OpenShell for TestOpenShell {
async fn get_sandbox_provider_access_token(
&self,
_request: tonic::Request<openshell_core::proto::GetSandboxProviderAccessTokenRequest>,
) -> Result<Response<openshell_core::proto::GetSandboxProviderAccessTokenResponse>, Status>
{
Err(Status::unimplemented("not used by this test server"))
}

async fn peer_notify_config_update(
&self,
_request: tonic::Request<openshell_core::proto::PeerNotifyConfigUpdateRequest>,
Expand Down
1 change: 1 addition & 0 deletions crates/openshell-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ openshell-crypto = { workspace = true, features = ["tls-tonic"] }
async-trait = "0.1"
openshell-extension-core = { path = "../openshell-extension-core" }
openshell-policy-schema = { path = "../openshell-policy-schema" }
openshell-spiffe = { path = "../openshell-spiffe" }
glob = { workspace = true }
prost = { workspace = true }
prost-types = { workspace = true }
Expand Down
62 changes: 62 additions & 0 deletions crates/openshell-core/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,42 @@ pub fn normalize_compute_driver_name(value: &str) -> Result<String, String> {
Ok(value.to_ascii_lowercase())
}

/// Gateway-owned SPIFFE Broker API issuance for Kubernetes sandbox identities.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SpiffeBrokerConfig {
pub spiffe_id_prefix: String,
pub endpoint: String,
pub server_spiffe_id: String,
pub workload_api_socket_path: String,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the gateway's own Workload API socket now comes from two places: this field for the Broker mTLS client, and OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET for the RFC 8693 client assertion (provider.rs:4779). Startup checks neither against the other. The driver's refusal of provider_spiffe_workload_api_socket_path with supervisor_topology = "gateway" is also gone (driver config.rs:610), so a shared topology without spiffe_broker starts cleanly and fails every grant at request time. One source for the socket, plus a refusal when spiffe_broker is missing, would catch both at startup.

}

impl SpiffeBrokerConfig {
/// Validate the identity and authenticated transport before use.
pub fn validate(&self, gateway_id: &str) -> Result<(), String> {
openshell_spiffe::SandboxSpiffeIdPrefix::parse(&self.spiffe_id_prefix)?
.sandbox_spiffe_id(gateway_id, "sandbox")?;
let endpoint = url::Url::parse(&self.endpoint)
.map_err(|_| "spiffe_broker.endpoint must be an HTTPS endpoint")?;
if endpoint.scheme() != "https"
|| endpoint.host_str().is_none()
|| !endpoint.username().is_empty()
|| endpoint.password().is_some()
|| endpoint.path() != "/"
|| endpoint.query().is_some()
|| endpoint.fragment().is_some()
|| !std::path::Path::new(&self.workload_api_socket_path).is_absolute()
{
return Err(
"spiffe_broker requires an HTTPS endpoint and absolute Workload API socket path"
.into(),
);
}
openshell_spiffe::SandboxSpiffeIdPrefix::parse(&self.server_spiffe_id)?;
Ok(())
}
}

/// Server configuration.
///
/// Built programmatically in [`crate::Config::new`] and the gateway CLI from
Expand Down Expand Up @@ -297,6 +333,8 @@ pub struct Config {
/// JWTs as `Principal::Sandbox`. Required for the per-sandbox identity
/// flow (issue #1354).
pub gateway_jwt: Option<GatewayJwtConfig>,
/// Gateway-only access to SPIRE; shared supervisors receive access tokens.
pub spiffe_broker: Option<SpiffeBrokerConfig>,

/// Database URL for persistence.
pub database_url: String,
Expand Down Expand Up @@ -1083,6 +1121,7 @@ impl Config {
provider_profile_sources: vec![GatewayProviderProfileSourceConfig::User],
mtls_auth: MtlsAuthConfig::default(),
gateway_jwt: None,
spiffe_broker: None,
database_url: String::new(),
database_max_connections: None,
database_lock_max_connections: None,
Expand Down Expand Up @@ -2057,4 +2096,27 @@ mod tests {
"compiled:default"
);
}
#[test]
fn broker_configuration_rejects_ambiguous_transport() {
let mut config = super::SpiffeBrokerConfig {
spiffe_id_prefix: "spiffe://example.org/openshell".into(),
endpoint: "https://spire-broker.spire.svc:8443".into(),
server_spiffe_id: "spiffe://example.org/spire/agent/node".into(),
workload_api_socket_path: "/spiffe-workload-api/spire-agent.sock".into(),
};
config.validate("gw").unwrap();
for endpoint in [
"http://broker:8443",
"https://user:password@broker",
"https://broker/path",
"https://broker/?query",
"https://broker/#fragment",
] {
config.endpoint = endpoint.into();
assert!(config.validate("gw").is_err());
}
config.endpoint = "https://broker:8443".into();
config.workload_api_socket_path = "tcp:broker:1234".into();
assert!(config.validate("gw").is_err());
}
}
37 changes: 37 additions & 0 deletions crates/openshell-core/src/dynamic_credential_key.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,27 @@ pub fn credential_identity(key: &str) -> Option<&str> {
Some(decode(key)?.1)
}

/// The key a gateway encoded, recovered from an owned key the supervisor may
/// have scoped with [`ProviderCredentialSnapshot::scoped_key`].
///
/// Only the revision and installation are removed, so their spelling never
/// distinguishes two keys. Keys without an endpoint selector or owner, and
/// malformed keys, return `None`. The result still names a configured
/// credential only if it equals one of the gateway's keys exactly.
#[must_use]
pub fn unscoped_owned_key(key: &str) -> Option<String> {
decode(key)?.0?;
let fields = key.split('\t').collect::<Vec<_>>();
match fields.as_slice() {
[selector @ .., owner, identity] | [selector @ .., owner, _, _, identity]
if selector.len() == 3 =>
{
Some(format!("{}\t{owner}\t{identity}", selector.join("\t")))
}
_ => None,
}
}

fn decode(key: &str) -> Option<(Option<EndpointSelector<'_>>, &str)> {
// A stack buffer bounds parsing and avoids allocating on each request.
let mut fields = [""; 7];
Expand Down Expand Up @@ -314,6 +335,22 @@ mod tests {
Some("/v1/**")
);
assert_eq!(credential_identity(&scoped), Some("provider:access_token"));
// Only the generation is removed; its spelling cannot select another key.
assert_eq!(unscoped_owned_key(&scoped), Some(key()));
assert_eq!(unscoped_owned_key(&key()), Some(key()));
assert_eq!(
unscoped_owned_key(&scoped.replace("rev:7", "rev:007")),
Some(key())
);
for unowned in [
"provider:access_token",
"rev:7\tinstallation:install\tprovider:access_token",
"api.example.com\t443\t/v1/**\tprovider:access_token",
"api.example.com\t443\t/v1/**\trev:7\tinstallation:install\tprovider:access_token",
"api.example.com\t443\t/v1/**\towner-a\trev:x\tinstallation:install\tprovider:access_token",
] {
assert_eq!(unscoped_owned_key(unowned), None, "{unowned:?}");
}
assert_eq!(
snapshot.scoped_key("provider:access_token"),
"rev:7\tinstallation:install\tprovider:access_token"
Expand Down
Loading
Loading