Repository navigation
Description
User Story
I run OpenShell on Kubernetes and operate agent sandboxes (coding agents such as opencode against a local OpenAI-compatible inference endpoint) that need operator-managed configuration files and secret material projected into the workload. I directly encountered the missing ConfigMap/Secret volume support: I verified that the only way to project a Secret into a sandbox today is to run an extra sidecar pod sharing a PVC with the sandbox and mount the secret through it. That workaround is not what I want to operate.
Problem Statement
The typed Kubernetes driver config (volumes[] in --driver-config-json / SandboxTemplate.driver_config) only supports persistent_volume_claim. The resource-admission inventory explicitly rejects configMap and secret volume sources. There is no supported way to mount an existing namespace-local ConfigMap or Secret into the agent container, even though these are the standard Kubernetes primitives for configuration and secret-file projection.
Impact / Why This Matters
Users who need config files or secret files in a sandbox must today:
- operate a sidecar pod plus a shared PVC to copy or mount the material (extra moving parts, secret data transiting shared storage, broader RBAC surface), or
- bake configuration into the sandbox image (image rebuild for every config change), or
- use
--upload/--env, which puts secret material in the caller's hands instead of keeping it operator-managed.
All of these are insufficient for the common pattern "mount this ConfigMap as my agent's config directory and this Secret as read-only files, while API keys stay in the credential store." It blocks standard Kubernetes configuration workflows for agent workloads and pushes users toward precisely the sidecar topology OpenShell's in-process driver model is meant to avoid.
Proposed Design
Extend the Kubernetes driver-config volume attachment surface so a caller can select a namespace-local ConfigMap or Secret as a volume source, with the same admission guarantees as PVCs:
- Each volume selects exactly one source: PVC, ConfigMap, or Secret.
- ConfigMap and Secret volumes are always read-only, resolve only from the sandbox namespace, and support the usual
items(key/path/mode),default_mode, andoptionalprojections. - Resource admission inventories the new sources and resolves object metadata only; secret values never pass through the gateway. Operator approval labels apply exactly as for PVCs.
- Gateway RBAC grants metadata
geton ConfigMaps/Secrets under the sameallow_driver_configconditions as PVCs (sandbox Role in shared mode, workspace ClusterRole otherwise). - Stored attachment mounts are revalidated on restart/reattach, excluding driver-managed volumes.
Suggested UX
openshell sandbox create \
--from registry.example.com/team/agent:1.0 \
--driver-config-json '{
"kubernetes": {
"volumes": [
{"name": "agent-config", "config_map": {"name": "agent-config"}},
{"name": "agent-secrets", "secret": {"secret_name": "agent-extra"}}
],
"containers": {
"agent": {
"volume_mounts": [
{"name": "agent-config", "mount_path": "/etc/agent"},
{"name": "agent-secrets", "mount_path": "/etc/agent-secrets"}
]
}
}
}
}' \
-- agentBoth objects carry the usual admission labels (openshell.ai/sandbox-attachable, openshell.ai/sandbox-attachable-workspace). API keys and tokens that should be resolved and injected without caller-visible values continue to use credential providers.
Acceptance Criteria
-
--driver-config-jsonand sandbox templates acceptconfig_mapandsecretvolume sources; a volume with zero or multiple sources is rejected with a clear error. - ConfigMap/Secret mounts are read-only;
read_only=falsemounts are rejected. - Objects resolve only from the sandbox namespace; admission requires operator labels and reads metadata only.
- Key-to-path projections, octal modes (including protobuf Struct integral floats), and
optionalbehave like the Kubernetes API; invalid paths/modes are rejected. - Chart RBAC grants metadata get on configmaps/secrets only when caller driver config is enabled, in the correct Role/ClusterRole per workspace mode; unittests cover both.
- Restart/reattach revalidates caller attachment mounts without rejecting driver-managed volumes.
- Docs (
runtimes.mdx,configuration.mdx) describe the new sources and admission behavior. - Unit tests cover parsing, validation, rendering, and admission for the new sources.
Alternatives Considered
- Sidecar pod + shared PVC (current workaround): works but adds a pod, copies secret material through shared storage, and widens RBAC; operationally heavy for simple file projection.
--upload/--env: exposes secret material to the caller and CLI process; wrong boundary for operator-managed secrets.- Baking config into the image: rebuilds per change; no runtime operator control.
- Credential providers: the right tool for injectable API keys (and complementary to this proposal), but they do not project arbitrary config/secret files.
- PVC-only status quo: forces every config/secret projection through one of the workarounds above.
Agent Investigation
Static review at dc236df1 (main):
crates/openshell-driver-kubernetes/src/driver.rs:KubernetesDriverVolumeConfigcarries onlypersistent_volume_claim;validate_kubernetes_driver_volumesrequires a claim name;From<&KubernetesDriverVolumeConfig> for Volumeemits only a PVC source.crates/openshell-driver-kubernetes/src/resource_admission.rs: the closed-set inventory comment states Secret/ConfigMap volumes are rejected "instead of expanding gateway RBAC for attachments callers cannot request."deploy/helm/openshell/templates/role.yaml/clusterrole.yaml: metadata-only PVCgetgated onallow_driver_config(and non-shared mode for the ClusterRole) is the existing pattern to extend.
A prototype implementation with unit tests (driver parsing/validation/rendering/admission) and Helm unittest updates exists on a fork branch and can be submitted as a PR once this issue is accepted. It was validated end-to-end on a kind cluster: an opencode sandbox received its config via ConfigMap mount, extra material via Secret mount, and its inference API key via a Vault-backed provider with L7 egress injection.
Activity