Skip to content

bug(helm): appVersion build metadata produces invalid Kubernetes labels #5692

Closed

Description

@freeqaz-openai

User Story

I use OpenShell's Helm chart to deploy gateways with digest-pinned images and chart app versions that include SemVer build metadata. I encountered an upgrade that Kubernetes rejected before the gateway rollout.

Problem Statement

When Chart.appVersion contains +, the common chart labels render that value unchanged as app.kubernetes.io/version. Kubernetes does not permit + in a label value, so resources using the common labels are invalid.

Impact / Why This Matters

The certificate-generation pre-upgrade ServiceAccount is rejected and Helm records a failed upgrade before applying ordinary gateway resources. Operators must repackage the chart to continue.

Acceptance Criteria

  • Packaging the chart with an app version such as 0.1.9+build.1 produces valid Kubernetes labels on ordinary resources and hooks.
  • The chart's app version and explicitly selected image tags or digests retain their original meaning.

Reproduction Steps

  1. From the OpenShell checkout, run helm package deploy/helm/openshell --version 0.1.9-build.1 --app-version '0.1.9+build.1'.
  2. Run helm template test ./helm-chart-0.1.9-build.1.tgz --set agentSandbox.preflight.enabled=false | grep 'app.kubernetes.io/version'.
  3. Observe the rendered label value 0.1.9+build.1. A real install or upgrade rejects the certgen ServiceAccount with an invalid metadata label error.

Environment

  • OpenShell: v0.1.9; the same label expression remains on the current default branch.
  • Deployment: Kubernetes Helm chart.
  • Helm: v3.19.2.

Logs

ServiceAccount "...-certgen" is invalid: metadata.labels: Invalid value: "0.1.9+build.1"

Activity

  1. TaylorMutch commented on Nov 18, 2026

    @TaylorMutch
    Collaborator

    📋 triage-agent

    Confirmed chart bug. The report includes first-hand Helm upgrade context, a concrete failure, environment details, and observable acceptance criteria.

    Reproduced from checkout 624aceea3 with Helm v4.2.0: packaging with --version 0.1.9-build.1 --app-version '0.1.9+build.1' and rendering offline with agentSandbox.preflight.enabled=false produces 19 app.kubernetes.io/version: "0.1.9+build.1" labels. The pre-install/pre-upgrade certgen ServiceAccount is among the affected resources. These values violate Kubernetes label syntax because + is forbidden. The same unmodified .Chart.AppVersion | quote expression remains on GitHub main in deploy/helm/openshell/templates/_helpers.tpl.

    This verifies the invalid manifests; a live Kubernetes upgrade was not attempted. The existing chart-version label helper already replaces + with _. Image helpers resolve tags/digests separately, so the repair can stay at the common-label boundary. Middleware and gateway extensions cannot repair objects rejected before rollout. No duplicate or open implementation was found in the issue/PR searches, and the issue has no assignee.

    Accepted at the maintainer's explicit request in this session. Scope: valid version labels on ordinary resources and hooks while preserving chart metadata and image-selection behavior. The implementation plan follows separately.

  2. TaylorMutch commented on Nov 18, 2026

    @TaylorMutch
    Collaborator

    🏗️ build-plan

    Fix the chart's common version label so SemVer build metadata cannot block gateway installs or upgrades. Keep this change scoped to #5692.

    1. In deploy/helm/openshell/templates/_helpers.tpl, normalize .Chart.AppVersion only when emitting app.kubernetes.io/version: replace + with _, cap the value at 63 characters, and trim boundary punctuation (-, _, .) so truncation cannot leave an invalid ending. Continue quoting the result and omitting the label when appVersion is empty. For example, 0.1.9+build.1 becomes 0.1.9_build.1. Reuse the common labels already consumed by workloads, RBAC, Services, and certgen hooks; avoid per-template fixes.

    2. Add focused helm-unittest coverage under deploy/helm/openshell/tests/ for the reported version on an ordinary resource and certgen hook resources, unchanged normal/prerelease versions, and a long build-metadata version whose truncation reaches punctuation. Check the rendered value, allowed character pattern, and 63-character bound. Verify an empty appVersion still omits the label. Extend the existing image-reference coverage as needed to demonstrate that explicit tags, digest precedence, and appVersion-based fallback are preserved. Do not rewrite appVersion or image references to the normalized label value; + in an implicit image tag is a separate image-reference concern, and digest-pinned/explicit-tag deployments remain the supported reproduction here.

    3. Reproduce against a packaged chart, preserving its archived Chart.yaml appVersion. Example:

      helm package deploy/helm/openshell --version 0.1.9-build.1 --app-version '0.1.9+build.1' --destination /tmp/openshell-5692-check
      helm template test /tmp/openshell-5692-check/helm-chart-0.1.9-build.1.tgz --set agentSandbox.preflight.enabled=false --set gateway.image.tag=stable-test

      Validate all rendered metadata/pod-template version labels, including certgen hook RBAC and Jobs; repeat with explicit image digests and assert references are unchanged. Run mise run helm:lint and mise run helm:test. Qualify the relevant Kubernetes install/upgrade E2E path with digest-pinned images and the metadata-bearing appVersion, confirming the pre-upgrade certgen hook passes and the gateway becomes ready. Rendering alone does not establish successful upgrade behavior.

    4. Review skills/debug-openshell-cluster/SKILL.md and related workflow references through sync-agent-infra, as required for gateway infrastructure changes; make only a minimal applicable troubleshooting update. No new chart values, CLI flags, gateway TOML fields, or broad documentation changes are needed.

    Alternative considered: stripping build metadata or modifying Chart.yaml/image helpers. Normalizing only the label preserves the requested chart and image identities and matches the existing chart-label convention. No extension point applies before Kubernetes validates these resources.

    Completion: ordinary resources and hooks have valid labels for 0.1.9+build.1; packaged appVersion and explicitly selected image tags/digests retain their original values; relevant Helm checks and Kubernetes upgrade qualification pass. This request authorizes planning only; implementation has not started.

  3. added
    area:platformSDKs, CLIs, APIs, gateway infrastructure, persistence, configuration, and session management.
    topic:kubernetesKubernetes operators, CRDs, deployment controllers, and native Kubernetes API integration.
    topic:operabilityInstalling/configuring deployments, upgrades, backup/restore, diagnostics, and maintenance.
    and removed
    state:triage-neededOpened without agent diagnostics and needs triage
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:platformSDKs, CLIs, APIs, gateway infrastructure, persistence, configuration, and session management.state:acceptedA maintainer decided OpenShell should pursue this issuetopic:kubernetesKubernetes operators, CRDs, deployment controllers, and native Kubernetes API integration.topic:operabilityInstalling/configuring deployments, upgrades, backup/restore, diagnostics, and maintenance.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions