fix(hosting): deploy ClickHouse from the official image instead of Bitnami#4249
fix(hosting): deploy ClickHouse from the official image instead of Bitnami#4249matt-aitken wants to merge 3 commits into
Conversation
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📜 Recent review details⏰ Context from checks skipped due to timeout. (6)
WalkthroughClickHouse configuration is updated across self-hosting documentation, Docker Compose, and Helm. Docker now uses the official image, updated credentials and paths, file limits, and a compatible data-path configuration. Helm replaces the Bitnami dependency with custom ClickHouse resources, pinned image settings, explicit ports, probes, persistence, security settings, generated URLs, and updated tests. Documentation records the ClickHouse 25.8+ requirement and external service port key. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🧭 Helm Chart Prerelease PublishedVersion: Install: helm upgrade --install trigger \
oci://ghcr.io/triggerdotdev/charts/trigger \
--version "4.5.4-pr4249.c53b098"
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…tnami The Bitnami free image catalog is EOL and its frozen bitnamilegacy archive tops out at ClickHouse 25.7.5, below the 25.8 floor the platform requires since v4.5.0. The Docker Compose stack and the Helm chart now run the official clickhouse/clickhouse-server image at 26.2, the same version the platform is developed and tested against. The Helm chart deploys ClickHouse with a chart-owned StatefulSet instead of the Bitnami subchart. Existing deployments keep their data with no manual steps: a config override keeps the on-disk layout compatible with volumes created by the Bitnami-based setup, Compose reuses the same named volume, and the Helm chart automatically adopts the data PVC left behind by the old subchart.
matt-aitken
force-pushed
the
fix/self-hosting-official-clickhouse
branch
from
e2e7e33 to
4608012
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…e inline credentials The chart-deployed ClickHouse now reads CLICKHOUSE_PASSWORD from a chart-owned Secret instead of a plaintext env value in the pod spec, and the CLICKHOUSE_URL helpers percent-encode inline usernames and passwords so special characters no longer produce an unparseable URL.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ntials urlquery encodes spaces as plus signs, which URL userinfo decoding keeps literal. A shared urlencode helper rewrites them to %20 (a real plus already encodes as %2B), so passwords containing spaces now work.
| template: | ||
| metadata: | ||
| annotations: | ||
| checksum/config: {{ .Values.clickhouse.configdFiles | toYaml | sha256sum }} |
There was a problem hiding this comment.
🟡 Config change detection misses the default data-paths file, so pods may not restart after a chart upgrade
The config checksum annotation only covers user-supplied override files (configdFiles | toYaml | sha256sum at hosting/k8s/helm/templates/clickhouse.yaml:76), but the ConfigMap also includes a hardcoded default data-paths.xml that is not part of configdFiles, so changes to that default in a future chart version won't alter the checksum and the StatefulSet pod won't be restarted.
Impact: After a chart upgrade that changes the default data-paths layout, ClickHouse keeps running with the old paths until manually restarted, because subPath volume mounts do not auto-update.
Mechanism: subPath mounts require pod restart, but checksum doesn't trigger one
The ConfigMap is built from two sources (hosting/k8s/helm/templates/clickhouse.yaml:10-32):
- A hardcoded
data-paths.xml(when not overridden by the user) - All entries from
.Values.clickhouse.configdFiles
But the checksum annotation at line 76 only hashes source (2):
checksum/config: {{ .Values.clickhouse.configdFiles | toYaml | sha256sum }}The config files are mounted via subPath (hosting/k8s/helm/templates/clickhouse.yaml:152-153), which means Kubernetes does NOT propagate ConfigMap updates to the running pod — a pod restart is required. Since the checksum doesn't change when only the hardcoded default changes, Helm won't roll the StatefulSet.
The fix would be to include the full rendered ConfigMap data in the checksum, or at minimum include the hardcoded data-paths.xml content in the hash.
Prompt for agents
In hosting/k8s/helm/templates/clickhouse.yaml line 76, the checksum/config annotation only hashes .Values.clickhouse.configdFiles but the ConfigMap also includes a hardcoded data-paths.xml (lines 10-27) that is not part of configdFiles. When this hardcoded content changes in a future chart version, the checksum won't change and the pod won't be restarted, even though subPath mounts require a restart to pick up ConfigMap changes.
To fix, compute the checksum over the full ConfigMap data rather than just the user-supplied configdFiles. One approach is to include the hardcoded data-paths.xml content in the hash as well. For example, concatenate a representation of the default data-paths.xml with the configdFiles YAML before hashing, or use a checksum of the entire rendered ConfigMap data block.
Was this helpful? React with 👍 or 👎 to provide feedback.
| # Production ClickHouse | ||
| # The bundled ClickHouse serves plain HTTP inside the cluster. For TLS, | ||
| # use an external ClickHouse (deploy: false) with secure: true (see below). | ||
| clickhouse: |
There was a problem hiding this comment.
🔍 Production example removed secure: true for bundled ClickHouse — intentional but worth noting
The production example at hosting/k8s/helm/values-production-example.yaml:73-75 removed secure: true from the ClickHouse section and added a comment explaining that the bundled ClickHouse serves plain HTTP inside the cluster. This is correct — the bundled single-node ClickHouse doesn't have TLS configured, so secure: true would have generated HTTPS URLs that the server can't serve. The comment directs users to use an external ClickHouse with deploy: false and secure: true for TLS. However, users upgrading who previously had secure: true in their custom values (copied from the old production example) would need to either remove it or set up an external ClickHouse with TLS.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
Self-hosted deployments now run ClickHouse from the official
clickhouse/clickhouse-serverimage instead ofbitnamilegacy/clickhouse. Bitnami's free image catalog is EOL and the frozen legacy archive tops out at ClickHouse 25.7.5, below the 25.8 minimum the platform requires since v4.5.0, which broke every ClickHouse insert on chart-bundled deployments. Both stacks now default to 26.2, the same version the platform is developed and tested against.Existing deployments keep their ClickHouse data with no manual migration.
Fixes #4197.
Details
Docker Compose: the
clickhouseservice uses the official image with its native env vars, plus the recommendednofileulimits. It reuses the same named volume as before: adata-paths.xmlconfig override points ClickHouse at thedata/subdirectory of the volume, which is exactly the layout the Bitnami image used, so old volumes work in place (including SQL-created users) and fresh installs get the identical layout.Helm chart: the Bitnami ClickHouse subchart is replaced by a chart-owned single-node StatefulSet and Service running the official image (non-root, HTTP
/pingprobes, config overrides mounted intoconfig.d, and the samedata-paths.xmllayout compatibility). On upgrade, the chart automatically adopts the data PVC left behind by the old subchart (data-<release>-clickhouse-shard0-0) vialookup, andfsGrouprelabeling handles the uid change on first mount. Existingclickhouse.*values keep working:auth,persistence(includingglobal.storageClass),resources,secure,external.*, andconfigdFiles. Bitnami-only keys (shards,replicaCount,keeper,resourcesPreset) are gone. The docs now state the 25.8 minimum for bring-your-own ClickHouse.One caveat:
lookupreturns nothing when manifests are rendered without cluster access (GitOps tools that usehelm template). For that case there's a newclickhouse.persistence.existingClaimvalue, documented in the values file and the Kubernetes self-hosting docs.Verification
main(Bitnami 25.5), wrote 5,000 rows, then brought the same project up with this branch's compose file. The official 26.2 server came up healthy on the same volume with all rows intact, SQL-created users working, and writes succeeding.helm lint,helm template(default values,existingClaimset, external ClickHouse, and the production example) and kubeconform all pass, mirroring the release CI steps.input_format_json_infer_array_of_dynamic_from_array_of_different_types(the setting that fails on 25.7.5) succeed on the upgraded volume.