Mount state to non-temporary directory - #474
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the Helm charts to configure Kafka app state directories under a non-temporary path and to mount the persistent volume at that same directory when running as a StatefulSet with persistence enabled.
Changes:
- Add
kafka.stateDirdefaults for streams, consumerproducer, and consumer charts. - Inject
KAFKA_STATE_DIRenv var fromkafka.stateDirinto each chart’s Deployment/StatefulSet template. - Mount the
datadirvolume atkafka.stateDir(instead of/tmp/...) and document the new value in chart READMEs.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| charts/streams-app/values.yaml | Adds default kafka.stateDir for streams state. |
| charts/streams-app/templates/deployment.yaml | Sets KAFKA_STATE_DIR and mounts datadir at the configured state dir. |
| charts/streams-app/README.md | Documents the new kafka.stateDir value. |
| charts/consumerproducer-app/values.yaml | Adds default kafka.stateDir for consumerproducer chart. |
| charts/consumerproducer-app/templates/deployment.yaml | Sets KAFKA_STATE_DIR and mounts datadir at the configured state dir. |
| charts/consumerproducer-app/README.md | Documents the new kafka.stateDir value. |
| charts/consumer-app/values.yaml | Adds default kafka.stateDir for consumer chart. |
| charts/consumer-app/templates/deployment.yaml | Sets KAFKA_STATE_DIR and mounts datadir at the configured state dir. |
| charts/consumer-app/README.md | Documents the new kafka.stateDir value. |
Suppressed comments (3)
charts/streams-app/templates/deployment.yaml:42
- volumeMounts.mountPath must be a string; rendering .Values.kafka.stateDir without quoting can produce invalid manifests if the value is not a plain path scalar. Quote it to force a string.
mountPath: {{ .Values.kafka.stateDir }}
charts/consumerproducer-app/templates/deployment.yaml:42
- volumeMounts.mountPath must be a string; rendering .Values.kafka.stateDir without quoting can produce invalid manifests if the value is not a plain path scalar. Quote it to force a string.
mountPath: {{ .Values.kafka.stateDir }}
charts/consumer-app/templates/deployment.yaml:30
- volumeMounts.mountPath must be a string; rendering .Values.kafka.stateDir without quoting can produce invalid manifests if the value is not a plain path scalar. Quote it to force a string.
mountPath: {{ .Values.kafka.stateDir }}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (3)
charts/streams-app/templates/deployment.yaml:29
KAFKA_STATE_DIR(i.e., Kafka configstate.dir) is set unconditionally, but thedatadirvolume is only mounted whenpersistence.enabled&&statefulSet. This changes behavior for non-persistent/non-stateful deployments and can point Kafka Streams at a directory that isn’t backed by any volume.
Consider only setting KAFKA_STATE_DIR when the datadir mount is present (same condition as the volumeMount), so state continues to use Kafka Streams’ default when not running statefully with persistence.
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}
charts/consumerproducer-app/templates/deployment.yaml:29
KAFKA_STATE_DIR(Kafka configstate.dir) is set unconditionally, but thedatadirvolume is only mounted whenpersistence.enabled&&statefulSet. This changes behavior for non-persistent/non-stateful deployments and can point the app at a state directory that isn’t backed by any volume.
Consider only setting KAFKA_STATE_DIR when the datadir mount is present (same condition as the volumeMount).
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}
charts/consumer-app/templates/deployment.yaml:19
KAFKA_STATE_DIR(Kafka configstate.dir) is set unconditionally, but thedatadirvolume is only mounted whenpersistence.enabled&&statefulSet. This changes behavior for non-persistent/non-stateful deployments and can point the app at a directory that isn’t backed by any volume.
Consider only setting KAFKA_STATE_DIR when the datadir mount is present (same condition as the volumeMount).
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (3)
charts/streams-app/README.md:60
kafka.stateDiris documented as a general setting, but the chart only applies it whenpersistence.enabledandstatefulSetare both true (it’s not passed asKAFKA_STATE_DIRotherwise). This can confuse users who setkafka.stateDirwithout enabling persistence/statefulset and see no effect.
| `kafka.stateDir` | Directory for Kafka Streams state. | `/state/kafka-streams` |
charts/consumerproducer-app/README.md:60
kafka.stateDiris documented as a general setting, but the chart only applies it whenpersistence.enabledandstatefulSetare both true (it’s not passed asKAFKA_STATE_DIRotherwise). Clarifying this avoids confusion for users running a plain Deployment.
| `kafka.stateDir` | Directory for app state. | `/state/kafka-consumer-producer` |
charts/consumer-app/README.md:60
kafka.stateDiris documented as a general setting, but the chart only applies it whenpersistence.enabledandstatefulSetare both true (it’s not passed asKAFKA_STATE_DIRotherwise). Consider documenting this condition explicitly.
| `kafka.stateDir` | Directory for app state. | `/state/kafka-consumer` |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Suppressed comments (3)
charts/streams-app/templates/deployment.yaml:45
mountPathis derived fromkafka.stateDir, but users can also setkafka.config["state.dir"](which becomesKAFKA_STATE_DIR) via the sharedcommon-envhelper. If those diverge, the app will write state somewhere that isn't on the mounted volume. Consider making the mountPath followkafka.config["state.dir"]when provided.
- name: datadir
mountPath: {{ .Values.kafka.stateDir | quote }}
{{- end }}
charts/consumerproducer-app/templates/deployment.yaml:44
mountPathis derived fromkafka.stateDir, but users can also setkafka.config["state.dir"](which becomesKAFKA_STATE_DIR) via the sharedcommon-envhelper. If those diverge, the app may write state somewhere that isn't on the mounted volume. Consider making the mountPath followkafka.config["state.dir"]when provided.
- name: datadir
mountPath: {{ .Values.kafka.stateDir | quote }}
charts/consumer-app/templates/deployment.yaml:32
mountPathis derived fromkafka.stateDir, but users can also setkafka.config["state.dir"](which becomesKAFKA_STATE_DIR) via the sharedcommon-envhelper. If those diverge, the app may write state somewhere that isn't on the mounted volume. Consider making the mountPath followkafka.config["state.dir"]when provided.
- name: datadir
mountPath: {{ .Values.kafka.stateDir | quote }}
9b09a79 to
eadb255
Compare
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (6)
charts/streams-app/templates/deployment.yaml:42
mountPathis derived from.Values.kafka.stateDir, but users can also override the effective Kafka Streams state directory viakafka.config["state.dir"](which becomesKAFKA_STATE_DIR). If those diverge, the app will write state outside the mounted volume.
mountPath: {{ .Values.kafka.stateDir | quote }}
charts/consumerproducer-app/templates/deployment.yaml:42
mountPathis derived from.Values.kafka.stateDir, but users can also overridestate.dirviakafka.config["state.dir"](rendered intoKAFKA_STATE_DIRbykafka-app.common-env). If these diverge, the app will write state outside the mounted volume.
mountPath: {{ .Values.kafka.stateDir | quote }}
charts/consumer-app/templates/deployment.yaml:30
mountPathis derived from.Values.kafka.stateDir, but users can also overridestate.dirviakafka.config["state.dir"](rendered intoKAFKA_STATE_DIRbykafka-app.common-env). If these diverge, the app will write state outside the mounted volume.
mountPath: {{ .Values.kafka.stateDir | quote }}
charts/streams-app/templates/deployment.yaml:29
kafka-app.common-envalready rendersKAFKA_*env vars from.Values.kafka.config(includingstate.dir->KAFKA_STATE_DIR). AddingKAFKA_STATE_DIRhere unconditionally can create duplicate env var entries when users setkafka.config["state.dir"], making the effective value unclear.
This issue also appears on line 42 of the same file.
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}
charts/consumerproducer-app/templates/deployment.yaml:29
kafka-app.common-envalready rendersKAFKA_*env vars from.Values.kafka.config(includingstate.dir->KAFKA_STATE_DIR). AddingKAFKA_STATE_DIRhere unconditionally can create duplicate env var entries when users setkafka.config["state.dir"], making the effective value unclear.
This issue also appears on line 42 of the same file.
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}
charts/consumer-app/templates/deployment.yaml:19
kafka-app.common-envalready rendersKAFKA_*env vars from.Values.kafka.config(includingstate.dir->KAFKA_STATE_DIR). AddingKAFKA_STATE_DIRhere unconditionally can create duplicate env var entries when users setkafka.config["state.dir"], making the effective value unclear.
This issue also appears on line 30 of the same file.
- name: KAFKA_STATE_DIR
value: {{ .Values.kafka.stateDir | quote }}



No description provided.