Skip to content

fix(cli): kates conformity lint refuses a key set twice to the same value, as the check does - #437

Merged
bmscomp merged 1 commit into
mainfrom
fix/conformity-duplicate-key
Oct 20, 2026
Merged

bmscomp merged 1 commit into
mainfrom
fix/conformity-duplicate-key

Conversation

@bmscomp

@bmscomp bmscomp commented Oct 20, 2026

Copy link
Copy Markdown
Owner

Why

kates conformity lint and kates conformity check disagreed about a key set twice to the same value. With count: 3 twice under spec.brokers, lint printed a duplicate-key warning and exited 0. Check printed the same warning, then failed with exit 2 before it called the API:

yaml: unmarshal errors: mapping key "count" already defined

That error comes from conformityJSON, which decodes into any, where yaml.v3 refuses a repeated key. The Kates API refuses one too (STRICT_DUPLICATE_DETECTION in KafkaReferenceReader). So the CLI Reference's promise that a lint warning doesn't stop the check was false for this case.

What changes

duplicate-key is now an error, still fixable: kates conformity fmt keeps the first, as it already does for duplicate-item (also an error that fmt fixes). Lint now refuses what check and the API refuse, and check stops at lint's own message instead of a yaml.v3 one.

I chose this over letting conformityJSON accept the repeat and send the first value. That would have made check send something the API refuses from any other client, and kept lint and the API apart.

  • cli/internal/conformity/compile.go: severity and the duplicates() comment.
  • TestDuplicates expects an error. TestConformityLintPassesWithWarningsOnly now uses a duplicate-value warning, which stays a warning because the API accepts a repeated list value.
  • TestConformityCheckStopsOnALintErrorBeforeTheAPI also covers a same-value duplicate key. It asserts lint exits 1, and that check exits 2 with lint's message, without calling the API. Without the fix it fails with the yaml.v3 error above.
  • CLI Reference: lint says a key set twice is an error (duplicate-key or conflicting-key), and fmt says it fixes duplicate-key, duplicate-item and duplicate-value.

Deliberately not done

  • No shared-corpus file. testdata/invalid is shared with KafkaReferenceCorpusTest, whose json() helper converts YAML with YAMLMapper().readTree. That keeps the last value of a repeated key (checked with the 2.18 jars: count: 3 twice becomes {"count":3}), so the reader never sees the duplicate and the Java test would fail. The CLI tests cover the agreement instead.
  • The Conformity Language Reference appendix (appendix-conformity-language.md) isn't on main yet. Its duplicate-key sentence in "The Document" and its duplicate-key row need the same change. The exact wording is in a hand-over note for the session writing it.

Tests

Run locally:

  • cli: go test ./..., go vet ./..., golangci-lint run --new-from-rev=origin/main ./... (0 issues), govulncheck ./...
  • scripts/check-cli-style.sh, bash scripts/check-cli-compat.sh
  • kates: ./mvnw test -Dquarkus.grpc.codegen.skip=true -Dquarkus.http.test-port=0 (cli/** triggers the backend's Build & Test). 1,490 tests, 0 failures, 1 skipped; KafkaReferenceCorpusTest 51/51.
  • scripts/check-book-style.sh, gen-version-matrix.sh --check, gen-chart-table.sh --check, check-book-contrast.py, check-mermaid.sh
  • Scratch-copy HTML render, then check-book-fragments.py (7,738 fragment links land)

Left to CI: the PDF render (the two added sentences are ASCII with code spans), *IT tests, the native image and Docker builds.

…alue, as the check does

A key set twice to the same value was a lint warning, so lint exited 0,
but kates conformity check then failed with exit 2 before it called the
API: conformityJSON decodes into any, and yaml.v3 refuses the repeated
key there. The Kates API refuses a repeated JSON key too
(STRICT_DUPLICATE_DETECTION). duplicate-key is now an error, still
fixable: kates conformity fmt keeps the first, as for duplicate-item.

The CLI tests hold lint and check to the same outcome for it. The shared
corpus can't: KafkaReferenceCorpusTest converts YAML with readTree,
which keeps the last value, so the reader never sees the repeat. The CLI
Reference says lint refuses a key set twice and fmt fixes it.
@bmscomp
bmscomp merged this pull request into main Oct 20, 2026
58 checks passed
bmscomp added a commit that referenced this pull request Oct 21, 2026
…alue, as the check does (#437)

A key set twice to the same value was a lint warning, so lint exited 0,
but kates conformity check then failed with exit 2 before it called the
API: conformityJSON decodes into any, and yaml.v3 refuses the repeated
key there. The Kates API refuses a repeated JSON key too
(STRICT_DUPLICATE_DETECTION). duplicate-key is now an error, still
fixable: kates conformity fmt keeps the first, as for duplicate-item.

The CLI tests hold lint and check to the same outcome for it. The shared
corpus can't: KafkaReferenceCorpusTest converts YAML with readTree,
which keeps the last value, so the reader never sees the repeat. The CLI
Reference says lint refuses a key set twice and fmt fixes it.
bmscomp added a commit that referenced this pull request Oct 21, 2026
…alue, as the check does (#437)

A key set twice to the same value was a lint warning, so lint exited 0,
but kates conformity check then failed with exit 2 before it called the
API: conformityJSON decodes into any, and yaml.v3 refuses the repeated
key there. The Kates API refuses a repeated JSON key too
(STRICT_DUPLICATE_DETECTION). duplicate-key is now an error, still
fixable: kates conformity fmt keeps the first, as for duplicate-item.

The CLI tests hold lint and check to the same outcome for it. The shared
corpus can't: KafkaReferenceCorpusTest converts YAML with readTree,
which keeps the last value, so the reader never sees the repeat. The CLI
Reference says lint refuses a key set twice and fmt fixes it.
bmscomp added a commit that referenced this pull request Oct 21, 2026
…dminless plan with what shipped (#444)

The book teaches the conformity language in its Conformity Checks
chapter, but had no place that lists every field, value kind, rule id,
report field and lint code. A new appendix does, after the version
matrix; the chapter, the CLI Reference, the Reference part's intro and
the README point to it.

It is written against main as of #443:
- a key set twice to the same value is a lint error that fmt fixes (#437);
- with rbac.scope=namespace, zones, skew and rack awareness read the
  EndpointSlices of the brokers' headless Service when nodes can't be
  listed, and podMonitor looks in the Kafka namespace only (#439);
- a refused get of the Kafka resource is a 503 naming the grant (#439),
  which the API Reference now says too;
- the chart sets the checked namespace and cluster from
  networkPolicy.kafka.namespace and kafka.clusterName (#441).

plans/adminless-kubernetes.md is the plan behind #439, with a status
block naming what shipped in #439 and #441-#443, where the work departed
from it (EndpointSlice zones; orphan recovery left on), and what is
still open.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant