Skip to content

ci: check_spec_sync.sh reports "Spec sync OK" when it is run with no input at all #655

Description

@Yaraslaut

Found while running the pre-push gate list for #636/#648 (PR #654), filed rather than folded in.

The finding

bash scripts/check_spec_sync.sh with nothing on stdin prints a green tick and exits 0:

$ echo -n "" | bash scripts/check_spec_sync.sh
Spec sync OK: the change touches no files.
EXIT=0

The branch responsible is scripts/check_spec_sync.sh:81-86:

changed="$(cat)"

if [ -z "${changed//[[:space:]]/}" ]; then
    echo "Spec sync OK: the change touches no files."
    exit 0
fi

That is correct for the CI workflow, which always feeds it a real path list, and an empty list there genuinely means an empty diff. It is wrong for a human or an agent at a terminal, because the script's documented usage is a pipeline:

# Usage: git diff --name-only <base> HEAD | bash scripts/check_spec_sync.sh

and the failure mode of a pipeline is that someone runs the right-hand side alone. Typing bash scripts/check_spec_sync.sh hangs on cat reading the terminal, and one Ctrl-D — the reflex when a command appears to have stalled — produces the line above. The message says "the change touches no files", which reads as a statement about the working tree; what it actually measured is that stdin was empty.

This is the shape AGENTS.md names first: a control that reports success while measuring nothing. The other four gates in the same pre-push list (check_mutation_survivors.py, check_nolint_directives.sh, check_bidi_controls.py, check_spec_citations.sh) all discover their own inputs and print a count of what they scanned, so none of them can be run wrongly in this way. check_spec_sync.sh is the only one whose correct invocation a caller can silently omit half of.

Why it matters here specifically

The gate exists because of morph#560, where a sub-domain missing from a word list was "not merely unchecked, it is indistinguishable from one that was deliberately exempted". A green tick from an empty pipe is the same defect one level up: a run that was not merely unperformed, but is indistinguishable from one that passed.

It is also a live hazard rather than a theoretical one, because a green check_spec_sync.sh is exactly the evidence someone would cite for "the spec is in sync" in a PR body.

Verification status

Reproduced, on branch laneCORE2-batch-636-648 at 4ef26fd8 (three commits on 0067b5bf), Linux, bash. Both outputs are real terminal output, not paraphrased:

$ echo -n "" | bash scripts/check_spec_sync.sh
Spec sync OK: the change touches no files.
EXIT=0

$ git diff --name-only origin/master HEAD | bash scripts/check_spec_sync.sh
Spec sync OK: 10 sub-domain(s) classified; every touched header sub-domain has a matching spec change.

The second is the same script on the same revision with its input supplied, and it is the run the branch's claim actually rests on.

What I did not verify:

  • That anyone has actually been misled by it. I have no evidence of a PR whose spec-sync claim came from an empty pipe; this is a reachable failure mode, not a measured incident. Weak evidence, labelled as such.
  • The CI path. .github/workflows/spec-sync.yml was not exercised here — I have only read that it pipes a path list in, so the workflow is not affected either way.
  • scripts/test_check_spec_sync.sh was not run, so I do not know whether it already covers the empty-input case deliberately (in which case this is a decision to revisit, not an oversight).

What would change the verdict

  • Close as invalid if the empty-input early exit is load-bearing for a CI path that legitimately supplies an empty list and must pass — then the fix is only in the wording, not the exit code.
  • Otherwise the smallest honest repair is to make the two cases distinguishable: exit 0 with a message that says stdin was empty and names the correct invocation, or read the diff itself when stdin is a TTY. Either removes the ambiguity; "Spec sync OK" for an unmeasured run is what should not survive.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciSubsystem: cibugSomething isn't workingtriage: validWell-framed; implement as written

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions