Skip to content

CI: report script and doc check findings on the changed lines - #3773

Merged
jimklimov merged 5 commits into
networkupstools:masterfrom
nmbro:ci-script-doc-checks
Oct 18, 2026
Merged

jimklimov merged 5 commits into
networkupstools:masterfrom
nmbro:ci-script-doc-checks

Conversation

@nmbro

@nmbro nmbro commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Make the script and documentation checks report problems where they are, on the changed lines, instead of somewhere in a long CI log, and add a ShellCheck check for the shell scripts that a PR changes.

Problem

  • Spellcheck errors are hard to find. make spellcheck printed unknown words with egrep -b byte offsets into the aspell output (e.g. 18011:& backticks 13 16: ...), not with a location in the source document. In CI, the failure is somewhere in the step log.
  • No real ShellCheck. make shellcheck is only a syntax check (sh -n/bash -n) with the system /bin/sh. Running the real shellcheck on the tree gives about 2900 findings (28 errors, 258 warnings), mostly deliberate portability idioms or old code, so checking the whole tree would always fail.
  • make shellcheck noise and coverage (Ignore $(...) vs backtick syntax warning in make shellcheck #1058, Extend make shellcheck to support our SHELL_PROGS envvar from some of the tests #1059): ksh93 (the /bin/sh on illumos and Solaris) prints a "... obsolete, use $(...)" warning for every backtick in syntax-check mode: 1442 of them for 47 scripts on current master, which buried the 15 other warnings it had. And there was no way to check with other shells than /bin/sh.
  • aspell resources on CentOS 7 (CI: More fixes for aspell usage on stranger platforms #2068): configure did not look in lib64, nor in aspell-0.60 style directories for a 4-component version like 0.60.6.1.

Changes

  1. build: SHELL_PROGS for make shellcheck -- make shellcheck (check-scripts-syntax) checks POSIX shell scripts with each interpreter listed in SHELL_PROGS (default still /bin/sh), like the NDE self-test already does; bash scripts are still checked with bash, and the error names the shell. The backticks warning is filtered out (keeping the syntax check exit code). Closes Ignore $(...) vs backtick syntax warning in make shellcheck #1058, closes Extend make shellcheck to support our SHELL_PROGS envvar from some of the tests #1059.
  2. ci: add ShellCheck check for changed scripts -- a new "GHA-07: ShellCheck" workflow for PRs runs tools/shellcheck-ci.pl, which checks the base and PR versions of the changed shell scripts and reports only new findings, matched by file, ShellCheck code and source line text (so moved code does not count as new). New "warning"/"error" findings fail the check; new "note" (info/style) ones are shown as notices. Findings show as annotations on the changed lines and in the job summary. The helper is Perl (like tools/check-source-nonascii.pl; Perl is already needed by autogen.sh in Git workspaces), uses only core Perl 5 modules and the gcc output format of shellcheck (no JSON module needed), and can be run locally: perl tools/shellcheck-ci.pl origin/master. A new .shellcheckrc disables the checks for deliberate NUT portability idioms: backticks (SC2006, the ShellCheck side of Ignore $(...) vs backtick syntax warning in make shellcheck #1058), x"$var" comparisons (SC2268) and expr (SC2003); it is distributed like .editorconfig. The workflow uses only actions/checkout, read-only permissions, and compares the PR merge commit with its first parent.
  3. docs: locate spellcheck errors, annotate in GHA -- unknown words are reported as file:line:col: <aspell result line>, with paths relative to the source root. In GitHub Actions they also become annotations on the source lines and a table in the job summary, each word once (the top-level recipe collects them, because the sequential retry after a failed parallel attempt and the ci_build.sh re-runs report the same errors again, and annotations per step are limited). No annotations with SPELLCHECK_ERROR_FATAL=no (the distcheck steps).
  4. m4: find aspell resources in lib64 and X.Y dirs -- also try ../lib64 and the major.minor version directory. The other part of CI: More fixes for aspell usage on stranger platforms #2068 (only enable spellcheck if aspell actually works) was already done by the live check from e519320, verified with a missing dictionary. Closes CI: More fixes for aspell usage on stranger platforms #2068.

Docs: docs/qa-guide.adoc (shell script and spelling checks sections), NEWS.adoc.

Example from the field

A test run of this branch on my fork showed what this is for: the "Spellcheck with old aspell-en 7.1 (as on CentOS 7)" job, which then failed for every PR, annotated docs/man/microdowell.txt:42:14 with initialisation from #3744, which is not in the old dictionary (commented there, since fixed by 3cb4c2f). Without this change, finding that needed a read through the job log.

Screenshots

From a test PR on my fork with deliberate findings (not part of this PR):
file selection shows warnings

shellcheck error on spellcheck

Testing

  • tools/shellcheck-ci.pl: the first version of build: keep full git version out of config.h (ccache-friendly) #3759 is reported for its one new SC2166 warning ([ p -a q ]), the final version passes; renamed, added, removed and non-shell files; a bad base revision (exit 2); local and GitHub Actions output modes.
  • make shellcheck SHELL_PROGS="dash ksh bash" (ksh 93u+m/1.0.8): passes, no backticks warnings left, other ksh warnings still shown; syntax errors in a POSIX or bash script fail, naming the file and shell.
  • Spellcheck with typos in NEWS.adoc, docs/man/upsc.txt and scripts/upsdrvsvcctl/README.adoc: correct file, line and column; each annotation and summary row once, also via BUILD_TYPE=default-spellcheck ./ci_build.sh; none with SPELLCHECK_ERROR_FATAL=no or without GITHUB_ACTIONS.
  • configure with a fake CentOS 7 aspell layout (version 0.60.6.1, resources in lib64/aspell-0.60): found where master found none, and make spellcheck passes with that --filter-path; unchanged paths on Ubuntu.
  • Test PR on my fork: annotations and job summaries for ShellCheck and both spellcheck jobs (current and old aspell-en).
  • actionlint 1.7.7 and zizmor 1.25.2 on the workflow, make spellcheck, make distcheck-light, ASCII-only check on the changes.

Open questions

  • The new .shellcheckrc dotfile in the source root, and its list of disabled checks: are there other idioms that should be in it?
  • Should new ShellCheck "warning" findings fail the check, or only "error" ones at first?

Checklist

  • Described the changes in the PR submission
  • Multiple commits, each with a functional change
  • Use of coding helper tools and AI disclosed: Claude Code was used for development assistance
  • Revised for NUT code style and ASCII-only text
  • For newly added files, the Makefile.am recipes were updated and make distcheck-light passes
  • Added bullet points into NEWS.adoc
  • Passed make spellcheck, updated docs/nut.dict (no words removed)

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

A ZIP file with standard source tarball and another tarball with pre-built docs for commit ce43b09 is temporarily available: NUT-tarballs-PR-3773.zip.

nmbro and others added 4 commits October 15, 2026 22:57
Let "make shellcheck" (check-scripts-syntax) check the POSIX shell
scripts with each interpreter listed in the SHELL_PROGS envvar, like
the NDE self-test already does, rather than only the system /bin/sh.
Bourne-Again shell scripts are still checked with bash. The shell is
named in the error message, so a failure is easy to tell apart.

Also hide the "`...` obsolete, use $(...)" warnings that e.g. ksh93
(the /bin/sh on illumos and Solaris) prints in syntax-check mode for
each use of backticks, which NUT scripts use deliberately for
portability: on current master, "ksh -n" printed 1442 of these for 47
scripts, which buried the 15 other warnings it had. The output is
captured before filtering, so the syntax check exit code is kept.
The filter uses $(EGREP) with bracket expressions, rather than
"grep -F", for older grep implementations.

Checked with SHELL_PROGS="dash ksh bash" (ksh 93u+m/1.0.8): all
scripts pass, no backticks warnings are left, the other ksh warnings
are still shown; and that syntax errors in a POSIX or a Bourne-Again
shell script fail the check, naming the file and the shell. Spellcheck
and distcheck-light pass.

Closes: networkupstools#1058
Closes: networkupstools#1059

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: nmbro <nmbro@users.noreply.github.com>
Add a "GHA-07: ShellCheck" workflow for pull requests, which runs
ShellCheck on the shell scripts changed by the PR. The existing scripts
have many known findings (about 2900 in the tree, 28 of them errors),
so checking the whole tree would always fail. Instead, the new
tools/shellcheck-ci.pl helper checks both the base and the PR versions
of the changed scripts, and reports only new findings, matched by file,
ShellCheck code and source line text (so moved code does not count as
new). It fails for new "warning" or "error" level findings; new "note"
level (info and style) ones are shown as notices. In GitHub Actions,
the findings are shown as annotations on the changed lines and listed
in the job summary, with links to the lines and the ShellCheck wiki.
It can be run locally against any base revision, including uncommitted
changes.

The helper is in Perl, like tools/check-source-nonascii.pl (Perl is
already needed for autogen.sh in Git workspaces). It only uses core
Perl 5 modules, and parses the "gcc" output format of shellcheck, so
no JSON module is needed on older systems.

The workflow uses only actions/checkout (no new third-party actions)
and compares the PR merge commit with its first parent.

A new .shellcheckrc file disables the checks for deliberate portability
idioms of NUT scripts: backticks (SC2006), x"$var" comparisons (SC2268)
and expr (SC2003). It is distributed like .editorconfig, to help the
tarball users contribute too.

Describe both in docs/qa-guide.adoc, replacing the note that the tool
could be introduced at a later time. The .shellcheckrc backticks setting
does for ShellCheck what the previous commit did for "make shellcheck"
(issue networkupstools#1058).

Checked with the "build: keep full git version out of config.h" series
(PR networkupstools#3759): its first version is reported for one new SC2166 warning
([ p -a q ]) and the final one passes. Checked renamed, added, removed
and non-shell files, a bad base revision, and the workflow with
actionlint and zizmor. In a test PR on a fork, deliberate findings
showed as annotations on the changed lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: nmbro <nmbro@users.noreply.github.com>
The spellcheck recipe reported unknown words as the aspell pipe-mode
result lines, prefixed by "egrep -b" byte offsets into the aspell
output, so finding the place in the source document took a search.
Count the input lines in the aspell output instead (it ends the results
for each line with an empty line), and report each unknown word as
"file:line:col: <aspell result line>", with the file name relative to
the root of NUT sources. The column offset aspell gives already matches
the original line, as the recipe prefixes each line with a space.

When running in GitHub Actions (GITHUB_ACTIONS=true), also print the
errors as workflow commands, so they show as annotations on the source
lines, with a few suggestions. The top-level spellcheck recipe collects
them into a log file, reported once at the end, because the sequential
retry after a failed parallel attempt reports the same errors again,
and GitHub limits the count of annotations per step. Likewise, the
summary re-run in ci_build.sh does not repeat them. No annotations are
made with SPELLCHECK_ERROR_FATAL=no, as used by the distcheck steps
after the spelling was checked by another step.

Also list the unknown words in the job summary (GITHUB_STEP_SUMMARY),
with links to the source lines and a hint on updating docs/nut.dict.
The ci_build.sh quiet first attempt (whose output is discarded) does
not make annotations nor the summary either, so only its verbose
attempt reports them.

The awk program avoids ternary operators and hash characters, for older
awk and make implementations.

Checked with typos in NEWS.adoc, docs/man/upsc.txt and
scripts/upsdrvsvcctl/README.adoc: correct file, line and column, each
annotation and summary table row printed once (also via
BUILD_TYPE=default-spellcheck ci_build.sh), none with SPELLCHECK_ERROR_FATAL=no or without
GITHUB_ACTIONS. A clean spellcheck passes, and distcheck-light passes
with GITHUB_ACTIONS=true and a typo (as the dist worker runs it).
In a test PR on a fork, typos showed as annotations on the changed
lines, also in the old aspell-en 7.1 job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: nmbro <nmbro@users.noreply.github.com>
NUT_CHECK_ASPELL looked for the filter resources (to pass the location
of "tex.amf" via --filter-path) only under "../lib" and "../share" from
the aspell binary, in "aspell-<version>", "aspell-<version minus the
last component>" or "aspell" subdirectories. On CentOS 7 the resources
are in /usr/lib64/aspell-0.60 for aspell 0.60.6.1, so neither the
"lib64" base nor the "aspell-0.60" subdirectory was tried, and the
check fell back to the aspell built-in paths.

Also try "../lib64" (after "../lib"), and the major.minor version
subdirectory (before the plain "aspell" one), in loops rather than
nested conditions. The order of the other candidates is not changed.

Checked with configure: on Ubuntu with aspell 0.60.8.1 the detected
paths are unchanged; with a fake CentOS 7 layout (aspell reporting
0.60.6.1, an empty "lib", resources in "lib64/aspell-0.60") the lib
and tex paths are found where master found "none", and make spellcheck
passes using that --filter-path.

The other part of the issue, to enable the spellcheck only if aspell
actually works, was done by the live check from commit e519320:
with a missing "en" dictionary (via ASPELL_CONF dict-dir), configure
reports "no" for "if we have all the tools mandatory for documentation
spell checks".

Closes: networkupstools#2068

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: nmbro <nmbro@users.noreply.github.com>
@nmbro
nmbro force-pushed the ci-script-doc-checks branch from 1adaabc to 2b36205 Compare October 15, 2026 20:58

@jimklimov jimklimov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks super, thanks!

@jimklimov jimklimov added CI Entries related to continuous integration infrastructure (here CI = tools + scripts + recipes) portability We want NUT to build and run everywhere possible labels Oct 16, 2026
@jimklimov jimklimov added this to the 2.8.6 milestone Oct 16, 2026
…ols#3773]

Avoid problems escaping double-quotes inside double-quoted backticked context.

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 17, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 17, 2026
@AppVeyorBot

Copy link
Copy Markdown

✅ Build nut 2.8.5.5779-master completed (commit 88b1710c64 by @jimklimov)

@AppVeyorBot

Copy link
Copy Markdown

✅ Build nut 2.8.5.5780-master completed (commit 88b1710c64 by @jimklimov)

@jimklimov
jimklimov merged commit 7d9e3f1 into networkupstools:master Oct 18, 2026
125 of 176 checks passed
@jimklimov jimklimov added the AI For good or bad, machine tools are upon us. Humans are still the responsible ones. label Oct 18, 2026
@nmbro
nmbro deleted the ci-script-doc-checks branch October 18, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI For good or bad, machine tools are upon us. Humans are still the responsible ones. CI Entries related to continuous integration infrastructure (here CI = tools + scripts + recipes) portability We want NUT to build and run everywhere possible

Projects

None yet

3 participants