Repository navigation
CI: report script and doc check findings on the changed lines - #3773
Merged
Merged
Conversation
|
A ZIP file with standard source tarball and another tarball with pre-built docs for commit ce43b09 is temporarily available: NUT-tarballs-PR-3773.zip. |
Let "make shellcheck" (check-scripts-syntax) check the POSIX shell scripts with each interpreter listed in the SHELL_PROGS envvar, like the NDE self-test already does, rather than only the system /bin/sh. Bourne-Again shell scripts are still checked with bash. The shell is named in the error message, so a failure is easy to tell apart. Also hide the "`...` obsolete, use $(...)" warnings that e.g. ksh93 (the /bin/sh on illumos and Solaris) prints in syntax-check mode for each use of backticks, which NUT scripts use deliberately for portability: on current master, "ksh -n" printed 1442 of these for 47 scripts, which buried the 15 other warnings it had. The output is captured before filtering, so the syntax check exit code is kept. The filter uses $(EGREP) with bracket expressions, rather than "grep -F", for older grep implementations. Checked with SHELL_PROGS="dash ksh bash" (ksh 93u+m/1.0.8): all scripts pass, no backticks warnings are left, the other ksh warnings are still shown; and that syntax errors in a POSIX or a Bourne-Again shell script fail the check, naming the file and the shell. Spellcheck and distcheck-light pass. Closes: networkupstools#1058 Closes: networkupstools#1059 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
Add a "GHA-07: ShellCheck" workflow for pull requests, which runs ShellCheck on the shell scripts changed by the PR. The existing scripts have many known findings (about 2900 in the tree, 28 of them errors), so checking the whole tree would always fail. Instead, the new tools/shellcheck-ci.pl helper checks both the base and the PR versions of the changed scripts, and reports only new findings, matched by file, ShellCheck code and source line text (so moved code does not count as new). It fails for new "warning" or "error" level findings; new "note" level (info and style) ones are shown as notices. In GitHub Actions, the findings are shown as annotations on the changed lines and listed in the job summary, with links to the lines and the ShellCheck wiki. It can be run locally against any base revision, including uncommitted changes. The helper is in Perl, like tools/check-source-nonascii.pl (Perl is already needed for autogen.sh in Git workspaces). It only uses core Perl 5 modules, and parses the "gcc" output format of shellcheck, so no JSON module is needed on older systems. The workflow uses only actions/checkout (no new third-party actions) and compares the PR merge commit with its first parent. A new .shellcheckrc file disables the checks for deliberate portability idioms of NUT scripts: backticks (SC2006), x"$var" comparisons (SC2268) and expr (SC2003). It is distributed like .editorconfig, to help the tarball users contribute too. Describe both in docs/qa-guide.adoc, replacing the note that the tool could be introduced at a later time. The .shellcheckrc backticks setting does for ShellCheck what the previous commit did for "make shellcheck" (issue networkupstools#1058). Checked with the "build: keep full git version out of config.h" series (PR networkupstools#3759): its first version is reported for one new SC2166 warning ([ p -a q ]) and the final one passes. Checked renamed, added, removed and non-shell files, a bad base revision, and the workflow with actionlint and zizmor. In a test PR on a fork, deliberate findings showed as annotations on the changed lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
The spellcheck recipe reported unknown words as the aspell pipe-mode result lines, prefixed by "egrep -b" byte offsets into the aspell output, so finding the place in the source document took a search. Count the input lines in the aspell output instead (it ends the results for each line with an empty line), and report each unknown word as "file:line:col: <aspell result line>", with the file name relative to the root of NUT sources. The column offset aspell gives already matches the original line, as the recipe prefixes each line with a space. When running in GitHub Actions (GITHUB_ACTIONS=true), also print the errors as workflow commands, so they show as annotations on the source lines, with a few suggestions. The top-level spellcheck recipe collects them into a log file, reported once at the end, because the sequential retry after a failed parallel attempt reports the same errors again, and GitHub limits the count of annotations per step. Likewise, the summary re-run in ci_build.sh does not repeat them. No annotations are made with SPELLCHECK_ERROR_FATAL=no, as used by the distcheck steps after the spelling was checked by another step. Also list the unknown words in the job summary (GITHUB_STEP_SUMMARY), with links to the source lines and a hint on updating docs/nut.dict. The ci_build.sh quiet first attempt (whose output is discarded) does not make annotations nor the summary either, so only its verbose attempt reports them. The awk program avoids ternary operators and hash characters, for older awk and make implementations. Checked with typos in NEWS.adoc, docs/man/upsc.txt and scripts/upsdrvsvcctl/README.adoc: correct file, line and column, each annotation and summary table row printed once (also via BUILD_TYPE=default-spellcheck ci_build.sh), none with SPELLCHECK_ERROR_FATAL=no or without GITHUB_ACTIONS. A clean spellcheck passes, and distcheck-light passes with GITHUB_ACTIONS=true and a typo (as the dist worker runs it). In a test PR on a fork, typos showed as annotations on the changed lines, also in the old aspell-en 7.1 job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
NUT_CHECK_ASPELL looked for the filter resources (to pass the location of "tex.amf" via --filter-path) only under "../lib" and "../share" from the aspell binary, in "aspell-<version>", "aspell-<version minus the last component>" or "aspell" subdirectories. On CentOS 7 the resources are in /usr/lib64/aspell-0.60 for aspell 0.60.6.1, so neither the "lib64" base nor the "aspell-0.60" subdirectory was tried, and the check fell back to the aspell built-in paths. Also try "../lib64" (after "../lib"), and the major.minor version subdirectory (before the plain "aspell" one), in loops rather than nested conditions. The order of the other candidates is not changed. Checked with configure: on Ubuntu with aspell 0.60.8.1 the detected paths are unchanged; with a fake CentOS 7 layout (aspell reporting 0.60.6.1, an empty "lib", resources in "lib64/aspell-0.60") the lib and tex paths are found where master found "none", and make spellcheck passes using that --filter-path. The other part of the issue, to enable the spellcheck only if aspell actually works, was done by the live check from commit e519320: with a missing "en" dictionary (via ASPELL_CONF dict-dir), configure reports "no" for "if we have all the tools mandatory for documentation spell checks". Closes: networkupstools#2068 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
nmbro
force-pushed
the
ci-script-doc-checks
branch
from
October 15, 2026 20:58
1adaabc to
2b36205
Compare
…ols#3773] Avoid problems escaping double-quotes inside double-quoted backticked context. Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
|
✅ Build nut 2.8.5.5779-master completed (commit 88b1710c64 by @jimklimov)
|
|
✅ Build nut 2.8.5.5780-master completed (commit 88b1710c64 by @jimklimov)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make the script and documentation checks report problems where they are, on the changed lines, instead of somewhere in a long CI log, and add a ShellCheck check for the shell scripts that a PR changes.
Problem
make spellcheckprinted unknown words withegrep -bbyte offsets into theaspelloutput (e.g.18011:& backticks 13 16: ...), not with a location in the source document. In CI, the failure is somewhere in the step log.make shellcheckis only a syntax check (sh -n/bash -n) with the system/bin/sh. Running the realshellcheckon the tree gives about 2900 findings (28 errors, 258 warnings), mostly deliberate portability idioms or old code, so checking the whole tree would always fail.make shellchecknoise and coverage (Ignore $(...) vs backtick syntax warning inmake shellcheck#1058, Extendmake shellcheckto support ourSHELL_PROGSenvvar from some of the tests #1059): ksh93 (the/bin/shon illumos and Solaris) prints a "...obsolete, use $(...)" warning for every backtick in syntax-check mode: 1442 of them for 47 scripts on current master, which buried the 15 other warnings it had. And there was no way to check with other shells than/bin/sh.aspellusage on stranger platforms #2068):configuredid not look inlib64, nor inaspell-0.60style directories for a 4-component version like0.60.6.1.Changes
build: SHELL_PROGS for make shellcheck--make shellcheck(check-scripts-syntax) checks POSIX shell scripts with each interpreter listed inSHELL_PROGS(default still/bin/sh), like the NDE self-test already does; bash scripts are still checked with bash, and the error names the shell. The backticks warning is filtered out (keeping the syntax check exit code). Closes Ignore $(...) vs backtick syntax warning inmake shellcheck#1058, closes Extendmake shellcheckto support ourSHELL_PROGSenvvar from some of the tests #1059.ci: add ShellCheck check for changed scripts-- a new "GHA-07: ShellCheck" workflow for PRs runstools/shellcheck-ci.pl, which checks the base and PR versions of the changed shell scripts and reports only new findings, matched by file, ShellCheck code and source line text (so moved code does not count as new). New "warning"/"error" findings fail the check; new "note" (info/style) ones are shown as notices. Findings show as annotations on the changed lines and in the job summary. The helper is Perl (liketools/check-source-nonascii.pl; Perl is already needed byautogen.shin Git workspaces), uses only core Perl 5 modules and thegccoutput format ofshellcheck(no JSON module needed), and can be run locally:perl tools/shellcheck-ci.pl origin/master. A new.shellcheckrcdisables the checks for deliberate NUT portability idioms: backticks (SC2006, the ShellCheck side of Ignore $(...) vs backtick syntax warning inmake shellcheck#1058),x"$var"comparisons (SC2268) andexpr(SC2003); it is distributed like.editorconfig. The workflow uses onlyactions/checkout, read-only permissions, and compares the PR merge commit with its first parent.docs: locate spellcheck errors, annotate in GHA-- unknown words are reported asfile:line:col: <aspell result line>, with paths relative to the source root. In GitHub Actions they also become annotations on the source lines and a table in the job summary, each word once (the top-level recipe collects them, because the sequential retry after a failed parallel attempt and theci_build.shre-runs report the same errors again, and annotations per step are limited). No annotations withSPELLCHECK_ERROR_FATAL=no(the distcheck steps).m4: find aspell resources in lib64 and X.Y dirs-- also try../lib64and the major.minor version directory. The other part of CI: More fixes foraspellusage on stranger platforms #2068 (only enable spellcheck ifaspellactually works) was already done by the live check from e519320, verified with a missing dictionary. Closes CI: More fixes foraspellusage on stranger platforms #2068.Docs:
docs/qa-guide.adoc(shell script and spelling checks sections),NEWS.adoc.Example from the field
A test run of this branch on my fork showed what this is for: the "Spellcheck with old aspell-en 7.1 (as on CentOS 7)" job, which then failed for every PR, annotated
docs/man/microdowell.txt:42:14withinitialisationfrom #3744, which is not in the old dictionary (commented there, since fixed by 3cb4c2f). Without this change, finding that needed a read through the job log.Screenshots
From a test PR on my fork with deliberate findings (not part of this PR):

Testing
tools/shellcheck-ci.pl: the first version of build: keep full git version out of config.h (ccache-friendly) #3759 is reported for its one new SC2166 warning ([ p -a q ]), the final version passes; renamed, added, removed and non-shell files; a bad base revision (exit 2); local and GitHub Actions output modes.make shellcheck SHELL_PROGS="dash ksh bash"(ksh 93u+m/1.0.8): passes, no backticks warnings left, other ksh warnings still shown; syntax errors in a POSIX or bash script fail, naming the file and shell.NEWS.adoc,docs/man/upsc.txtandscripts/upsdrvsvcctl/README.adoc: correct file, line and column; each annotation and summary row once, also viaBUILD_TYPE=default-spellcheck ./ci_build.sh; none withSPELLCHECK_ERROR_FATAL=noor withoutGITHUB_ACTIONS.configurewith a fake CentOS 7 aspell layout (version 0.60.6.1, resources inlib64/aspell-0.60): found where master foundnone, andmake spellcheckpasses with that--filter-path; unchanged paths on Ubuntu.make spellcheck,make distcheck-light, ASCII-only check on the changes.Open questions
.shellcheckrcdotfile in the source root, and its list of disabled checks: are there other idioms that should be in it?Checklist
Makefile.amrecipes were updated andmake distcheck-lightpassesNEWS.adocmake spellcheck, updateddocs/nut.dict(no words removed)🤖 Generated with Claude Code