Repository navigation
Conversation
|
A ZIP file with standard source tarball and another tarball with pre-built docs for commit a734af3 is temporarily available: NUT-tarballs-PR-6.zip. |
nmbro
force-pushed
the
ci-shell-portability
branch
7 times, most recently
from
October 29, 2026 22:33
b0f88e2 to
8aa66bb
Compare
Since b3fdec6 [networkupstools#2529] every *.mo had two rules: the relative target depended on $(abs_builddir)/<same path>, which ran msgfmt. In an in-tree build both names are the same file, and OpenBSD make, which identifies targets by their resolved path, sees the file depend on itself: `make all` stops with "Cycle found", and in scripts/python alone it skips regenerating the *.mo files. The indirection was there to keep "$@" in the build directory: a make that finds the git-tracked *.mo in srcdir via VPATH may set "$@" to that copy, and in distcheck srcdir is read-only. Name the output file in $(builddir) directly instead, with one rule per file. Tested with GNU make 4.4.1, bmake 20260912 and OpenBSD make (ibara/make, OpenBSD 6.9), each in-tree and in a VPATH build with a read-only srcdir and the *.po files newer than the *.mo: all regenerate the three *.mo files in the build directory. Before, OpenBSD make regenerated none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DaCi2xEdgb4PUExwg58nKu Signed-off-by: nmbro <nmbro@users.noreply.github.com>
…ects ShellCheck 0.11.0 (newer than the 0.9.0 GHA-07 had used) reports three spots where a value is not what the code meant: * tools/nut-recorder.sh (SC2327, SC2328): testResult="`upsc ... > file`" captured upsc's standard output, which the redirect sent to /dev/null or the dump file, so "$devName: $testResult" never showed the error. Capture stderr instead (2>&1 >file); $? is still upsc's exit code. Also quote $devName and the dump file path on these lines. * tests/NIT/nit.sh (SC2319): the "upsd does not respond" message of testcase_upsd_allow_no_device() printed $? after `[ "$CMDRES" = 0 ]`, i.e. always 1; print $CMDRES, the exit code of upsc. * scripts/upsdrvsvcctl/nut-driver-enumerator.sh.in (SC2319): upslist_equals() returned $? after a failed `[ ... ]`, which is always 1 there; say `return 1`, as the function's other "not equal" cases do. No change in behaviour: both callers only compare the result with 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DaCi2xEdgb4PUExwg58nKu Signed-off-by: nmbro <nmbro@users.noreply.github.com>
…ands) Check on GitHub-hosted runners what the Jenkins "shellcheck" quick tests check with one build agent per OS. Those tests run `make shellcheck`, an alias of `make check-scripts-syntax`; the ShellCheck tool is a separate check, in GHA-07. One job per autotools set: autoconf 2.69/automake 1.13 as on CentOS 7, 2.69/1.15, 2.71/1.16 and 2.72/1.18, built from GNU release tarballs checked against the GNU keyring. Each job runs autogen.sh, `<shell> -n configure` and a real configure under dash, bash, ksh93, mksh, yash, posh, busybox sh and zsh as sh, and compares config.h and the Makefiles across shells. The newest set's job also runs `<shell> -n` on the tracked and generated scripts, the driver enumerator test with every shell, and `make check-scripts-syntax` with GNU make and bmake. Six more jobs build NUT (with NUT-Monitor) and run the driver enumerator test with PATH overlays of other OSes' sh, make, sed, grep and awk: FreeBSD 15 userland (chimerautils) with its bmake, NetBSD 9/11-like (dash, one-true-awk, two bmake generations), OpenBSD (oksh, OpenBSD make), busybox, and dash with mawk. One job builds the BSD tools from sources pinned by commit or sha256 and passes them on as an artifact. Both tool sets are cached; only master pushes and scheduled runs save the caches. Failures show as error annotations, on the source line where the output names one (shell syntax errors, OpenBSD make's dependency cycles on the Makefile.am rule), and in a job summary with log excerpts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DaCi2xEdgb4PUExwg58nKu Signed-off-by: nmbro <nmbro@users.noreply.github.com>
GHA-07 ran whatever ShellCheck the runner image had: 0.9.0 on ubuntu-24.04, while pip, brew and newer distros give 0.11.0. The two disagree both ways, so a script could pass ShellCheck locally and get new notes in CI. Download the release tarball instead, check its sha256 (the digest GitHub publishes for the asset) before use, and put it first in PATH. Base and PR versions are still checked with the same binary, so the bump does not make old findings look new. On the whole tree 0.11.0 reports 1381 findings against 1533 with 0.9.0: it no longer reports 72 SC2317 and 70 SC2015 notes, and found the SC2319/SC2327/SC2328 cases fixed in the previous commits. Name the version in .github/CI.md for running tools/shellcheck-ci.pl locally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DaCi2xEdgb4PUExwg58nKu Signed-off-by: nmbro <nmbro@users.noreply.github.com>
nmbro
force-pushed
the
ci-shell-portability
branch
from
October 30, 2026 00:07
8aa66bb to
a734af3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft for review on the fork; not proposed upstream yet.
Four commits, each signed and signed off:
scripts/python/Makefile.am: build each NUT-Monitor*.mowith one rule. This fixes a dependency cycle with OpenBSD make.scripts, tests, tools: fix what ShellCheck 0.11.0 finds on$?and redirects (nut-recorder.sh,tests/NIT/nit.sh,nut-driver-enumerator.sh.in).ci: add the GHA-10 shell portability check.ci: GHA-07: pin ShellCheck 0.11.0.The fixes come first, so the tree passes GHA-10 after every commit. Each commit has its
NEWS.adocentry.GHA-10
Adds a GitHub Actions workflow, "GHA-10: Shell portability", that runs what the Jenkins "shellcheck" quick tests check on one Linux runner. On ci.networkupstools.org those tests run on one agent per OS and make. They take about 19 agent-hours per PR build and compete for agents with the slow matrix.
NUT now has two different checks called shellcheck. "GHA-07: ShellCheck" runs the ShellCheck tool on the scripts a PR changes.
make shellcheck, which the Jenkins quick tests run, is an alias ofmake check-scripts-syntax(Makefile.am:1134says so), which runs<shell> -non the scripts. This PR covers the second kind and the configure checks around it. It calls the targetcheck-scripts-syntaxand keeps the name ShellCheck for the tool.Autotools sets x shells
One job per autotools set:
Each job:
autogen.shon a fresh worktree;<shell> -n configurewith dash, bash, ksh93, mksh, yash, posh, busybox sh and zsh as sh;configureunder each of those shells (CONFIG_SHELL), in parallel, and diffsconfig.hand everyMakefileagainst the dash run.The u26 job also runs
<shell> -nwith all eight shells on the tracked scripts (the same selectioncheck-scripts-syntaxmakes) and on the scripts configure generated. It runstests/nut-driver-enumerator-test.shwith every shell, andmake -n allplusmake check-scripts-syntaxwith GNU make and bmake.The autotools sets are built from GNU release tarballs, checked against the GNU keyring, and cached per set.
Userland personalities
A second matrix builds NUT and runs the driver enumerator test under six userland "personalities". Each personality is a directory of symlinks put first in PATH, so configure, the Makefile recipes and NUT's scripts meet another OS's sh, make, sed, grep and awk.
These jobs configure with
--with-nut_monitor=yes(PyQt5 and gettext installed), because the NUT-Monitor.morules are where the makes differ. A "Build BSD tools" job builds the tools once per run from sources pinned by commit or sha256, or restores them from the cache, and hands them to the six jobs as an artifact.Reporting
Each failed check prints an error annotation. Where the output names a location, the annotation goes on that source line:
sh -nerrors in tracked scripts, errors in generated scripts (on their.intemplate), and OpenBSD make's "Cycle found" (on the rule inMakefile.am). The generatedconfigureis not in the repo, so a syntax error there names the configure line, and the job summary shows the lines around it.The job summary starts with PASS or FAIL and the number of failed checks. Each failure follows with its log excerpt, and the table of all checks sits in a collapsed section. The logs of a failed job are uploaded as an artifact.
CI rules
Following
.github/CI.md, only master pushes and scheduled runs save the caches. The workflow runs onpull_requestwith a read-only token and uses onlyactions/*. It passes zizmor 1.25.2 (repo config) and actionlint 1.7.12, and its scripts pass ShellCheck.GHA-07: ShellCheck pinned to 0.11.0
GHA-07 ran whatever ShellCheck the runner image had: 0.9.0 on ubuntu-24.04, while pip, brew and newer distros give 0.11.0. The two disagree both ways. A script here passed 0.11.0 locally and got an SC2015 note from 0.9.0 in CI. GHA-07 now downloads the 0.11.0 release tarball, checks its sha256 (the digest GitHub publishes for the asset) and puts it first in PATH.
.github/CI.mdnames the version for runningtools/shellcheck-ci.pllocally.Base and PR versions of a file are still checked with the same binary, so the bump does not make old findings look new. On the whole tree 0.11.0 reports 1381 findings against 1533 with 0.9.0. It no longer reports 72 SC2317 and 70 SC2015 notes, and adds 4 warnings and 2 errors on existing code (SC2319 in
tests/NIT/nit.shandnut-driver-enumerator.sh.in, SC2327/SC2328 intools/nut-recorder.sh). Only a PR that changes those lines would see them. Dependabot does not track the pin, so bumps are manual: version and sha256 together.Tested on PR networkupstools#3782's commits
shell-matrix-test-broken: 2031bd5, where configure breaks under autoconf 2.69 and Jenkins'make shellcheckstages failed. The c7 and u18 jobs (autoconf 2.69) fail. autogen reportsFAILED: configure script did not pass shell interpreter syntax checks, and dash, bash, ksh, mksh, yash, posh and busybox sh all stop at configure line 29123 (c7) or 29107 (u18). u22 and u26 pass. Runshell-matrix-test-fixed: 8ca85cf, the fix. All four jobs pass, 2 to 3 minutes each, including building the autotools set from source (no cache yet outside master). RunThose two test runs predate the personality jobs and the reporting changes.
The OpenBSD make cycle (commit 1)
On master (run) all four autotools jobs and five personalities pass. OpenBSD make stops in
scripts/python:and the same for
itandru. Since b3fdec6 (networkupstools#2529, in v2.8.3) eachapp/locale/<lang>/LC_MESSAGES/NUT-Monitor.motarget depends on$(abs_builddir)/app/locale/<lang>/LC_MESSAGES/NUT-Monitor.mo, which runs msgfmt. In an in-tree build both names are the same file. OpenBSD make identifies targets by their resolved path, so it sees the file depend on itself; GNU make and bmake don't. The build fails the same way on a real OpenBSD 6.9 machine with Python 3 and PyQt5 installed. The farm's OpenBSD agent doesn't catch it because NUT-Monitor isautothere and its prerequisites are missing.The indirection kept
$@in the build directory. A make that finds the git-tracked*.moin srcdir via VPATH may set$@to that copy, and srcdir is read-only in distcheck, which is how networkupstools#2529 failed. Commit 1 names the output file in$(builddir)directly and keeps one rule per file.Tested on the NUT CI prototype VM with GNU make 4.4.1, bmake 20260912 and OpenBSD make. Each make ran in-tree and in a VPATH build with a read-only srcdir, with the
*.pofiles touched newer than the*.mo:With the old rules, OpenBSD make in
scripts/pythonalone exits 0 but regenerates nothing; from the top-levelmake -j allit stops with the cycle. With the new rules the openbsd personality builds the whole tree. Sun make, which networkupstools#2529 was about, isn't available to test here.ShellCheck 0.11.0 findings (commit 2)
tools/nut-recorder.sh(SC2327, SC2328):testResult="`upsc $devName > file`"captured upsc's standard output, which the redirect sent away, so"$devName: $testResult"never showed the error. It now captures stderr (2>&1 >file), and$?is still upsc's exit code. Checked with a stand-inupscin dash and bash, including a device name with a space.tests/NIT/nit.sh(SC2319): a "upsd does not respond" message printed$?after[ "$CMDRES" = 0 ], which is always 1. It now prints$CMDRES.nut-driver-enumerator.sh.in(SC2319):upslist_equals()returned$?after a failed[ ... ], which is always 1 there. It now saysreturn 1, like the function's other "not equal" cases. Both callers only compare the result with 0. The driver enumerator test passes with all eight shells and in all six personalities.Separately,
tests/NIT/nit.sh:3866prints$OUT4in the message forOUT5. ShellCheck doesn't flag it, and this PR leaves it alone.Spellcheck
GHA-01 already runs the docs spellcheck with the current aspell-en and with the CentOS 7 era aspell-en 7.1 word lists (networkupstools#3752). It puts an annotation on the source line of each unknown word and writes a job summary (networkupstools#3773). This workflow leaves spellcheck out. Jenkins still runs its own spellcheck quick test (
dynacfgPipeline['spellcheck'], on every distro with adoc-spellcheckeragent), which duplicates GHA-01.🤖 Generated with Claude Code
https://claude.ai/code/session_01DaCi2xEdgb4PUExwg58nKu