Jenkinsfile-dynamatrix: modernize dynacfgPipeline['spellcheck'] …

INSTALL.nut.adoc

@@ -267,6 +267,32 @@ will be 'ups' and 'nut' respectively.
 Be sure the new user is a member of the new group!  If you forget to
 do this, you will have problems later on when you try to start upsd.
 
+Repetitive build/test optimizations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are several optimizations in NUT recipes and scripts aimed at speeding
+up repetitive builds and tests, which are not too useful for one-off end-user
+builds, but can bring significant benefits on CI and packaging systems and
+for developers iterating on some features.
+
+* `ccache` can be used to NOT recompile objects for which no sources have
+  changed. On many systems it suffices to add e.g. `/usr/lib/ccache` early
+  in `PATH` and populate it with symlinks to e.g. `/usr/bin/ccache` using
+  various compiler program names.
+* If you are a developer iterating with debug builds on different branches,
+  `export CI_AUTOGEN_STABLE_VERSION=true` can additionally ensure that the
+  `AC_INIT` in the generated `configure` script would not follow NUT SEMVER
+  but use a shorter and temporally stable version string. This helps on some
+  systems where enabling debug options embeds all macros into compiled objects,
+  so an ever-changing `PACKAGE_VERSION` in `config.h` which does follow NUT
+  SEMVER prevents `ccache` direct mode (because each build is deemed unique).
+* Using `export DO_USE_NUTCI_CACHE=yes` (and derived environment variables
+  like `DO_USE_AUTOCONF_CACHE` or `DO_USE_NIT_TESTCERT_CACHE`) can be used
+  to speed up re-builds with `ci_build.sh` and re-runs of NUT Integration
+  Test suite. Exports like this can be placed into your `~/.profile` or
+  exported from CI system recipes.
+* When you iterate code development, `./configure --with-doc=skip ...` can
+  avoid time loss from parsing documentation files you never even edited.
 
 Build and install
 ~~~~~~~~~~~~~~~~~

Jenkinsfile-dynamatrix

@@ -98,9 +98,13 @@ import org.nut.dynamatrix.*;
 
     // In modern builds, use the ci_build.sh recipe which first checks
     // quietly for things that succeed, and summarizes errors in the end
-    dynacfgPipeline['spellcheck_prepconf'] = false
-    dynacfgPipeline['spellcheck_configure'] = false
-    dynacfgPipeline['spellcheck'] = '(BUILD_TYPE=default-spellcheck-quick ./ci_build.sh)'
+    dynacfgPipeline['spellcheck'] = [
+        prepconf: false,
+        configure: false,
+        tryAllNodes: false,
+        tryAllDistros: true,
+        spellcheck: '(BUILD_TYPE=default-spellcheck-quick ./ci_build.sh)'
+    ]
 
 /*
     // For older builds, with only autotools in the tree:
@@ -233,6 +237,41 @@ import org.nut.dynamatrix.*;
     dynacfgPipeline.excludeCombos_AUTOTOOLS_NOT_REQUIRE_TWEAK =
         [[~/OS_DISTRO=(?!(openbsd-6\.|macos|netbsd))/]]
 
+    // Prepare to exclude not-newest features in most-popular (master-branch)
+    // builds for RISC-V platforms (which realistically only need modern C/C++
+    // coverage as they were first introduced in 2020's)
+
+    /** Exclude older language standards on RISC-V platforms */
+    dynacfgPipeline.axisCombos_RISCV_LEGACY_CSTD = [
+        ~/ARCH64=riscv64/,
+        ~/CSTDVERSION_c=(89|99|11)\b/
+    ]
+
+    /** Exclude older language standards on RISC-V platforms */
+    dynacfgPipeline.axisCombos_RISCV_LEGACY_CXXSTD = [
+        ~/ARCH64=riscv64/,
+        ~/CSTDVERSION_cxx=(98|11)\b/
+    ]
+
+    /** Exclude strict ANSI/C modes on RISC-V if GNU variants are sufficient */
+    dynacfgPipeline.axisCombos_RISCV_STRICT_C = [
+        ~/ARCH64=riscv64/,
+        ~/CSTDVARIANT=c/
+    ]
+
+    /** Exclude any older compiler revisions that may remain installed on RISC-V
+     * builders in the future (in addition to newer installed versions) */
+    dynacfgPipeline.axisCombos_RISCV_NOT_MAX_COMPILER = [
+        ~/ARCH64=riscv64/,
+        ~/COMPILERVER=(NOT_MAX|!MAX)/
+    ]
+
+    dynacfgPipeline.excludeCombos_RISCV_DEFAULT = [
+        dynacfgPipeline.axisCombos_RISCV_LEGACY_CSTD,
+        dynacfgPipeline.axisCombos_RISCV_LEGACY_CXXSTD,
+        dynacfgPipeline.axisCombos_RISCV_NOT_MAX_COMPILER
+    ]
+
     dynacfgPipeline.branchStableRegex = ~/^(master|main|stable)$/
     if ( env?.BRANCH_NAME ==~ /^fightwarn.*$/ && (!env?.CHANGE_TARGET) ) {
         dynamatrixGlobalState.branchDefaultStable = 'fightwarn'
@@ -509,6 +548,7 @@ set | sort -n """
                 //dynamatrixAxesLabels: [~/^OS/, '${COMPILER}VER', 'ARCH${ARCH_BITS}'],
                 dynamatrixAxesLabels: [~/^OS_DISTRO/, 'COMPILER'],
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + dynacfgPipeline.excludeCombos_AUTOTOOLS_REQUIRE_TWEAK
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
@@ -552,6 +592,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD]
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
@@ -705,6 +746,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_STRICT_C
+                    // + dynacfgPipeline.excludeCombos_RISCV_DEFAULT	// Commented-away: do give a shot to base language revision and distcheck on RISC-V too after all
                 ], body)
             }, // getParStages
         'bodyParStages': dynacfgPipeline.slowBuildDefaultBody_ci_build
@@ -744,6 +786,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
             }, // getParStages
@@ -826,6 +869,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesLabels': 'replace', 'commonLabelExpr': 'replace', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD]
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
@@ -879,6 +923,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesLabels': 'replace', 'commonLabelExpr': 'replace', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD]
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
@@ -922,6 +967,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesLabels': 'replace', 'commonLabelExpr': 'replace', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD]
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                     + dynacfgPipeline.excludeCombos_AUTOTOOLS_NOT_REQUIRE_TWEAK // skip systems covered with "autotools-driven builds" (MASTER-0)
@@ -964,6 +1010,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC]	// checking non-GCC builds; GCC is covered in more detail below
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
                 ], body)
@@ -1004,7 +1051,8 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT
-                    //+ [dynacfgPipeline.axisCombos_STRICT_C] // TODO: revise to match the purpose of this filter
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT	// Because axisCombos_RISCV_LEGACY_CSTD excludes CSTDVERSION_c=89 on ARCH64=riscv64, RISC-V builds are skipped entirely in this phase
+                    //+ [dynacfgPipeline.axisCombos_STRICT_C]	// TODO: revise to match the purpose of this filter
                     + [[~/OS_DISTRO=openbsd-6\./, ~/COMPILER=GCC/, ~/GCCVER=[01234].+/]]	// Does not trust pragmas to quiesce known "infractions" which are not problems
                     + [[~/OS_DISTRO=freebsd-15(\.\d+|)$/, ~/CSTDVARIANT=c/]]	// Disabled `long long` pre-C99 precludes use of `strto{,u}ll()` [#3583]
                     + [dynacfgPipeline.axisCombos_COMPILER_NOT_GCC]
@@ -1211,6 +1259,7 @@ set | sort -n """
                 runAllowedFailure: true,
                 mergeMode: [ 'excludeCombos': 'merge', 'dynamatrixAxesCommonEnv': 'replace' ], // NOTE: We might want to replace other fields, but excludeCombos must be merged to filter compiler versions vs. language standards as centrally defined!
                 excludeCombos: dynacfgPipeline.excludeCombos_DEFAULT_CPPUNIT_STRICT_C
+                    + dynacfgPipeline.excludeCombos_RISCV_DEFAULT	// Skips C89/C99/C11 standards on RISC-V, executing solely the gnu17/gnu++17 hard-warning check against the newest GCC (GCCVER: 'MAX')
                     + [dynacfgPipeline.axisCombos_COMPILER_NOT_GCC]
                     + [dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD]
                     + [dynacfgPipeline.axisCombos_WINDOWS_CROSS]
@@ -1250,7 +1299,7 @@ set | sort -n """
                     dynacfgPipeline.axisCombos_GNU_C,
                     dynacfgPipeline.axisCombos_WINDOWS_CROSS,
                     dynacfgPipeline.axisCombos_WINDOWS
-                    ]
+                    ] + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                 ], body)
             }, // getParStages
         'bodyParStages': dynacfgPipeline.slowBuildDefaultBody_ci_build
@@ -1413,7 +1462,7 @@ set | sort -n """
                     dynacfgPipeline.axisCombos_COMPILER_GCC_TOO_OLD,
                     dynacfgPipeline.axisCombos_WINDOWS,
                     dynacfgPipeline.axisCombos_NOT_WINDOWS_CROSS
-                    ]
+                    ] + dynacfgPipeline.excludeCombos_RISCV_DEFAULT
                 ], body)
             }, // getParStages
         'bodyParStages': dynacfgPipeline.slowBuildDefaultBody_ci_build

NEWS.adoc

@@ -873,6 +873,14 @@ https://github.com/networkupstools/nut/milestone/13
       [#3489, #3495, #3498]
     * Jenkins used as the NUT CI farm core has bumped JDK requirements for its
       controllers and build agents, `docs/config-prereqs.txt` revised.
+    * `Jenkinsfile-dynamatrix` recipe revised to generate fewer RISC-V recipes
+      specifically (very slow worker), focusing on the newest compiler and C/C++
+      revisions available on the worker. On the other side, this CI recipe was
+      extended to require more "quick phase" builds to cover spellcheck on many
+      distributions, as their base dictionaries like different things. [PR #3757]
+    * Further rearrangement of AppVeyor CI builds and security hardening of
+      GitHub Actions used to prepare the dist tarballs and report on progress
+      there. [PRs #3753, #3754, #3756]
 
 
 Release notes for NUT 2.8.5 - what's new since 2.8.4

docs/developers.txt

@@ -333,6 +333,9 @@ technologies,NUTCI_farm_technologies] and
 linkdoc:qa-guide[Continuous Integration (NUT CI farm)
 build agent preparation,NUTCI_farm_agents].
 
+See also "Repetitive build/test optimizations" in `INSTALL.nut.adoc` file
+for suggestions specific to this area.
+
 Integrated Development Environments (IDEs) and debugging NUT
 ------------------------------------------------------------