Background
I've bumped into PSScriptAnalyzer / GitHub code-scanning warnings on files in the tests/ folder a couple of times when writing test helpers that start with verbs that (in a public API) have some conventions around then (like New). These rules aren't really applicable to test code. A recent example: a test helper named New-Frame tripped PSUseShouldProcessForStateChangingFunctions because New is an approved state-changing verb, so the analyzer expects the function to support -WhatIf/-Confirm.
These are linter conventions meant for exported, public cmdlets, not for test scaffolding. The tests/ folder isn't part of our public API, so applying the full PSGallery ruleset there generates noise (and a review-bot comment on every PR that adds such a helper).
Where these come from
The warnings are produced by the analysis workflow in .github/workflows/analysis.yml:
```powershell
Invoke-ScriptAnalyzer -Path . -Recurse -Settings PSGallery | ConvertTo-SARIF -FilePath results.sarif
```
The resulting SARIF is uploaded to GitHub code scanning, which is what renders the inline PR comments. GitHub has no per-path mute for third-party SARIF — the only lever is what we feed into the analyzer.
Suggested fix
Run the analyzer in two passes: the full ruleset on modules/, and a relaxed ruleset on tests/ that excludes the rules which are inherently meaningless for test helpers:
```powershell
$results = Invoke-ScriptAnalyzer -Path ./modules -Recurse -Settings PSGallery
$results += Invoke-ScriptAnalyzer -Path ./tests -Recurse -Settings PSGallery -ExcludeRule PSUseShouldProcessForStateChangingFunctions, PSUseApprovedVerbs, PSAvoidUsingWriteHost $results | ConvertTo-SARIF -FilePath results.sarif \``
We can add to the the -ExcludeRule list over time if there are other analyzers that don't make sense for the tests.
Alternatives considered
Neither of these are ideal:
- Drop
tests/ from the scan entirely (-Path ./modules): simplest, but loses all linting on tests.
- Repo-wide
PSScriptAnalyzerSettings.psd1 with ExcludeRules: not path-aware, so it would also disable the rule for modules/, where we want it.
Context: came up while reviewing #135.
Background
I've bumped into PSScriptAnalyzer / GitHub code-scanning warnings on files in the
tests/folder a couple of times when writing test helpers that start with verbs that (in a public API) have some conventions around then (likeNew). These rules aren't really applicable to test code. A recent example: a test helper namedNew-FrametrippedPSUseShouldProcessForStateChangingFunctionsbecauseNewis an approved state-changing verb, so the analyzer expects the function to support-WhatIf/-Confirm.These are linter conventions meant for exported, public cmdlets, not for test scaffolding. The
tests/folder isn't part of our public API, so applying the full PSGallery ruleset there generates noise (and a review-bot comment on every PR that adds such a helper).Where these come from
The warnings are produced by the analysis workflow in
.github/workflows/analysis.yml:```powershell
Invoke-ScriptAnalyzer -Path . -Recurse -Settings PSGallery | ConvertTo-SARIF -FilePath results.sarif
```
The resulting SARIF is uploaded to GitHub code scanning, which is what renders the inline PR comments. GitHub has no per-path mute for third-party SARIF — the only lever is what we feed into the analyzer.
Suggested fix
Run the analyzer in two passes: the full ruleset on
modules/, and a relaxed ruleset ontests/that excludes the rules which are inherently meaningless for test helpers:```powershell
$results = Invoke-ScriptAnalyzer -Path ./modules -Recurse -Settings PSGallery
$results += Invoke-ScriptAnalyzer -Path ./tests -Recurse -Settings PSGallery -ExcludeRule
PSUseShouldProcessForStateChangingFunctions, PSUseApprovedVerbs, PSAvoidUsingWriteHost $results | ConvertTo-SARIF -FilePath results.sarif \``We can add to the the
-ExcludeRulelist over time if there are other analyzers that don't make sense for the tests.Alternatives considered
Neither of these are ideal:
tests/from the scan entirely (-Path ./modules): simplest, but loses all linting on tests.PSScriptAnalyzerSettings.psd1withExcludeRules: not path-aware, so it would also disable the rule formodules/, where we want it.Context: came up while reviewing #135.