Fleet suppression & baseline audit: the fleet reports 185 PHPMD violations; the true figure is 8766

[rubenvdlinde]

Fleet suppression & baseline audit — reported vs true, across 22 repos

Measurement pass only. Nothing was removed or changed in any repo. Every number below comes from neutralising suppressions in a throwaway copy and re-running the tool in a container (PHP 8.4 / PHPMD 2.15.0), checking the exit code on every run.

Related: hydra#473 (composer check:strict green-but-dead), #146 (a disabled analyser leg publishes a green check named after the analyser).

The fleet table

reported = what composer phpmd prints today. true = baseline neutralised and @SuppressWarnings stripped.

repo	PHPMD reported	true	delta	phpmd.baseline	psalm-baseline	phpstan-baseline	@SuppressWarnings (bare)
openregister	0	3266	+3266	517 (flagged)	208 wired	1184	2645 (1305 bare)
openconnector	0	1144	+1144	306 (flagged)	–	84	431 (390 bare)
softwarecatalog	0	855	+855	8 (flagged)	–	127	335 (289 bare)
launchpad	0	694	+694	235 ⚠️ auto-discovered	–	–	158 (73 bare)
pipelinq	0	456	+456	39 (flagged)	–	39	428 (59 bare)
shillinq	0	395	+395	–	–	–	379 (218 bare)
hrmq	0	389	+389	209 (flagged)	21 wired	4	12 (1 bare)
opencatalogi	0	318	+318	56 (flagged)	0 wired	–	204 (194 bare)
hermiq	0	289	+289	–	0 wired	2	319 (8 bare)
procest	0	286	+286	–	–	–	270 (153 bare)
openbuild	6	218	+212	174 (flagged)	–	2	13 (11 bare)
doriath	37	207	+170	–	–	–	138 (62 bare)
portaliq	23	58	+35	–	0 wired	–	30 (4 bare)
zaakafhandelapp	0	50	+50	–	–	22	20 (0 bare)
nldesign	0	40	+40	9 (flagged)	–	11	29 (0 bare)
docudesk	0	24	+24	–	6 ⚠️ ORPHAN	–	20 (12 bare)
larpingapp	0	23	+23	–	–	–	27 (27 bare)
decidesk	0	21	+21	–	–	1	19 (3 bare)
planix	0	5	+5	–	–	–	4 (2 bare)
scholiq	119	124	+5	–	–	–	6 (5 bare)
nextcloud-app-template	0	4	+4	–	0 wired	–	3 (2 bare)
nextcloud-vue	n/a (no PHP)	–	–	–	–	–	315 eslint-disable
TOTAL	185	8766	+8581	1553	235	1476	5100 (2410 bare)

The fleet reports 185 PHPMD violations. The true figure is 8766. 17 of 21 PHP repos report a green zero over a non-zero true count.

Four systemic mechanisms

1. PHPMD auto-discovers phpmd.baseline.xml — launchpad

launchpad's composer script is vendor/bin/phpmd lib text phpmd.xml — no --baseline-file. The 235-entry baseline applies anyway, because PHPMD auto-discovers that filename in the working directory.

Positive control, same ruleset and command: 4 violations / exit 2 without the file, 0 / exit 0 with it. The arms genuinely differ.

Consequence: deleting the flag does not disable a baseline — only deleting the file does. Any repo that "removed the baseline" by dropping the flag still has it fully active.

2. A nested <exclude-pattern> is INERT — 19 of 21 repos

PHPMD honours <exclude-pattern> only as a direct child of <ruleset>. 19 repos nest it inside a <rule> element, where it is silently ignored:

<rule ref="rulesets/unusedcode.xml/UnusedFormalParameter">
  <exclude-pattern>*Migration*</exclude-pattern>   <!-- INERT -->
</rule>

openregister is the only repo with a working direct child (*/Migration/*) — and correspondingly the only one showing 0 Migration hits. Everywhere else the intended exemption never applied: doriath 64, opencatalogi 32, openconnector 153, larpingapp 6, planix 3 UnusedFormalParameter findings sit in Migration/. launchpad has no exclude-pattern at all and carries 80.

This cuts both ways: a meaningful slice of the +8581 is interface-mandated Migration noise that a one-line template fix legitimately retires. Fix the template first, then re-measure, then burn down.

3. Orphan baselines — files that protect nothing

• docudesk: psalm-baseline.xml with 6 entries, but psalm.xml declares no errorBaseline. Suppresses nothing; reads like protection.
• hrmq: no CI at all — no .github/workflows, no .forgejo/workflows. Carries phpmd/psalm/phpstan baselines, phpcs/phpmd/psalm/phpunit/playwright/eslint/stylelint configs, and nothing executes any of it. Baselines captured to green a gate that does not run.

4. Exit-swallowing — mostly a non-issue, and the live CI is clean

I checked composer.json scripts, all .github/workflows and all .forgejo/workflows. No live GitHub workflow swallows a gate's exit code, and there is no continue-on-error on any gate.

The E=0; … || E=1; … exit $E idiom in every repo's check:strict is correct and is not reported. The 10 || true hits in live workflows are all justified and commented (user seeding, log capture on failure, release-diagnostic ls). The one genuine soft-fail — doriath/.forgejo/workflows/pre-merge-check-strict.yaml, whose header says psalm/phpstan/phpmd/test:all "are wrapped in || echo skipping so they never" fail — is Codeberg residue on a retired forge and does not run.

🔴 One outright bug, not debt

openregister — the PDF→ODT anonymisation fallback ("Path B") is unreachable in production. Full write-up: ConductionNL/openregister#2339.

• PdfTextReplacer::validateOutput() accepts bool $strict and never reads it (body lines 321–455). Hidden by phpmd.baseline.xml:251.
• REASON_VALIDATION_FAILED is never constructed anywhere in lib/.
• PdfOdtFallbackOrchestrator::attempt() rethrows unless $cause->getReason() === REASON_VALIDATION_FAILED — so the whole fallback subsystem below that guard can never execute.
• Four docblocks still promise the removed fail-closed guarantee.

Named pattern: the test supplies the precondition that production can no longer produce

PdfOdtFallbackOrchestratorTest hand-constructs the trigger it should be provoking:

$cause = new PdfAnonymisationException(
    reason: PdfAnonymisationException::REASON_VALIDATION_FAILED,
    message: 'Path A failed'
);

…at eight separate call sites. Every test passes, over a subsystem production cannot reach. The suite is evidence about its own constructor call.

This is the fourth instance of this shape this session:

• launchpad — a method_exists()-guarded restore loop that can never be true for NC Entity magic setters; every test stubbed the collaborator, so the dead branch was never exercised.
• decidesk — three never-executed production branches (PublicationRepository:258, OpenCatalogiPublisher:125, MeetingFolderListener:91), same method_exists/__call cause.
• larpingapp — 17 specs asserting against a non-existent seeded object, green because a silent seed failure substituted 'seed-missing'.
• openregister — this one.

A test that constructs its own trigger is the same failure as a stub looser than production. Both make a whole class of assertion unwritable while reading as coverage. The discriminator is one question asked of the fixture, not the assertion: what in production constructs this input?

Worth a gate: flag a test that constructs a domain exception/sentinel which no lib/ code path ever constructs.

What the numbers do NOT mean

Rule names are not defect signals. Measured, not assumed:

• UndefinedVariable is overwhelmingly false positives. All 17 in doriath are by-ref named arguments (openssl_*(output: $var)); fleet-wide the pattern is preg_match($re, $s, $matches). PHPMD does not track by-ref out-params. I checked these before reporting and they are not defects.
• UnusedFormalParameter is mostly interface-mandated — 75 of doriath's 88; 153 of openconnector's 200 are Migration signatures.
• A dead catch is usually not missing auth. openregister's Dead catch – NotAuthorizedException in Merge/Transition/Aggregation controllers looked like the orphan-auth shape. I read the code: MergeService enforces RBAC via _rbac: true on every ObjectService call. The catch is dead because the service declares only @throws RuntimeException — so unauthorized attempts surface as 404 rather than 403. Misleading error semantics, not an IDOR.

The productive discriminator is justification quality, not rule type: does a written reason exist, and does it still hold? Fleet split: 5100 @SuppressWarnings, 2690 with a reason, 2410 bare. The bare ones are the burn-down target. The heuristic that actually found the openregister bug was narrower — a parameter whose name promises behaviour but is never read ($strict, $fallbackCn) — which surfaced exactly 3 candidates fleet-wide, of which 1 was a real defect and 2 were documented as intentional.

Prioritised fleet burn-down

1. Fix the nested <exclude-pattern> in the shared template (19 repos, one line) — then re-measure. Retires interface-mandated noise legitimately and shrinks the real target.
2. openregister#2339 — the Path B bug. The only item here that is a defect rather than debt.
3. Wire up CI for hrmq — until then its baselines suppress findings for nobody.
4. launchpad — decide the 235-entry auto-discovered baseline explicitly; right now it is active by accident.
5. Resolve orphan baselines (docudesk psalm) — delete or wire.
6. Triage 2410 bare @SuppressWarnings, biggest first: openregister 1305, openconnector 390, softwarecatalog 289, shillinq 218, opencatalogi 194.
7. Consider a gate for the named pattern above, and one asserting no phpmd.baseline.xml/psalm-baseline.xml exists without a matching declaration.

Per-repo issues

repo	issue	repo	issue
decidesk	#404	openbuild	#124
docudesk	#378	opencatalogi	#802
doriath	#156	openconnector	#1144
hermiq	#151	openregister	#2338 + 🔴 #2339
hrmq	#67	pipelinq	#700
larpingapp	#262	planix	#314
launchpad	#48	portaliq	#38
nextcloud-app-template	#124	procest	#729
nldesign	#215	scholiq	#275
shillinq	#435	softwarecatalog	#431
zaakafhandelapp	#323	nextcloud-vue	#597

What I did NOT do

• Removed nothing. No suppression, baseline, config or line of code was changed in any repo. No commits, no branches, no PRs.
• Did not re-run psalm or phpstan with baselines neutralised — they need each repo's vendor/ autoloader and stubs. Their baseline entry counts are reported instead (that is what the file hides by construction), but I have no measured true count for psalm/phpstan. The +8581 delta is PHPMD only.
• Did not measure PHPCS reported-vs-true. The phpcs.xml config is uniform fleet-wide (5 exclude-patterns, 16 excluded sniffs, no severity="0"), so there is no per-repo drift to chase, but the inline phpcs:ignore count is not quantified in violations — shillinq (749) and launchpad (181) are the outliers and remain unmeasured.
• Did not adjudicate all 5100 @SuppressWarnings individually. The reason/bare split is mechanical (does text follow the closing paren); I did not verify that each written reason is still true.
• Did not verify the historical figures in the brief (docudesk's "246 hidden by phpmd.baseline.xml"). docudesk has no phpmd.baseline.xml on development today — it was deleted per its own openspec archive — so that number is not reproducible against current HEAD. My measurement is 0 → 24.
• Did not audit non-app repos (valtimo, openklant, opentalk, sapp, n8n-nextcloud, hermiq-exec), which the org enumeration surfaced but which are outside the briefed scope. mydash skipped (archived).
• Did not measure the JS/TS side beyond counting — no eslint re-run with disables stripped.