Run the differential check in CI, and broaden its corpus - #7
Merged
Conversation
The check landed with no way to run it automatically, so it only ran when someone remembered to type it. A test nobody runs decays, and this one carries a guarantee that decays quietly: it fails when a documented divergence stops happening, which is what keeps LIMITATIONS.md honest. That is worth nothing if it is never executed. Wiring it in needed a portability fix first. The runner dropped to the postgres account unconditionally, which is right in the development container, where it runs as root and the server will not accept a root connection, and wrong in CI, where it runs as an unprivileged user who already owns a superuser role. It now drops privileges only when running as root, and PLX_PGUSER overrides the role either way. The psql it invokes comes from PG_CONFIG's bindir rather than a hardcoded container path, so it follows whichever server the rest of the build is targeting. The step runs after installcheck, which is what creates contrib_regression and leaves the extension installed in it. Verified both privilege paths against PG18.4 before wiring: as root through make differentialcheck, and as the unprivileged postgres account, which is the shape CI runs in. Both report 374 matching and 14 documented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first eight cases covered branching, counting loops, integer division and remainder, three-valued booleans, interpolation and one query loop. Five more take it into the areas most likely to hide a defect. safediv catches a raised error and returns a fallback, which is the whole BEGIN/EXCEPTION path. plxgo and plxcobol cannot express catching at all, since Go's panic maps to RAISE with no recover() and COBOL has no handler construct, so both are declared absent rather than skipped quietly. arraysum iterates an array, including an empty one, one holding a NULL element, and a NULL array, which has to raise the way the reference raises rather than iterating nothing. cmp exercises == and != against SQL = and <>, where a NULL operand leaves both comparisons unknown and neither branch is taken. strupper checks that a SQL call reaches upper() from every dialect and that NULL stays NULL. strupper_native is the interesting one. Written with each language's own uppercase idiom, only plxgo agrees with the reference, because plxgo maps a subset of the Go standard library and no other dialect maps anything. Elsewhere s.upcase, s.toUpperCase(), s.upper() and strtoupper($s) transpile without complaint and then fail when the function is called. That is the design, a dialect gives you its syntax over SQL semantics rather than its standard library, but it was only implied by the per-dialect notes and is the kind of thing someone reaches for out of habit on the first day. It is now stated plainly in the shared constraints and recorded as a documented divergence, so the check fails if any dialect ever gains the mapping and the documentation stops being true. 13 cases over 571 comparisons: 542 agree with the reference, 29 are documented, none unexplained. Verified on PostgreSQL 18.4: 13/13 installcheck, differentialcheck clean, and the check still reports and fails on an injected defect in one of the new cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits: make the check actually run, then give it more to check.
1. Run it in CI
The differential check landed in #4 with no way to run it automatically, so it only ran when someone remembered to type it. That is bad for any test and worse for this one, because it carries a guarantee that decays silently: it fails when a documented divergence stops happening, which is what keeps
LIMITATIONS.mdhonest.Wiring it in needed a portability fix first. The runner dropped to the
postgresaccount unconditionally, which is correct in the development container (runs as root, server refuses a root connection) and wrong in CI (runs as an unprivileged user who already owns a superuser role). It now drops privileges only when running as root, withPLX_PGUSERas an override, and takes itspsqlfromPG_CONFIG's bindir instead of a hardcoded path.Both privilege paths were exercised on PG18.4 before wiring: as root via
make differentialcheck, and as the unprivilegedpostgresaccount, which is the shape CI runs in.2. Broaden the corpus
Five new cases, into the areas most likely to hide a defect:
safedivarraysumcmp==/!=against SQL=/<>, with NULL leaving both unknownstrupperupper()from every dialectstrupper_nativestrupper_nativeis the one worth reading. Written with each language's native uppercase idiom, only plxgo agrees with the reference, because plxgo maps a subset of the Go standard library and no other dialect maps anything. Elsewheres.upcase,s.toUpperCase(),s.upper()andstrtoupper($s)transpile without complaint and then fail when the function is called:That is the design (a dialect gives you its syntax over SQL semantics, not its standard library) and not a defect, so it is recorded as a documented divergence rather than fixed. But it was only implied by the per-dialect notes, and it is exactly what someone reaches for out of habit on day one, so it is now stated plainly in the shared constraints. Being a documented divergence means the check fails if any dialect ever gains the mapping and the documentation stops being true.
Where a case cannot cover a dialect it is declared, not skipped:
safedivomits plxgo and plxcobol because neither can express catching at all.Totals
13 cases over 571 comparisons: 542 agree with the reference, 29 documented, 0 unexplained, 0 build errors.
Verification
PG18.4: 13/13 installcheck,
differentialcheckclean, and the check still reports and fails on a defect injected into one of the new cases. CI on this PR is itself the proof that the wiring works.🤖 Generated with Claude Code