Add a differential check, and fix the plxgo Sprintf bug it found - #4
Conversation
The suite so far records what each dialect produces. It cannot say whether
what it produces is right, because the expectation was written by reading the
same transpiler under test.
test/differential.py supplies the missing oracle. Each case is one small
program written once as a plpgsql reference and once per dialect, then called
with the same arguments. The reference is the plpgsql a PostgreSQL developer
would have written for the same logic, so a disagreement between a dialect and
the reference is a defect rather than a difference of opinion. Values are
compared by their text form with NULL distinguished from the empty string, and
a case that raises must raise the same SQLSTATE.
Intended divergences are recorded per case with the dialects, the calls, and
the reason, and are reported rather than failed. The check also fails when a
recorded divergence stops happening, so a limitation cannot outlive the
documentation describing it, and it prints the dialects a case cannot cover
instead of passing over them silently.
Eight cases over 388 comparisons: 374 agree with the reference and 14 are the
documented NULL interpolation behaviour.
The first run found a real bug. In expression position fmt.Sprintf passed its
Go format string straight into SQL format(), which understands only %s, %I, %L
and %%. Every other verb raised "unrecognized format() type specifier" when the
function was called, so fmt.Sprintf("%d", n), the ordinary way to format an
integer in Go, transpiled cleanly and then failed at run time. Go's verbs now
become the %s that format() understands, keeping a - flag and a width and
dropping the flags and precision format() has no equivalent for. Covered by
three new cases in the plxgo suite and recorded in the docs, which had
described only the RAISE statement path.
Verified on PostgreSQL 18.4: clean build with no warnings, 13/13 installcheck,
and differentialcheck clean. The check was confirmed to detect an injected
boundary defect and a stale exemption, and to exit non-zero when it does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three defects found reviewing the previous commit against its own intent.
The Sprintf rewriter passed a '%' that starts no directive straight through.
The point of the change was that Sprintf should stop producing functions that
fail when called, but a lone '%' is itself an error to format(), so
fmt.Sprintf("100%") still raised "unterminated format() type specifier" at run
time. It is now escaped to %% and reaches the caller as a literal percent.
The documentation understated what dropping a verb costs. Saying only that
flags and precision are dropped implies the difference is padding, but the
verbs that change an operand's representation are affected too: %x, %o, %b, %e
and %q all render what %s renders, so fmt.Sprintf("%x", 255) yields 255 rather
than ff. That is a wrong value rather than a wrong width, which is worse than
an error because it is silent, so both doc/plxgo.md and doc/LIMITATIONS.md now
say so and point at an explicit conversion.
The differential probe returned values verbatim on one output line per probe.
A value containing a newline, or a blank value dropped by the runner's filter,
would shift every later result onto the wrong probe. Counts usually catch that,
but a two-line value and a blank value in the same run cancel out and the
mis-mapping is silent. The probe now returns a sentinel for the empty string
and escapes newlines, so one probe is always exactly one line, and the runner
no longer discards blank lines.
Regression cases added for the bare percent, a percent followed by punctuation,
and the representation-changing verbs.
Verified on PostgreSQL 18.4: clean build with no warnings, 13/13 installcheck,
differentialcheck clean at 374 matching and 14 documented, and the check still
exits 1 on an injected defect after the probe change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial reviewReviewed against the change's own stated intent rather than for style. Three defects, all now fixed in b3affbf. 1. The fix did not fully do what it claimed. The stated goal was that CREATE FUNCTION a1() RETURNS text LANGUAGE plxgo AS $$
return fmt.Sprintf("100%")
$$;
SELECT a1();
-- ERROR: unterminated format() type specifierNot a regression (it failed before too), but the same class of run-time failure the commit set out to remove, and the code comment said "leave the '%' alone" as if that were safe. Now escaped to 2. The documentation understated the cost. Saying only that flags and precision are dropped implies the difference is padding. It is not, for the verbs that change an operand's representation: SELECT g_sprintf_repr(255); -- fmt.Sprintf("%x|%o|%b|%q", n, n, n, n)
-- 255|255|255|255
3. The differential probe could silently mis-map results. One probe was assumed to be one output line. A value containing a newline, or a blank value dropped by the runner's Regression cases added for all three. VerificationPostgreSQL 18.4: clean build with no warnings, 13/13 installcheck, 🤖 Generated with Claude Code |
Why
The regression suite records what each dialect produces. It cannot say whether what it produces is right, because the expected output was written by reading the same transpiler that is under test. A front end can be confidently, consistently wrong and the goldens will agree with it.
The check
test/differential.py(make differentialcheck) supplies the missing oracle. Each case is one small program written once as a plpgsql reference and once per dialect, then called with the same arguments. The reference is the plpgsql a PostgreSQL developer would have written for the same logic, so a disagreement between a dialect and the reference is a defect rather than a difference of opinion.Eight cases, 388 comparisons: 374 agree with the reference, 14 are the documented NULL-interpolation behaviour, 0 unexplained.
The bug it found
In expression position
fmt.Sprintfpassed its Go format string straight into SQLformat(), which understands only%s,%I,%Land%%. Every other verb raisedunrecognized format() type specifierwhen the function was called:The transpile succeeded and the failure only appeared at run time, so
fmt.Sprintf("%d", n), the ordinary way to format an integer in Go, produced a function that could not run. Go's verbs are now rewritten to the%sthatformat()understands, keeping a-flag and a width and dropping the flags and precisionformat()has no equivalent for.doc/plxgo.mdhad described only theRAISEstatement path and is corrected; the dropped precision is recorded indoc/LIMITATIONS.md.Verification
On PostgreSQL 18.4: clean build with no warnings, 13/13 installcheck,
differentialcheckclean. The check itself was verified to catch an injected boundary defect (>=changed to>) and a stale exemption, and to exit non-zero when it does.🤖 Generated with Claude Code