Skip to content

Backpatch specialized row decoders without breaking API changes - #56

Draft
mzabani wants to merge 52 commits into
masterfrom
backpatch-row-decs-pinned-byte-arrays
Draft

Backpatch specialized row decoders without breaking API changes#56
mzabani wants to merge 52 commits into
masterfrom
backpatch-row-decs-pinned-byte-arrays

Conversation

@mzabani

@mzabani mzabani commented Sep 13, 2026

Copy link
Copy Markdown
Owner

This is an attempt to do #54 without any breaking API changes, for two reasons:

  • Obviously not having to alert users to a possibly breaking change
  • Not having to rewrite hpgsql-simple-compat because the new API is too much of a mismatch for what postgresql-simple offers. We might have to write a low-level API wrapper before we merge the other PR.

mzabani added 26 commits September 8, 2026 17:24
@mzabani
Claude-written rust benchmarks using tokio-postgres
941926f
@mzabani
Improve benchmarks and peak memory measurement methodology
7ff38f0
@mzabani
Tidy up
fb63872
@mzabani
A special Nix shell for benchmarks
d5b6943 
So cargo and rustc aren't necessary for the development shell
@mzabani
Add a C# benchmark
7066356
@mzabani
Tidy up a bit more
2fd85dc
@mzabani
Rewrite benchmark script in Nushell
c1668bf
@mzabani
Add Rust and C# equivalents of the "Record List" benchmark
bc0a088
@mzabani
Try a new kind of very specialized parser for small types
613c248
@mzabani
Float decodesSqlNullTo outside and add strictness for better inlining
8a79a66 
This is the prize I was looking for.

Now the row decoders are built with the NULL handling parts a lot more
inlined, which even means in a fully inlined row decoder we no longer
box into a `Maybe a` to then case match on it and fail on `Nothing`,
when the target record has a field typed as `a` (not a Maybe).
@mzabani
Trying a specialized notConst method
3f3a630 
Types like `Scientific` are not being decoded optimally otherwise, and
they can do better than in the current state
@mzabani
Create new strict and lazy ByteString-like PinnedByteArray types
7d0d744 
And use them almost everywhere.
@mzabani
Wire up new benchmarks
a2cd77d
@mzabani
Change FieldDecoder back to what it was in master
a648f7e
@mzabani
Rewrite rules to transform singleField fieldDecoder to `notInlinedS…
a4d6779 
…ingleFieldRowDecoder`
@mzabani
Improve benchmarks for Maybe a instances
7a0c7e8 
The rewrite rules for these weren't really being tested be for
`FromPgField (Maybe a)` instances, and this shows that clearly
@mzabani
Add rewrite rules for many (Maybe a) decoders
e8e4fee
@mzabani
Make queries in C# and Rust match Haskell's
fc8a359
@mzabani
Update TODO
ba524a1
@mzabani
Tidy up and avoid breaking changes
31fa9e0
@mzabani
A few more types with specialized row decoders and rewrite rules
5f1c5ea
@mzabani
Actually test Field Decoders in tests now
80a0fb1
@mzabani
Important TODO entry
6c9186b
@mzabani
Expose the not inlined method and choose better names
590eb85 
Rewrite rules are not reliable
@mzabani
Important TODO entry
d579188
@mzabani
More specialized instances
e0e7ec5
@mzabani
mzabani force-pushed the backpatch-row-decs-pinned-byte-arrays branch from 5d2024d to e0e7ec5 Compare September 13, 2026 14:21
@mzabani
mzabani force-pushed the backpatch-row-decs-pinned-byte-arrays branch from 998cb9f to bd9b6ec Compare September 19, 2026 15:47
@mzabani
Fix compilation for GHC 9.6
0e11e1e 
`$` is not levity polymorphic enough and the type used for
`Data.Text.Text`'s internal representation is imported from a different
module
@mzabani
Specialized row decoders for Ratio Integer, Oid, and Char
3c7ffcb
@mzabani
More specialized row decoders
66dc79d
@mzabani
Rename inlinedConstFieldDecoder to just constFieldDecoder
7f2d6c0
@mzabani
Remove the two const and not-const parser methods, keep just one of them
a7f7165 
The inliner sees through `const $ ..` anyway, so there's no use having
two separate methods
@mzabani
Stop exposing inlinedFieldRowDecoder
9d7d4d4 
We'll measure and think about this later
@mzabani
Add a test for arrayField
f5f8ca2
@mzabani
mzabani force-pushed the backpatch-row-decs-pinned-byte-arrays branch from 4781c6b to f5f8ca2 Compare September 26, 2026 12:48
@mzabani
Expose PinnedByteArray as an Internal module, improve docs and write …
60cbb4e 
…tests
@mzabani
Store a reversed list of chunks in LazyPinnedByteArray
bf347f0 
This makes appending a strict PinnedByteArray a clearly non-quadratic
operation. Memory usage goes up a bit in benchmarks, but runtime is
untouched.

We don't have benchmarks for really large rows. Those would maybe point
to an improvement.
@mzabani
Make Int64 decoder the same as the one for Int
eab791f
@mzabani
Rename internal modules, add test for smaller-than-4 specialized deco…
8dbe5e1 
…ding function
@mzabani
Be more defensive with take and drop
6809719
@mzabani
Move Builder module back
ccab50f 
No need to break API compatibility here yet
@mzabani
mzabani force-pushed the backpatch-row-decs-pinned-byte-arrays branch from ab36565 to ccab50f Compare September 26, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mzabani