spec: a pool of serve sessions behind an executor interface

[turbolytics]

Spec only. No code, no config, nothing shipped.

What this changes

sqlflow serve answers one request at a time: every request waits on one mutex around one DuckDB connection. Measured against the Bluesky demo on Render on 2026-09-16, after rollup tables cut the query itself to 13 ms:

Clients	Throughput	p50	p95	Failures
1	5.8/s	164 ms	251 ms	0
4	12.2/s	303 ms	418 ms	0
8	12.1/s	613 ms	855 ms	0
16	13.4/s	1,191 ms	1,318 ms	0
32	13.2/s	2,328 ms	2,682 ms	0

Throughput stops climbing at four clients and latency grows in proportion after that. The same request reports elapsed_ms of 13 idle and 945–1,219 under sixteen clients: the query is unchanged, and the rest is queueing counted as work. Before #295 the same test failed 66% of requests at sixteen clients, so cheaper queries raised the ceiling from 1.1/s to 13/s; only concurrency raises it again.

The spec proposes a pool of sessions behind an Executor interface, plus the metrics that size it, since serve records none today.

The parts worth reviewing

• The seam. Executor, Session and Statement interfaces in internal/serve/executor.go, with internal/serve/duckdb.go the only file importing ADBC. The interface exists because the engine underneath may change: the Postgres sink had to leave DuckDB's postgres extension for pgx in A keyed Postgres sink on pgx, and the leak loops that found the demo's growth #290, and that move was expensive because the driver was welded into the write path.
• Arrow in the interface. Session.Run returns array.RecordReader and serve keeps one encoder. Arrow is sqlflow's in-process format, and the four documented rendering rules (UTC timestamps, naive timestamps, exact decimals, NaN as a string) each exist because Arrow's own rendering was wrong for JSON. One encoder is how they stay true across backends. The spec states the costs: a non-Arrow backend pays a conversion, columns[].type is a SQL name reconstructed from an Arrow type, and buffer lifetimes become part of the interface.
• Default pool size 4, with the memory measurement behind it.
• Serve pins SET TimeZone='UTC' on every session, rather than trusting the config to reach them.
• Metrics off by default. GET /metrics on the existing listener, since Render routes one port, but the labels name every dataset and the port is public.

Measurements in the spec

DuckDB 1.5.2 through ADBC, two connections to one in-memory database with a Postgres attached:

Behaviour	Result
ATTACH on connection 1	Connection 2 queries pg.* without attaching
ATTACH the same alias again	Binder Error: database with name "pg" already exists
SET TimeZone='UTC' on connection 1	Connection 2 reads America/New_York, the host zone
SET pg_connection_limit = 4 on connection 1	Connection 2 reads 4
Two slow queries, one per connection	200 ms, against 377 ms one after the other

So commands must run once, and the timezone must be set per session or a correct config returns buckets in the host zone.

Resident memory, memory_limit='128MB', N sessions each running the demo's fold at once:

Sessions	Widest the grain ladder allows	Unbounded
1	52 MiB	100 MiB
4	99 MiB	279 MiB
8	146 MiB	every query failed, out of memory

Four sessions fit a 256 MB box. The bounds from #295 are what make that true: memory_limit is one budget shared by every session, so without a cap on what a request may read, concurrent requests fail together rather than queueing.

Verification

• Spec only; no code changed. go build ./... and go test -short ./... unaffected.
• uv run --locked pytest tests/tooling -q — no registry or status file changed
• make coverage-page — no feature status changed
• uv run --locked pytest tests/release -q — no CLI surface changed
• make soak — no allocating path changed

Notes for the reviewer

• Two numbers are not yet trustworthy and the spec says so. The memory figures are macOS, where Maxrss is bytes and on Linux it is KiB, so build order task 1 repeats the measurement in the release container and changes the default if it says so. And whether an attached Postgres shares its connections across sessions or holds them per session is an integration test, not an assumption.
• Two judgement calls worth challenging: metrics default to off, and there is no session_commands block, so the timezone is the only per-session setting until something needs more.
• A "Scaling out" section records what is true before a cache exists: serve is stateless and more instances need no code change, but the worst case is instances × pool.size × pg_connection_limit against one database, and instances share nothing, so each pulls the same hot rows for the same popular ranges. Scaling out today trades a bottleneck for a worse one. That reframes caching as the prerequisite for horizontal scale rather than a latency optimisation.

[turbolytics]

Implemented in #322, which contains these spec and plan commits. Closing this in favour of that one once it is reviewed.