[lovasoa]

Summary

Alternative to #642: use SQLite's sqlite3_serialize() to export the main database without closing and reopening its connection, rather than restoring individual registrations after reopening.

• Preserve custom scalar/aggregate functions, update hooks, prepared statements/cursor positions, PRAGMAs, TEMP objects, attached databases and connection counters.
• Copy the serialized buffer into an independent Uint8Array, then free the SQLite allocation and restore the stack, including on errors.
• Read the 64-bit output length without adding a BigInt requirement; handle empty databases and reject exports on closed connections.
• Make WAL snapshots independently reopenable by setting the exported copy's file-format read/write versions to rollback-journal mode. The source connection stays in WAL mode.

Transaction semantics

Keep SQLite's serialization semantics: the snapshot includes uncommitted changes visible to the connection. Export neither commits nor rolls back transactions/savepoints; the original connection can still commit or roll back afterwards, without changing the snapshot.

This intentionally replaces the previous implicit rollback caused by closing the connection. Database.export()'s doc comment documents this behavior, preserved connection state, the main-database-only scope, snapshot ownership and the fact that JS callbacks/connection-local settings are not persisted in the exported file.

Regression coverage

• Empty databases, including an empty open transaction.
• Main-database rows/blobs and a non-default page size.
• Preserved functions, aggregates, extension functions, hooks, PRAGMAs, TEMP/attached objects, counters, bound cursors and statement iterators.
• Repeated exports, hook unregistration and independent/mutable output arrays.
• Uncommitted writes, savepoint rollback, full rollback and commit after export.
• WAL snapshots reopened after closing the source.
• Closed-database export errors.

Validation

No builds or tests run locally for this change, as requested for the resource-limited development machine. Relying on this PR's CI to rebuild distribution bundles, run lint/the full WebAssembly and distribution test suites, and generate documentation. git diff --check passes.

[lovasoa]

Measured comparison: serialize (#643) vs backup (#644)

Recommendation for the stated priority of large databases: prefer the backup implementation in #644. It removes the need for a whole-database allocation in WASM, at a measured ~5× latency cost on the largest case. This is a WASM-headroom improvement, not a low-total-memory download implementation: the current destination is still MEMFS. A genuinely low-memory save/download should use a separate file-backed backup API (OPFS/real filesystem), not return a full Uint8Array.

Actual CI validation

• fix: export database snapshots without closing the connection #643 at 0aa9423: green CI run 40301139618.
• fix: preserve database state when exporting #644 at e8e4065: green CI run 40303987097, including the side-by-side comparison; independent repeat run 40303980124 also passed with the same qualitative results.
• Both pass 29 test modules on each of five WebAssembly variants, plus lint, browser/worker/isolated distribution checks and documentation generation.
• Shared regressions verify preserved callbacks, functions/aggregates, statements/cursors, PRAGMAs, TEMP/attached objects, counters, independent output arrays, non-default page sizes, empty databases and WAL round-trips.
• Transaction tests verify the intentional distinction: serialize includes uncommitted writes; backup rejects write transactions promptly without changing the transaction or savepoints. Both allow read transactions and preserve query_only.
• No builds, tests or benchmarks were run on the limited development machine for these PRs.

Same-runner timing and memory

The table below is from run 40303987097: Node v24.21.0 on Linux / AMD EPYC 7763. Both optimized WASM bundles ran on the same runner, in separate child processes, with alternating strategy order. Each size/strategy uses 3 fresh processes and 7 repeated exports per process, with GC between exports. First-export medians have 3 samples; repeated-export medians have 21 samples. Exported data was reopened and verified with PRAGMA integrity_check and row/blob-length checks.

Actual output	Serialize first / repeated	Backup first / repeated	Serialize peak RSS	Backup peak RSS	Serialize WASM before → after	Backup WASM before → after
1.14 MiB	1.61 / 0.40 ms	4.54 / 1.54 ms	93.7 MiB	97.6 MiB	21.1 → 21.1 MiB	21.1 → 21.1 MiB
18.05 MiB	9.84 / 6.40 ms	53.55 / 48.32 ms	181.9 MiB	215.6 MiB	21.1 → 25.4 MiB	21.1 → 21.1 MiB
72.19 MiB	33.96 / 23.74 ms	162.37 / 155.05 ms	304.5 MiB	339.5 MiB	21.1 → 79.4 MiB	21.1 → 21.1 MiB
144.37 MiB	57.34 / 45.84 ms	284.15 / 277.08 ms	520.0 MiB	522.8 MiB	21.1 → 151.6 MiB	21.1 → 21.1 MiB
577.53 MiB	201.83 / 177.18 ms	882.93 / 895.48 ms	1821.2 MiB	1845.7 MiB	21.1 → 584.8 MiB	21.1 → 21.1 MiB

Peak RSS is a whole-process high-water mark measured before round-trip validation; it includes setup/runtime memory and is not an isolated SQLite allocation measurement. These are synthetic 4 KiB BLOB rows with an expression index, not guarantees for every schema or browser. The independent repeat measured 150 / 730 ms repeated-export medians for the largest case, reinforcing the ~5× difference.

The decisive memory-limit test

Both production WASM modules declare a 2048 MiB maximum. For a controlled limit test, the script lowered only the WASM memory section's maximum to 64 MiB, leaving the initial memory and application code unchanged. The WebAssembly engine enforced the limit.

With a 108.28 MiB database:

• Serialize: failed with Unable to serialize database; the source remained usable.
• Backup: exported successfully and passed integrity/row-count checks; WASM stayed at 21.1 MiB before and after.

This establishes that backup can export databases larger than the permitted WASM heap. It does not establish a tested multi-gigabyte browser file-size ceiling. Available JS/process memory, filesystem buffering, browser buffer limits and storage quotas still matter.

Serialization additionally uses one sqlite3_malloc64() allocation: this SQLite build hard-limits a single allocation to 2,147,483,391 bytes, and actual available contiguous WASM space is smaller. Merely raising MAXIMUM_MEMORY would not remove that SQLite allocation ceiling. Once WASM grows for serialization, freeing the buffer does not shrink its linear memory.

Bundle sizes

Combined optimized JS + WASM: serialize 732,429 bytes raw / 353,854 bytes gzip; backup 732,552 raw / 353,688 gzip. The difference is negligible.

What a low-memory download needs

The current byte-array APIs both create two full-size additional buffers: serialize uses WASM snapshot + JS output; backup uses a MEMFS destination + JS output. For file size D, both can approach roughly 3D total residency including the source, plus overhead.

The next architectural step would be backing the backup destination with a seekable disk-backed file, e.g. an OPFS SyncAccessHandle through a filesystem/VFS adapter in a dedicated worker. Once complete, a browser File and object URL can expose the result for download without materializing a full JS byte array. The temporary file must remain valid while consumed. This is a proposed follow-up, not implemented or browser-tested in #644.

Incremental backup alone is not a download stream: SQLite may rewrite earlier pages and finalize the header at completion, so a forward-only sink cannot directly consume the writes. new Blob([db.export()]) is also not a zero-copy workaround.

The one-off artifact-dependent CI step has been removed after recording these results; the standalone comparison script remains reusable with two dist directories:

node test/compare_exports.cjs /path/to/serialize-dist /path/to/backup-dist

[lovasoa]

Optimized backup results — supersedes the earlier ~895 ms measurement

The preallocated implementation at bf9de3d now writes backup pages directly into the final JS-owned result buffer. It does not repeatedly grow the temporary file and does not make a final FS.readFile() copy. The private file is closed/unlinked and its existing buffer is returned as a Uint8Array.

Same-runner comparison

Green CI run 40311200414: Node v24.21.0, Linux, AMD EPYC 7763. Compared current backup, previous backup at 5adeaa9 and serialize at 0aa9423 on the same runner. Three fresh processes per implementation/size; seven repeated exports per process; alternating implementation order and GC between exports. Times below are median individual export calls, not totals over the repetitions.

Actual output	Previous backup, repeated	Preallocated backup, repeated	Serialize, repeated
1.14 MiB	2.05 ms	1.45 ms	0.53 ms
18.05 MiB	51.54 ms	12.18 ms	6.85 ms
72.19 MiB	152.13 ms	34.48 ms	24.40 ms
144.37 MiB	282.15 ms	62.00 ms	46.54 ms
577.53 MiB	892.13 ms	233.44 ms	170.46 ms

For the largest case:

• First-export medians: previous backup 918.89 ms, preallocated backup 253.32 ms, serialize 195.84 ms.
• Peak process RSS, before opening the validation copy: previous backup 1866.6 MiB, preallocated backup 1248.1 MiB, serialize 1820.1 MiB.
• WASM memory: both backups stayed at 21.1 MiB; serialize grew from 21.1 to 584.8 MiB.
• This is a 3.8× speedup over previous backup and about 619 MiB lower peak process RSS. Compared with serialize, optimized backup was about 37% slower on this runner, with much lower memory use.

Independent green repeat 40311191909, on an AMD EPYC 9V45: largest-case repeated medians were 700.57 ms previous backup / 152.89 ms optimized backup / 139.48 ms serialize. Peak RSS was 1828.0 / 1276.5 / 1834.2 MiB, respectively. Exact timing and memory numbers are machine-dependent, but both runs show the same improvement.

Correctness and memory limits

• All five WASM suites pass 29 test modules each, including new deterministic checks that no typed-array copy during a multi-megabyte export exceeds SQLite's maximum page size (64 KiB), exact-sized output-buffer ownership, output validity after source close, and 512/4096/65536-byte page sizes.
• Lint, browser/worker/isolated distribution checks and generated documentation also pass.
• Round-trip benchmark validation checks PRAGMA integrity_check and row counts/blob lengths.
• With an engine-enforced 64 MiB WASM maximum, optimized backup exported and integrity-checked a 108.28 MiB database at 21.1 MiB WASM memory; serialize failed. No multi-gigabyte browser limit is claimed from that test.
• The ordinary export now needs one additional full-size JS buffer, not both a full temporary-file buffer and a separate output buffer. It remains a synchronous in-memory byte-array API, not a streaming or disk-backed download API.

Complexity clarification

The original MEMFS 12.5%-growth strategy is amortized O(n), not O(n²); earlier discussion overstated the asymptotic issue. The substantial constant overhead came from repeated growth copies, the extra final output copy and the backup write path. The optimization preallocates once and fills the result in a linear page pass. The combined optimization was measured; the individual contributors were not separately profiled.

Recommendation: use the optimized backup implementation in #644. It now approaches serialize's throughput on large files without requiring a database-sized WASM allocation, and materially reduces total peak process memory.

The one-off artifact-download benchmark CI step has been removed after recording these results. No builds, tests or benchmarks ran on the limited development machine.

[lovasoa]

Superseded by the merged #644. The optimized backup implementation preserves the connection, avoids a database-sized WASM allocation, and now writes directly into the result buffer. Comparative CI measurements are recorded above. Shipping in v1.15.2.