[lovasoa]

Summary

Export the main database using SQLite's Online Backup API instead of closing and reopening its connection. Preserve prepared statements/cursor positions, custom functions and aggregates, update hooks, PRAGMAs, TEMP objects, attached databases and change counters.

The backup writes pages directly into one preallocated JS-owned result buffer. Close and unlink only the private destination, then return that buffer without an extra full-file copy or a database-sized WASM allocation. Normalize the exported header so WAL sources produce independently reopenable files. Clean up all temporary resources on success and failure.

Transaction behavior: export permits read transactions but rejects active write transactions without committing, rolling back or changing savepoints. This replaces the previous implicit rollback on export and is documented in .export()'s doc comment. Callers must commit or roll back writes before exporting.

Includes the 1.15.2 package/lockfile version bump for release. Supersedes #642 and #643.

Validation

• Regression coverage for preserved connection state/cursors/callbacks, empty databases, non-default page sizes, WAL round trips, write-transaction failures and recovery, independent output buffers, and closed-database errors.
• Deterministic copy-size checks guard against growing/full-file intermediate copies and verify exact buffer ownership.
• Final CI passed for the exact 1.15.2 revision, including all five WASM variants, distribution checks, lint and documentation generation. No local builds or tests run.

The experimental comparison script and temporary CI steps have been removed from the final diff. Recorded CI measurements and methodology: a 577.53 MiB export improved from 892 ms to 233 ms, with peak process RSS reduced from 1867 to 1248 MiB and WASM memory remaining at 21 MiB. A 108 MiB database also exported successfully under a hard 64 MiB WASM cap.

[lovasoa]

Measured comparison: serialize (#643) vs backup (#644)

Recommendation for the stated priority of large databases: prefer the backup implementation in #644. It removes the need for a whole-database allocation in WASM, at a measured ~5× latency cost on the largest case. This is a WASM-headroom improvement, not a low-total-memory download implementation: the current destination is still MEMFS. A genuinely low-memory save/download should use a separate file-backed backup API (OPFS/real filesystem), not return a full Uint8Array.

Actual CI validation

• fix: export database snapshots without closing the connection #643 at 0aa9423: green CI run 40301139618.
• fix: preserve database state when exporting #644 at e8e4065: green CI run 40303987097, including the side-by-side comparison; independent repeat run 40303980124 also passed with the same qualitative results.
• Both pass 29 test modules on each of five WebAssembly variants, plus lint, browser/worker/isolated distribution checks and documentation generation.
• Shared regressions verify preserved callbacks, functions/aggregates, statements/cursors, PRAGMAs, TEMP/attached objects, counters, independent output arrays, non-default page sizes, empty databases and WAL round-trips.
• Transaction tests verify the intentional distinction: serialize includes uncommitted writes; backup rejects write transactions promptly without changing the transaction or savepoints. Both allow read transactions and preserve query_only.
• No builds, tests or benchmarks were run on the limited development machine for these PRs.

Same-runner timing and memory

The table below is from run 40303987097: Node v24.21.0 on Linux / AMD EPYC 7763. Both optimized WASM bundles ran on the same runner, in separate child processes, with alternating strategy order. Each size/strategy uses 3 fresh processes and 7 repeated exports per process, with GC between exports. First-export medians have 3 samples; repeated-export medians have 21 samples. Exported data was reopened and verified with PRAGMA integrity_check and row/blob-length checks.

Actual output	Serialize first / repeated	Backup first / repeated	Serialize peak RSS	Backup peak RSS	Serialize WASM before → after	Backup WASM before → after
1.14 MiB	1.61 / 0.40 ms	4.54 / 1.54 ms	93.7 MiB	97.6 MiB	21.1 → 21.1 MiB	21.1 → 21.1 MiB
18.05 MiB	9.84 / 6.40 ms	53.55 / 48.32 ms	181.9 MiB	215.6 MiB	21.1 → 25.4 MiB	21.1 → 21.1 MiB
72.19 MiB	33.96 / 23.74 ms	162.37 / 155.05 ms	304.5 MiB	339.5 MiB	21.1 → 79.4 MiB	21.1 → 21.1 MiB
144.37 MiB	57.34 / 45.84 ms	284.15 / 277.08 ms	520.0 MiB	522.8 MiB	21.1 → 151.6 MiB	21.1 → 21.1 MiB
577.53 MiB	201.83 / 177.18 ms	882.93 / 895.48 ms	1821.2 MiB	1845.7 MiB	21.1 → 584.8 MiB	21.1 → 21.1 MiB

Peak RSS is a whole-process high-water mark measured before round-trip validation; it includes setup/runtime memory and is not an isolated SQLite allocation measurement. These are synthetic 4 KiB BLOB rows with an expression index, not guarantees for every schema or browser. The independent repeat measured 150 / 730 ms repeated-export medians for the largest case, reinforcing the ~5× difference.

The decisive memory-limit test

Both production WASM modules declare a 2048 MiB maximum. For a controlled limit test, the script lowered only the WASM memory section's maximum to 64 MiB, leaving the initial memory and application code unchanged. The WebAssembly engine enforced the limit.

With a 108.28 MiB database:

• Serialize: failed with Unable to serialize database; the source remained usable.
• Backup: exported successfully and passed integrity/row-count checks; WASM stayed at 21.1 MiB before and after.

This establishes that backup can export databases larger than the permitted WASM heap. It does not establish a tested multi-gigabyte browser file-size ceiling. Available JS/process memory, filesystem buffering, browser buffer limits and storage quotas still matter.

Serialization additionally uses one sqlite3_malloc64() allocation: this SQLite build hard-limits a single allocation to 2,147,483,391 bytes, and actual available contiguous WASM space is smaller. Merely raising MAXIMUM_MEMORY would not remove that SQLite allocation ceiling. Once WASM grows for serialization, freeing the buffer does not shrink its linear memory.

Bundle sizes

Combined optimized JS + WASM: serialize 732,429 bytes raw / 353,854 bytes gzip; backup 732,552 raw / 353,688 gzip. The difference is negligible.

What a low-memory download needs

The current byte-array APIs both create two full-size additional buffers: serialize uses WASM snapshot + JS output; backup uses a MEMFS destination + JS output. For file size D, both can approach roughly 3D total residency including the source, plus overhead.

The next architectural step would be backing the backup destination with a seekable disk-backed file, e.g. an OPFS SyncAccessHandle through a filesystem/VFS adapter in a dedicated worker. Once complete, a browser File and object URL can expose the result for download without materializing a full JS byte array. The temporary file must remain valid while consumed. This is a proposed follow-up, not implemented or browser-tested in #644.

Incremental backup alone is not a download stream: SQLite may rewrite earlier pages and finalize the header at completion, so a forward-only sink cannot directly consume the writes. new Blob([db.export()]) is also not a zero-copy workaround.

The one-off artifact-dependent CI step has been removed after recording these results; the standalone comparison script remains reusable with two dist directories:

node test/compare_exports.cjs /path/to/serialize-dist /path/to/backup-dist

[lovasoa]

Optimized backup results — supersedes the earlier ~895 ms measurement

The preallocated implementation at bf9de3d now writes backup pages directly into the final JS-owned result buffer. It does not repeatedly grow the temporary file and does not make a final FS.readFile() copy. The private file is closed/unlinked and its existing buffer is returned as a Uint8Array.

Same-runner comparison

Green CI run 40311200414: Node v24.21.0, Linux, AMD EPYC 7763. Compared current backup, previous backup at 5adeaa9 and serialize at 0aa9423 on the same runner. Three fresh processes per implementation/size; seven repeated exports per process; alternating implementation order and GC between exports. Times below are median individual export calls, not totals over the repetitions.

Actual output	Previous backup, repeated	Preallocated backup, repeated	Serialize, repeated
1.14 MiB	2.05 ms	1.45 ms	0.53 ms
18.05 MiB	51.54 ms	12.18 ms	6.85 ms
72.19 MiB	152.13 ms	34.48 ms	24.40 ms
144.37 MiB	282.15 ms	62.00 ms	46.54 ms
577.53 MiB	892.13 ms	233.44 ms	170.46 ms

For the largest case:

• First-export medians: previous backup 918.89 ms, preallocated backup 253.32 ms, serialize 195.84 ms.
• Peak process RSS, before opening the validation copy: previous backup 1866.6 MiB, preallocated backup 1248.1 MiB, serialize 1820.1 MiB.
• WASM memory: both backups stayed at 21.1 MiB; serialize grew from 21.1 to 584.8 MiB.
• This is a 3.8× speedup over previous backup and about 619 MiB lower peak process RSS. Compared with serialize, optimized backup was about 37% slower on this runner, with much lower memory use.

Independent green repeat 40311191909, on an AMD EPYC 9V45: largest-case repeated medians were 700.57 ms previous backup / 152.89 ms optimized backup / 139.48 ms serialize. Peak RSS was 1828.0 / 1276.5 / 1834.2 MiB, respectively. Exact timing and memory numbers are machine-dependent, but both runs show the same improvement.

Correctness and memory limits

• All five WASM suites pass 29 test modules each, including new deterministic checks that no typed-array copy during a multi-megabyte export exceeds SQLite's maximum page size (64 KiB), exact-sized output-buffer ownership, output validity after source close, and 512/4096/65536-byte page sizes.
• Lint, browser/worker/isolated distribution checks and generated documentation also pass.
• Round-trip benchmark validation checks PRAGMA integrity_check and row counts/blob lengths.
• With an engine-enforced 64 MiB WASM maximum, optimized backup exported and integrity-checked a 108.28 MiB database at 21.1 MiB WASM memory; serialize failed. No multi-gigabyte browser limit is claimed from that test.
• The ordinary export now needs one additional full-size JS buffer, not both a full temporary-file buffer and a separate output buffer. It remains a synchronous in-memory byte-array API, not a streaming or disk-backed download API.

Complexity clarification

The original MEMFS 12.5%-growth strategy is amortized O(n), not O(n²); earlier discussion overstated the asymptotic issue. The substantial constant overhead came from repeated growth copies, the extra final output copy and the backup write path. The optimization preallocates once and fills the result in a linear page pass. The combined optimization was measured; the individual contributors were not separately profiled.

Recommendation: use the optimized backup implementation in #644. It now approaches serialize's throughput on large files without requiring a database-sized WASM allocation, and materially reduces total peak process memory.

The one-off artifact-download benchmark CI step has been removed after recording these results. No builds, tests or benchmarks ran on the limited development machine.