Skip to content

Serialize database destruction with concurrent opens - #787

Open
kriszyp wants to merge 17 commits into
mainfrom
kris/serialize-destroy-open
Open

Serialize database destruction with concurrent opens#787
kriszyp wants to merge 17 commits into
mainfrom
kris/serialize-destroy-open

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 15, 2026

Copy link
Copy Markdown
Member

Summary

Rebases this PR onto latest main. While this was in flight, main independently landed an equivalent-but-different fix for the same destroy-vs-open race (2c2066aa, 84157bf9): a resolved-identity, per-entry gate (DBKey{path, readOnly, secondaryPath}) instead of this PR's original path-only destroyingPaths set. Per Kris's direction, this rebase adopts main's gate rather than porting the PR's original design: the first commit skips the PR's now-duplicate commit and adapts every dependent commit onto main's per-entry/cross-key conditions, and the second carries forward the PR's quarantine/retry/compaction-cancellation work and its test fixtures on top of that. destroyingPaths itself is re-derived (not ported) as a separate, lock-free-compatible gate for the window between "registry entries erased" and "physical files deleted" — physical deletion runs without databasesMutex held (it's I/O-bound), so main's design alone left that window ungated; this was found by test failures against the carried-forward fixtures, not assumed from the PR.

This is the rocksdb-js root-cause fix for Harper PR #2169: it moves a lifecycle invariant Harper worked around in JavaScript (locking root opens) into the native registry, makes teardown failures observable/recoverable via quarantine + database:closeFailed, and adds cancellation tokens so a manual compaction cannot block teardown indefinitely.

Second rebase: onto main's WriteBufferManager stall watchdog (#824)

main moved again (7213b98a, "Make a WriteBufferManager write stall observable") and this branch was CONFLICTING. Four conflicts, all resolved keeping both sides:

  • binding.cpp Shutdown — main's watchdog begin…Shutdown()/join…Watchdog() bracket now wraps this PR's try/catch
    • throw. Two ordering constraints had to be reconciled: this PR moved GlobalEvents::Shutdown() after DBRegistry::Shutdown() (a quarantining close emits database:closeFailed, which needs its listeners to still exist — test/fixtures/fork-shutdown-failure.mts asserts the event), and the watchdog join must run before the throw, or a failed shutdown() leaves the 1 Hz thread alive. Same combination in the last-env cleanup hook.
  • backup.cpp — both hunks were pure PR-side additions (BackupInFlightClaim, the napi_cancelled in-flight decrement) that git could not place.
  • db_registry.h — this PR's CloseResult CloseDB(...) return type alongside main's CollectWriteBufferManagerInventory.
  • AGENTS.md — invariant renumbering. Main's WBM-stall invariant becomes 21; this PR's three become 22/23/24, with the two internal cross-references (see invariant 21 → 22, see invariant 22 → 23) updated. Main's droppedColumns.clear() in finishClose() and its two-argument ColumnFamilyDescriptor constructor were carried into this PR's rewritten finishClose(bool destroying) / OpenDB().

Fixed by the rebase's own independent pre-push review

Two full rounds against the rebased head (codex + gemini + harper-domain; Cursor legs are disabled on any diff that edits AGENTS.md). Round 1 found five real items, all fixed; round 2 verified each against HEAD and produced no new actionable findings.

  • Stale transaction-log cache survives a foreign close into the reopen — a regression from this PR's own invariant-18 fix. The ownerThreadId guard that makes a foreign close napi-safe also means a cross-env destroy()/shutdown() can no longer clear logRefs, so a reopened handle handed useLog() back a TransactionLog whose TransactionLogHandle::store weak_ptr pointed at the unregistered store of the closed lifecycle. Only addEntry re-resolves; every read accessor reported an empty log. DBHandle::open() now releases the cache on the owning thread before DBRegistry::OpenDB. test/fixtures/fork-foreign-close-log-cache.mts holds the stale TransactionLog alive across the foreign shutdown() (the cache entry is a weak napi_ref, so letting it be collected would mask the bug) and asserts log size, queried entry count, and object identity — it reports 0 bytes, expected 31 without the fix, 3/3 clean with it.
  • A destroy-cleanup tombstone omitted transactionDetailsRegistryStatusDB declares it non-optional, so a monitor reading entry.transactionDetails.length threw on exactly the entry shape that only appears when a physical destroy failed, i.e. when the diagnostic is needed.
  • Three test-delay seams still called ::getenv off the JS threadROCKSDB_JS_BACKUP_DELAY_MS (libuv worker), ROCKSDB_JS_DESTROY_DELAY_MS and ROCKSDB_JS_CLOSE_RETRY_DELAY_MS (any teardown thread), all added by this PR, now snapshotted in initializeTestSeams() like the fault flags beside them. AGENTS.md's entries for the three say so.
  • fork-shutdown-retry.mts raced the retry claim — the worker posts before calling shutdown(), so the parent's open could reach the still quarantined entry and fail with "previous close failed" instead of measuring the wait; and a handle opened while the process-wide shutdown() loop is still scanning can be force-closed before the data assertion reads it. Both barriers the two-descriptor fixture already had.
  • fork-compact-cancel-async.mts could drop the destroy resultawait outcome yields to the event loop with no 'message' listener attached, so a { destroyed: true } posted in that window was delivered to a zero-listener emitter and dropped, hanging the fixture to its runner timeout. The promise is now claimed before the await. The sync sibling is not affected (compactSync() blocks the loop, so the queued message is only delivered after the next listener attaches).

Red CI caught a real path-identity defect

The first rebase push turned macOS CI red (Bun + Deno) with four quarantine tests failing on /private/var/... vs /var/.... Root cause, not a test problem: a destroy-cleanup tombstone in registryStatus() and every database:closeFailed event reported the registry key's resolved identity, so a caller matching either against the path it opened did not recognize it. That is exactly what AGENTS.md invariant 19 forbids ("returning only the resolved identity breaks callers that match paths against the spelling they supplied") and what registryStatus() already did correctly for a live descriptor.

  • emitCloseFailures() reports descriptor->path.
  • DBRegistryEntry::reportedPath remembers the opening caller's spelling, because a tombstone has no descriptor left to ask. DestroyDB captures it from the first claimed descriptor under databasesMutex, and a retry of a failed destroy — which finds only the tombstone — falls back to the spelling that entry already remembered (the round-3 review finding).

Not caused by the rebase: the pre-rebase head carried identical code and was green on the older macos-26-arm64 runner image (2026072820260831), whose TMPDIR made the latent mismatch reachable. test/destroy.test.ts now covers it on every platform with an explicit symlink rather than depending on macOS's /var link, and asserts both the initial failure and the retry. Each report site was reverted individually to confirm the test fails — with expected undefined to be true, the same assertion macOS CI produced, and expected [ …(2) ] to match object [ …(2) ] for the retry.

A CI "flake" that was a real use-after-free

The first two rebase pushes each had exactly one job die with SIGSEGV in fork-destroy-open.mts — a different runtime each time (Bun/ubuntu, then Deno/ubuntu), which reads like flake. It is not. Running that fixture six-way concurrently reproduced it locally at 2/60, and gdb named the frame:

__strlen_avx2
v8::String::NewFromUtf8
napi_set_named_property
rocksdb_js::DBRegistry::RegistryStatus

databasesMutex covers the registry map, not a descriptor's own maps. registryStatus() walked descriptor->columns — guarded by columnsMutex — while a cross-env destroy()'s finishClose() cleared that map from the worker thread, so name.c_str() pointed into a freed map node and napi_set_named_property() strlen()ed it. locks.size() was read the same way (a count, so a torn read rather than a fault). The column summary is now snapshotted under columnsMutex (plus the per-CF userSharedBuffersMutex for its buffer count) with the N-API values built after releasing it — holding it across those calls would risk a finalizer re-entering the same non-recursive mutex on this thread, which is why transactions already had this shape under txnsMutex. locks.size() is read under locksMutex.

No lock-order inversion: OpenDB and CollectWriteBufferManagerInventory already establish databasesMutex → columnsMutex, getUserSharedBuffer is a leaf, and no locksMutex region reaches the registry. Pre-existing on main (its registryStatus() walks columns unguarded too); this PR is what makes it reachable, because destroy() now force-closes every descriptor and the PR's own fixtures poll registryStatus() across that window.

test/fixtures/fork-registry-status-column-race.mts makes it deterministic instead of leaving it to CI luck: a worker churns dropSync() against the shared descriptor while the main thread polls registryStatus(), with the new ROCKSDB_JS_REGISTRY_STATUS_COLUMNS_DELAY_MS seam parking the walk per column family so an erase lands inside it. The column names run past libstdc++'s 15-char small-string buffer so the erase frees a separate heap allocation — an SSO name usually survives the free intact and hides the bug, which is why the first attempt at this fixture (driving the close-time columns.clear() instead) reproduced 0/7 and was deleted rather than shipped unreferenced. 5/5 abort without the fix, 3/3 clean with it, and the concurrent fork-destroy-open.mts stress went 2/60 → 0/132. Reverting the snapshot fails the churn fixture, so the close-time path is covered by the same net.

Review-thread adjudication

All three previously-unresolved threads were re-checked against the current code; none is left unanswered, and the one new thread is ruled on below.

  • shutdown() can abort process exit and skip later cleanup (new, binding.cpp) — claim verified, prescribed fix overruled. Details under ## For the human reviewer.
  • Raw path aliases bypass destruction ownership (db_registry.cpp) — unchanged ruling: real, pre-existing (registry identity has always been the raw path string), and canonicalizing changes a user-visible identity contract. Wants its own PR.
  • Quarantined descriptors still admit transaction commits (transaction.cpp) — unchanged
    ruling: the window is real but is not specific to quarantine, and the suggested fix (reject admission when descriptor->isClosing()) is what AGENTS.md invariant 18 exists to prevent. The fix that closes it — running the closables sweep before the flush in finishClose() — reorders the most delicate path here and wants its own change.

Third rebase: onto main's tsdown bump

main moved again (dependabot's tsdown bump, merged as 531af655). No conflicts — the only delta between the previous head's merge-base and origin/main was package.json/pnpm-lock.yaml, and git rebase origin/main replayed all 11 commits clean. Build, pnpm test:native (194/194), and the full Vitest suite (935 passed / 9 skipped) all pass post-rebase; pnpm fmt:check/lint/ type-check are clean.

Independent pre-push review (round 25, full — a force-push is never an ancestor of the prior review): codex (graded) and Gemini both ran; the Harper-domain adjudicator hit its time budget and was killed (SIGKILL, timeout) before it could rank/filter, so I adjudicated the raw output myself against the code:

  • Codex's three "surviving findings" (iterator mutex cost, an atomic load's default memory order, a comment-style nit) are unchanged carries-forward from the ~24 prior rounds already reflected in this description — not new, not acted on.
  • Gemini reprised the shared-benchmark-db-path claim already dismissed above, plus a new minor claim that benchmark/setup.ts's teardown resolve()-then-throw silently reports success: it conflates the serialization gate (activeBenchmark/promise, used only to sequence between benchmarks) with the current benchmark's own result (the async setup() call's returned promise, which the trailing throw genuinely rejects — throws: true is set exactly so vitest surfaces it). Not applied.
  • Gemini's one new claim outside prior rounds — db_registry.cpp:1029 holds databasesMutex across napi_create_string_utf8 calls in RegistryStatus(), and if V8 GC runs a DBHandle finalizer synchronously mid-allocation, that finalizer's DBRegistry::CloseDB would re-lock the same non-recursive mutex on the same thread — checks out as a real hazard class (it's exactly what invariant 6's columnsMutex/txnsMutex/locksMutex narrow-scoping exists to avoid for the other locks in this same function), but the outer databasesMutex hold is byte-for-byte unchanged from origin/main (confirmed via git show origin/main:src/binding/database/db_registry.cpp) — it predates this PR and every prior round. Left alone as out-of-scope for a rebase; flagged separately for its own issue.

For the human reviewer

shutdown() now throws — the review claim is correct, and I kept the behavior. Your call to overrule me. The bot's facts check out, and I verified the Node semantics directly: a throw from a process.on('exit') listener skips every exit listener registered after it (unconditionally), and flips the exit code to 1 unless an uncaughtException handler is installed. Harper core calls it exactly that way — resources/RocksTransactionLogStore.ts:19, process.on('exit', () => shutdown()), bare, registered at module load — so on a close-time flush failure at exit it would skip later exit listeners, including ones registered by application components. On main, shutdown() genuinely never throws (descriptor->close() is void and the close-time flush status is discarded), so this is a behavior change, not a clarification.

Three reasons I did not adopt "keep shutdown() non-throwing":

  1. db.close() already throws the identical error (database.cpp:211, with the "Call shutdown() to retry close, or destroy() to delete the database" suffix). Making shutdown() silent would have the two close entry points disagree about whether a failed close is an error.
  2. Three of DBRegistry::Shutdown()'s four throw sites are lifecycle timeouts, not close
    failures: shutdownMutex contention, the per-descriptor drain, and the destroy-in-flight wait. database:closeFailed cannot carry those — no descriptor failed. A blanket non-throwing shutdown() would return normally while databases are still open or files still being deleted, which is a worse silent failure than the one being avoided.
  3. Harper does not listen for database:closeFailed (no listener anywhere in the repo), so today the throw is the only channel that reaches it. Event-only reporting would make a close-time flush failure fully invisible there.

What I did instead is make the contract explicit where a caller will see it: the README already documented the throw and the try/catch exit-listener pattern, and the exported shutdown now carries the same JSDoc. The harper-side guard is still missing and is a one-line change at resources/RocksTransactionLogStore.ts:19; it needs to land with or before this, and it is outside this repo. If you would rather not ask that of consumers, the alternative is a return value plus a separate throwing API, and I'd want your direction before building it.

Carried from the original PR body, still open (unchanged by either rebase):

  • Registry identity compares raw path strings, so a ../symlink alias can still bypass the destroy/open gate (narrower than it reads — RocksDB's own LOCK file blocks a second read-write open through an alias; a read-only alias during destroy is the live hazard). Wants its own PR — canonicalizing changes a user-visible identity contract (registryStatus().path, lock/backup file paths, TransactionLogStoreRegistry keys).
  • A transaction can still be admitted against a quarantined descriptor via the legacy libuv commit path, since the closables sweep never ran for that descriptor. The fix (running the closables sweep before the flush in finishClose()) reorders the most delicate path here and wants its own change.
  • Last-env module cleanup still keys off an unsynchronized --moduleRefCount == 0; a fresh env loading the module concurrently isn't coordinated against a concurrent Shutdown()/Teardown(). Pre-existing.
  • DBIteratorHandle::Next()'s unconditional per-row iteratorMutex lock/unlock on the hottest read path. Estimated low single digits percent overhead against a several-hundred-ns per-row N-API cost — real, but the obvious fix (a relaxed-load gate) is insufficient (a closer could still free the iterator between the load and the mutex), and a correct reader/closer handshake plus a pnpm bench range-scan comparison is more than a rebase should take on.

Declined this round, with reasons:

  • isClosing() as a memory_order_relaxed load (raised as a major by Gemini, downgraded to a nit by the adjudicator in both rounds, and independently scored a nit by Codex). Its own estimate is that the LDAR is dwarfed by iterator->Next(), and getKeysCount() is not a request hot path. The flag is also read under mutexes in the registry paths, so changing the default ordering of a widely-used accessor is not a local change.
  • The aggregate comment-narration nit (a handful of blocks in db_registry.cpp, db_descriptor.{h,cpp}, async.h, db_handle.cpp, closable.h, binding.cpp that narrate implementation history or address the reviewer). The technical content is accurate; churning ten comment blocks across a 69-file lifecycle diff adds review surface for no behavior change. Better as a follow-up sweep over the whole file set at once.
  • RegistryStatusDB.userSharedBuffers is declared non-optional but no branch of registryStatus() sets a top-level property of that name (it is per-column-family, which is what the README documents) — so entry.userSharedBuffers > 0 silently reads undefined > 0. Verified pre-existing on main; same for columnFamilies being typed string[] against an object. Wants its own change.
  • A cached TransactionLog retained across a foreign close and never reopened still reports an empty log to its holder. Unchanged from main (which deleted the weak ref on close, leaving any retained object equally stale), and the open-time invalidation above fixes the reopen path, which is the reachable one.

Remaining CI failure, not from this PR

test/txn-close-commit-uaf.test.ts aborts on Deno/macOS only. That is the pre-existing worker-env teardown abort the test itself names and already mitigates — const retry = process.versions.deno && process.platform === 'darwin' ? 1 : 0 with a comment pointing at #746 — and it exhausted that single retry. It appeared on the same job before the path-spelling and use-after-free commits, and nothing in this PR touches that repro's path.

Verification

  • pnpm check (type-check + lint + fmt) — clean.
  • pnpm test:native — 194/194 passed.
  • pnpm test (Vitest) — 933 passed, 9 skipped (expected Deno/GC-related skips), 0 failed.
  • test/destroy.test.ts alone — 31/31, including the new foreign-close log-cache fixture.
  • test/destroy.test.ts alone — 32/32, run 5× to check the new symlink test for flakiness.
  • Independent pre-push review, six rounds on the rebased head. Round 1 (--full, forced by the rebase — the prior review is no longer an ancestor) found the five items above. Round 2 (delta) confirmed all five fixed against HEAD and surfaced no new actionable finding; its three surviving items are the two declined nits plus the pre-existing userSharedBuffers type mismatch, and it recorded a correction to round 1 (which had claimed the tombstone branch omitted userSharedBuffers too — never a top-level field). Round 3, on the path-spelling fix, found the tombstone-retry gap, fixed above. Round 4 converged: the graded leg reported zero findings.
  • Rounds 5 and 6 covered the use-after-free fix above: round 5 found the unreferenced fixture (removed) and a comment of mine that overclaimed which RegistryStatusDB fields the tombstone branch fills (reworded); round 6 converged, with every item previously adjudicated.
  • Gemini claims checked and rejected rather than applied, across the rounds: checkpoint.cpp leaking operationsInFlight when admission/queueing fails (CheckpointInFlightClaim is the RAII equivalent of BackupInFlightClaim, and handedOff is set only after both succeed), and benchmark/setup.ts making concurrent workers share one database path (they already did before this PR — the declaration moved within the same scope chain, and sharing one database across workers is the point of those benchmarks); checkpoint.cpp's CheckpointInFlightClaim again; ReleaseLogRefsByEnv "permanently leaking" napi_refs for an erased descriptor (they are env-owned and reclaimed at env teardown, and once the entry is erased no foreign close() can reach the handle through closables, which is the hazard invariant 18 exists for); and, for the third time, a per-call __cxa_guard_acquire on the seam flags (a std::atomic<int> with a constexpr constructor is constant-initialized, so no guard is emitted — confirmed by inspecting the generated assembly in an earlier round).
  • Earlier rounds on the pre-rebase head (four, converged) are unchanged and described in the commit history.

Refs #787

🤖 Generated with Claude Code

Final maintenance pass — 2026-09-09

This note supersedes earlier rebase, review-coverage, and verification statements above. The branch is rebased cleanly onto 17fceeef; PR head ac31bdea contains the final review-feedback fix.

  • The new registryStatus() thread was valid: carrying a shared_ptr<DBDescriptor> beyond databasesMutex could make a racing last-handle close see an extra owner and skip its purge with no release-side retry. RegistryStatusEntry now contains only copied values, captured under the registry and owning child locks, and all N-API construction happens after unlocking. The audited registry-to-child ordering is documented beside each mutex, and invariant 6 records the ownership-pin failure mode.
  • The new worker fixture uses a dedicated status worker, asserts that the close actually overlaps the delayed registry walk, and then verifies the entry was purged. It is wired into the destroy lifecycle suite. The earlier column-map race fixture now reports its own timeout before Vitest's outer deadline.
  • The three older open threads remain human decisions: canonical path identity changes user-visible behavior; rejecting transaction admission on isClosing() violates invariant 18 and the safer closables-before-flush reorder needs a separate change; and shutdown() error reporting is a consumer-compatibility choice rather than a mechanical review fix.

Changes

Verification

  • pnpm check — clean (type-check, lint, formatting); git diff --check — clean.
  • pnpm test:native194/194 passed.
  • pnpm test956 passed, 9 skipped, 0 failed.
  • The regression failed on 87842947 with a retained registry entry (followed by a teardown SIGSEGV) and passes on this head; the strengthened overlap assertion also passes.
  • Planning fallback after the default Claude leg hit quota: Framing-Verdict: chosen-approach-sound (Gemini). Final full review produced no new actionable
    finding: Gemini repeated an adjudicated benchmark false positive and the existing comment nit; Claude failed at startup, Cursor was policy-pruned for the AGENTS.md edit, and the domain leg timed out. These are review-infrastructure coverage gaps, not test failures.
  • Current-head CI has reported: validation and security checks pass; platform, native, benchmark, and stress jobs are running with no reported failure.

Complexity: complicated

Review-Coverage: authored=codex; ran=gemini; blocked=claude(exit-1),domain(timeout); declined=cursor-grok,cursor-composer; rounds=8 @ ac31bde

Human-Review-Need: 4 @ ac31bde

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust database lifecycle management for RocksDB JS bindings. It implements a timed-wait mechanism (lifecycleWaitSeconds) for open, destroy, and shutdown operations to prevent concurrent lifecycle conflicts. It also introduces a "quarantine" state for database paths when a native close, flush, compaction, or physical directory cleanup fails, preventing subsequent opens until the cleanup is retried via destroy() or shutdown(). Additionally, it ensures that in-flight operations (like backups and checkpoints) are safely awaited before destruction, and that thread-affine N-API references are cleaned up safely. There are no review comments, so I have no feedback to provide.

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 23.94K ops/sec 41.77 39.80 1,886.019 0.138 119,706
🥈 rocksdb 2 10.30K ops/sec 97.11 93.34 24,057.195 0.972 51,486

getSync() > sequential keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 27.67K ops/sec 36.15 34.82 538.799 0.108 138,329
🥈 rocksdb 2 10.49K ops/sec 95.37 91.88 610.244 0.050 52,426

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 25.53K ops/sec 39.17 35.81 1,993.039 0.294 127,659
🥈 rocksdb 2 15.67K ops/sec 63.80 56.34 1,121.35 0.119 78,373

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 338.72 ops/sec 2,952.271 110.053 70,710.142 22.82 678
🥈 lmdb 2 26.35 ops/sec 37,946.363 429.727 1,215,067.958 136.3 64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 38.65K ops/sec 25.87 11.99 20,829.827 0.842 193,273
🥈 lmdb 2 440.34 ops/sec 2,270.948 239.641 27,258.768 1.55 2,202

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 737.12K ops/sec 1.36 1.16 5,042.571 0.209 3,685,590
🥈 lmdb 2 430.95K ops/sec 2.32 1.33 7,042.059 0.549 2,154,750

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 855.36 ops/sec 1,169.104 1,010.818 1,910.376 0.283 1,711
🥈 lmdb 2 1.14 ops/sec 874,242.212 825,381.715 951,517.939 3.64 10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 23.34K ops/sec 42.85 29.66 20,336.246 2.07 46,672
🥈 lmdb 2 808.49 ops/sec 1,236.868 55.24 14,550.659 5.40 1,617

Results from commit 347546f

@kriszyp
kriszyp marked this pull request as ready for review August 15, 2026 11:52
Comment thread src/binding/database/db_registry.cpp Outdated
Comment thread src/binding/binding.cpp Outdated
Comment thread src/binding/database/db_registry.cpp Outdated
Comment thread src/binding/core/test_seam.h Outdated
Comment thread src/binding/database/db_handle.cpp
Comment thread src/binding/database/db_registry.cpp Outdated
Comment thread src/binding/database/db_handle.cpp
Comment thread src/binding/database/db_registry.cpp Outdated
Comment thread src/binding/database/database.cpp
Comment thread src/binding/iterator/db_iterator.cpp Outdated
Comment thread AGENTS.md Outdated
Comment thread benchmark/setup.ts
Comment thread test/destroy.test.ts
Comment thread src/binding/iterator/db_iterator.cpp Outdated
Comment thread src/binding/database/db_registry.cpp Outdated
Comment thread src/binding/database/database.cpp
@cb1kenobi

Copy link
Copy Markdown
Member

Reviewed f24ef7a5 — no issues found. This PR looks good, nice job!

Re-review of the one new commit since b35ad3f7 ("Address remaining lifecycle review threads"). Both previously-open findings are confirmed fixed in the code, not just marked resolved:

  • db_iterator.cpp:289 (Medium, per-row getenv) — fixed. The lookup is hoisted into initializeTestSeams(), which runs as the first statement of NAPI_MODULE_INIT, and Next() now does a relaxed atomic load. Verified at the object-code level: ROCKSDB_JS_ITERATOR_NEXT_DELAY_MS no longer appears in db_iterator.o (only in binding.o), and the compiled DBIterator::Next contains zero getenv calls. The new ROCKSDB_JS_COUNT_DELAY_MS seam got the same treatment up front.
  • db_registry.cpp:960/961 (Medium, closeError asymmetry) — fixed. The four copies of the finishClose() → erase-or-quarantine → notify → emit tail are collapsed into closeClaimedDescriptors(), and the policy that had drifted is now one named, documented option (failOnCompletedWithError). The gate reduces exactly to the old DestroyDB behavior when false and the old unconditional behavior when true, so the refactor is behavior-preserving while making the remaining asymmetry deliberate rather than accidental.
  • The earlier getCount Low is also addressed: countRemaining() polls isClosing() per row and reports the abort instead of a partial count, on both the database and transaction paths, and the inaccurate "compaction is the only unbounded in-flight op" comment is corrected.

Also checked and cleared: the dropped if (condition) null guard in PurgeIfUnreferenced is safe (both DBRegistryEntry constructors make_shared the condition; the old guard only mattered because the notify used to sit outside the if (descriptor) block); the newly-added closeRetrying = false on the PurgeAll/PurgeIfUnreferenced quarantine paths is a no-op, since only DestroyDB/Shutdown ever latch it and beginClose() is single-shot.

Verification: full suite 55 files, 773 passed / 1 skipped / 0 failed; targeted destroy + ranges 64/64 including the new aborts an in-flight getCount() when a foreign destroy begins fixture. CI green on the head (Windows jobs still pending at review time).

One merge-ordering note, not a defect in this PR: finishClose() still holds txnsMutex across cancelForDB(), which takes writerMutex_, and this PR makes that a routine path because destroy() now force-closes every descriptor. If #744 lands with PurgeIfUnreferenced still on the wake callback path, its lock-order inversion becomes materially more likely — worth sequencing #744's fix before or with this.


Generated by Barber AI

kriszyp added a commit that referenced this pull request Aug 25, 2026
…GetCount against concurrent close

finishClose() cleared compactCancelRequested right after the operationsInFlight
drain, but an async compact() releases its OperationGuard at setup handoff and
is not awaited until the closables sweep — so it can still be running after
the drain returns, and clearing the token there left it able to stall
teardown (and every concurrent open on the path) indefinitely. Keep the token
armed for finishClose()'s whole duration instead, and have the close-time
compact-on-close pass opt out via a new compactRange() `cancellable` param
rather than relying on the shared flag being cleared.

Transaction::GetCount now takes an OperationGuard and checks isClosing()
before scanning: without it, finishClose()'s drain can return immediately and
the closables sweep can roll back the transaction while the count scan is
parked between rows, reading freed memory.

Carries the in-progress PR #787 lifecycle repair plan describing the fuller
atomic-admission fix these two changes are a first slice of.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CvCKcGKG7vyhRts1Mv4Wh
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from f24ef7a to 5b459e4 Compare August 25, 2026 13:31
kriszyp added a commit that referenced this pull request Aug 25, 2026
…GetCount against concurrent close

finishClose() cleared compactCancelRequested right after the operationsInFlight
drain, but an async compact() releases its OperationGuard at setup handoff and
is not awaited until the closables sweep — so it can still be running after
the drain returns, and clearing the token there left it able to stall
teardown (and every concurrent open on the path) indefinitely. Keep the token
armed for finishClose()'s whole duration instead, and have the close-time
compact-on-close pass opt out via a new compactRange() `cancellable` param
rather than relying on the shared flag being cleared.

Transaction::GetCount now takes an OperationGuard and checks isClosing()
before scanning: without it, finishClose()'s drain can return immediately and
the closables sweep can roll back the transaction while the count scan is
parked between rows, reading freed memory.

Carries the in-progress PR #787 lifecycle repair plan describing the fuller
atomic-admission fix these two changes are a first slice of.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CvCKcGKG7vyhRts1Mv4Wh
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from 5b459e4 to 542b058 Compare August 25, 2026 14:59
@cb1kenobi

Copy link
Copy Markdown
Member

Reviewed 542b058a — no issues found. This PR looks good, nice job!

Re-review of the one new commit since f24ef7a5 (the branch was rebased onto main after rocksdb-js#744 merged; verified via git range-diff that all 35 prior commits carried forward unchanged modulo rebase context, with commit 542b058a new at the tip).

542b058a fixes a real race: compactCancelRequested now stays armed for finishClose()'s whole duration (an async compact-on-close pass opts out via a new cancellable param instead), and Transaction::GetCount now takes an OperationGuard + isClosing() check so finishClose()'s drain can't return early and let the closables sweep roll back the transaction mid-scan. Both changes are consistent with the existing OperationGuard/ACQUIRE_OPERATIONS_LOCK pattern elsewhere in the codebase.

Also re-verified at object-code level:

  • DBIterator::Next() has zero getenv calls in its compiled disassembly (ROCKSDB_JS_ITERATOR_NEXT_DELAY_MS is absent from db_iterator.o's string table); the seam is now a relaxed atomic load, set once in initializeTestSeams().
  • closeClaimedDescriptors() remains the single teardown tail for all four callers (CloseDB, DestroyDB, PurgeAll, Shutdown), with the completed-but-errored policy as the named ClaimedCloseOptions.failOnCompletedWithError option — false only for destroy(), true (fatal) everywhere else.
  • finishClose() still takes txnsMutex and holds it across cancelForDB(), which itself takes VT's writerMutex_ — the txnsMutex → writerMutex_ ordering is unchanged by rocksdb-js#744 merging.

pnpm test (destroy.test.ts lifecycle suite: 19/19), pnpm test:native (148/148, 3 expected macOS skips), and pnpm check all pass clean at this head.


Generated by Barber AI

kriszyp added a commit that referenced this pull request Aug 25, 2026
…GetCount against concurrent close

finishClose() cleared compactCancelRequested right after the operationsInFlight
drain, but an async compact() releases its OperationGuard at setup handoff and
is not awaited until the closables sweep — so it can still be running after
the drain returns, and clearing the token there left it able to stall
teardown (and every concurrent open on the path) indefinitely. Keep the token
armed for finishClose()'s whole duration instead, and have the close-time
compact-on-close pass opt out via a new compactRange() `cancellable` param
rather than relying on the shared flag being cleared.

Transaction::GetCount now takes an OperationGuard and checks isClosing()
before scanning: without it, finishClose()'s drain can return immediately and
the closables sweep can roll back the transaction while the count scan is
parked between rows, reading freed memory.

Carries the in-progress PR #787 lifecycle repair plan describing the fuller
atomic-admission fix these two changes are a first slice of.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CvCKcGKG7vyhRts1Mv4Wh
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch 2 times, most recently from 542b058 to 3cdd9f9 Compare August 25, 2026 17:20
Comment thread src/binding/database/db_registry.cpp
kriszyp added a commit that referenced this pull request Aug 26, 2026
…GetCount against concurrent close

finishClose() cleared compactCancelRequested right after the operationsInFlight
drain, but an async compact() releases its OperationGuard at setup handoff and
is not awaited until the closables sweep — so it can still be running after
the drain returns, and clearing the token there left it able to stall
teardown (and every concurrent open on the path) indefinitely. Keep the token
armed for finishClose()'s whole duration instead, and have the close-time
compact-on-close pass opt out via a new compactRange() `cancellable` param
rather than relying on the shared flag being cleared.

Transaction::GetCount now takes an OperationGuard and checks isClosing()
before scanning: without it, finishClose()'s drain can return immediately and
the closables sweep can roll back the transaction while the count scan is
parked between rows, reading freed memory.

Carries the in-progress PR #787 lifecycle repair plan describing the fuller
atomic-admission fix these two changes are a first slice of.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CvCKcGKG7vyhRts1Mv4Wh
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from 107b216 to ab5cd96 Compare August 26, 2026 15:02
Comment thread AGENTS.md Outdated
Comment thread src/binding/database/db_descriptor.h

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase with main and resolve the merge conflicts.

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed ba0e1c0 and found no blocking issues. No new blocking defects were confirmed on changed lines. Existing findings were not repeated.


Generated by Barber AI

kriszyp added a commit that referenced this pull request Sep 2, 2026
…GetCount against concurrent close

finishClose() cleared compactCancelRequested right after the operationsInFlight
drain, but an async compact() releases its OperationGuard at setup handoff and
is not awaited until the closables sweep — so it can still be running after
the drain returns, and clearing the token there left it able to stall
teardown (and every concurrent open on the path) indefinitely. Keep the token
armed for finishClose()'s whole duration instead, and have the close-time
compact-on-close pass opt out via a new compactRange() `cancellable` param
rather than relying on the shared flag being cleared.

Transaction::GetCount now takes an OperationGuard and checks isClosing()
before scanning: without it, finishClose()'s drain can return immediately and
the closables sweep can roll back the transaction while the count scan is
parked between rows, reading freed memory.

Carries the in-progress PR #787 lifecycle repair plan describing the fuller
atomic-admission fix these two changes are a first slice of.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CvCKcGKG7vyhRts1Mv4Wh
Comment thread src/binding/database/db_handle.cpp Outdated
kriszyp added a commit that referenced this pull request Sep 9, 2026
Rebase of PR #787 onto latest main. main independently landed a
resolved-identity/per-entry destroy-vs-open gate (secondary/read-only
aware) while this branch was in flight, so the two designs are merged
here rather than replayed commit-by-commit:

- Keep main's DBKey{identityPath, readOnly, secondaryPath} cross-key
  gate as the identity/correctness foundation instead of this PR's
  original path-only destroyingPaths set.
- Layer this PR's close/destroy quarantine on top: a failed close or
  destroy leaves the registry entry with closeError set instead of
  erasing or leaking it, so shutdown()/destroy() can retry and
  registryStatus()/`database:closeFailed` surface the failure.
  destroyingPaths returns as a path-level gate solely for the window
  between "registry entries erased" and "physical files deleted" --
  that deletion runs without databasesMutex held (it's I/O), so the
  registry alone can't gate a concurrent open once its entries are
  gone.
- Bound every lifecycle wait (open/destroy/shutdown) by a new
  lifecycleWaitSeconds setting (default 30s) instead of waiting
  unboundedly, consistent with this codebase's existing
  ROCKSDB_JS_PARK_TIMEOUT_MS precedent.
- Add DBRegistry::Teardown(), run from the module's last env-cleanup
  hook after Shutdown(): a descriptor quarantined at process exit must
  not survive to the registry singleton's static destructor, which
  runs after RocksDB's own statics and aborts the process.
- Move handle attach/adoption inside DBRegistry::OpenDB() under the
  same lock as its lifecycle waits (was: caller-side, after the lock
  released) -- closes a real race in main's current code where a
  concurrent destroy() could tear down a descriptor between OpenDB()
  returning and the caller's separate attach() call.
- Add close-time compaction cancellation (two tokens: descriptor-wide
  for synchronous callers, per-handle for async, per invariant 6) and
  make async-work admission/cancellation atomic under one mutex so a
  registration can never land after a close has already observed zero
  in-flight work.
- Serialize iterator calls against a foreign forced close and add a
  destroy()-tolerant readOnly-aware destroy() precondition on the JS
  side (Store::readOnly, checked even for a never-opened handle).

AGENTS.md invariants 17, 19 (renumbered from main's 18), 21-23
document the merged design; invariant 6 documents the quarantine and
compaction-cancellation additions in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from d082b42 to e728a97 Compare September 9, 2026 01:21
Comment thread src/binding/binding.cpp
kriszyp added a commit that referenced this pull request Sep 9, 2026
Rebase of PR #787 onto latest main. main independently landed a
resolved-identity/per-entry destroy-vs-open gate (secondary/read-only
aware) while this branch was in flight, so the two designs are merged
here rather than replayed commit-by-commit:

- Keep main's DBKey{identityPath, readOnly, secondaryPath} cross-key
  gate as the identity/correctness foundation instead of this PR's
  original path-only destroyingPaths set.
- Layer this PR's close/destroy quarantine on top: a failed close or
  destroy leaves the registry entry with closeError set instead of
  erasing or leaking it, so shutdown()/destroy() can retry and
  registryStatus()/`database:closeFailed` surface the failure.
  destroyingPaths returns as a path-level gate solely for the window
  between "registry entries erased" and "physical files deleted" --
  that deletion runs without databasesMutex held (it's I/O), so the
  registry alone can't gate a concurrent open once its entries are
  gone.
- Bound every lifecycle wait (open/destroy/shutdown) by a new
  lifecycleWaitSeconds setting (default 30s) instead of waiting
  unboundedly, consistent with this codebase's existing
  ROCKSDB_JS_PARK_TIMEOUT_MS precedent.
- Add DBRegistry::Teardown(), run from the module's last env-cleanup
  hook after Shutdown(): a descriptor quarantined at process exit must
  not survive to the registry singleton's static destructor, which
  runs after RocksDB's own statics and aborts the process.
- Move handle attach/adoption inside DBRegistry::OpenDB() under the
  same lock as its lifecycle waits (was: caller-side, after the lock
  released) -- closes a real race in main's current code where a
  concurrent destroy() could tear down a descriptor between OpenDB()
  returning and the caller's separate attach() call.
- Add close-time compaction cancellation (two tokens: descriptor-wide
  for synchronous callers, per-handle for async, per invariant 6) and
  make async-work admission/cancellation atomic under one mutex so a
  registration can never land after a close has already observed zero
  in-flight work.
- Serialize iterator calls against a foreign forced close and add a
  destroy()-tolerant readOnly-aware destroy() precondition on the JS
  side (Store::readOnly, checked even for a never-opened handle).

AGENTS.md invariants 17, 19 (renumbered from main's 18), 21-23
document the merged design; invariant 6 documents the quarantine and
compaction-cancellation additions in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from e728a97 to 42b42aa Compare September 9, 2026 02:50
kriszyp added a commit that referenced this pull request Sep 9, 2026
Rebase of PR #787 onto latest main. main independently landed a
resolved-identity/per-entry destroy-vs-open gate (secondary/read-only
aware) while this branch was in flight, so the two designs are merged
here rather than replayed commit-by-commit:

- Keep main's DBKey{identityPath, readOnly, secondaryPath} cross-key
  gate as the identity/correctness foundation instead of this PR's
  original path-only destroyingPaths set.
- Layer this PR's close/destroy quarantine on top: a failed close or
  destroy leaves the registry entry with closeError set instead of
  erasing or leaking it, so shutdown()/destroy() can retry and
  registryStatus()/`database:closeFailed` surface the failure.
  destroyingPaths returns as a path-level gate solely for the window
  between "registry entries erased" and "physical files deleted" --
  that deletion runs without databasesMutex held (it's I/O), so the
  registry alone can't gate a concurrent open once its entries are
  gone.
- Bound every lifecycle wait (open/destroy/shutdown) by a new
  lifecycleWaitSeconds setting (default 30s) instead of waiting
  unboundedly, consistent with this codebase's existing
  ROCKSDB_JS_PARK_TIMEOUT_MS precedent.
- Add DBRegistry::Teardown(), run from the module's last env-cleanup
  hook after Shutdown(): a descriptor quarantined at process exit must
  not survive to the registry singleton's static destructor, which
  runs after RocksDB's own statics and aborts the process.
- Move handle attach/adoption inside DBRegistry::OpenDB() under the
  same lock as its lifecycle waits (was: caller-side, after the lock
  released) -- closes a real race in main's current code where a
  concurrent destroy() could tear down a descriptor between OpenDB()
  returning and the caller's separate attach() call.
- Add close-time compaction cancellation (two tokens: descriptor-wide
  for synchronous callers, per-handle for async, per invariant 6) and
  make async-work admission/cancellation atomic under one mutex so a
  registration can never land after a close has already observed zero
  in-flight work.
- Serialize iterator calls against a foreign forced close and add a
  destroy()-tolerant readOnly-aware destroy() precondition on the JS
  side (Store::readOnly, checked even for a never-opened handle).

AGENTS.md invariants 17, 19 (renumbered from main's 18), 21-23
document the merged design; invariant 6 documents the quarantine and
compaction-cancellation additions in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from c9cecc1 to 215ee4d Compare September 9, 2026 05:19
kriszyp added a commit that referenced this pull request Sep 9, 2026
Rebase of PR #787 onto latest main. main independently landed a
resolved-identity/per-entry destroy-vs-open gate (secondary/read-only
aware) while this branch was in flight, so the two designs are merged
here rather than replayed commit-by-commit:

- Keep main's DBKey{identityPath, readOnly, secondaryPath} cross-key
  gate as the identity/correctness foundation instead of this PR's
  original path-only destroyingPaths set.
- Layer this PR's close/destroy quarantine on top: a failed close or
  destroy leaves the registry entry with closeError set instead of
  erasing or leaking it, so shutdown()/destroy() can retry and
  registryStatus()/`database:closeFailed` surface the failure.
  destroyingPaths returns as a path-level gate solely for the window
  between "registry entries erased" and "physical files deleted" --
  that deletion runs without databasesMutex held (it's I/O), so the
  registry alone can't gate a concurrent open once its entries are
  gone.
- Bound every lifecycle wait (open/destroy/shutdown) by a new
  lifecycleWaitSeconds setting (default 30s) instead of waiting
  unboundedly, consistent with this codebase's existing
  ROCKSDB_JS_PARK_TIMEOUT_MS precedent.
- Add DBRegistry::Teardown(), run from the module's last env-cleanup
  hook after Shutdown(): a descriptor quarantined at process exit must
  not survive to the registry singleton's static destructor, which
  runs after RocksDB's own statics and aborts the process.
- Move handle attach/adoption inside DBRegistry::OpenDB() under the
  same lock as its lifecycle waits (was: caller-side, after the lock
  released) -- closes a real race in main's current code where a
  concurrent destroy() could tear down a descriptor between OpenDB()
  returning and the caller's separate attach() call.
- Add close-time compaction cancellation (two tokens: descriptor-wide
  for synchronous callers, per-handle for async, per invariant 6) and
  make async-work admission/cancellation atomic under one mutex so a
  registration can never land after a close has already observed zero
  in-flight work.
- Serialize iterator calls against a foreign forced close and add a
  destroy()-tolerant readOnly-aware destroy() precondition on the JS
  side (Store::readOnly, checked even for a never-opened handle).

AGENTS.md invariants 17, 19 (renumbered from main's 18), 21-23
document the merged design; invariant 6 documents the quarantine and
compaction-cancellation additions in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from 215ee4d to 5e8803d Compare September 9, 2026 20:25
kriszyp and others added 14 commits September 9, 2026 14:28
Rebase of PR #787 onto latest main. main independently landed a
resolved-identity/per-entry destroy-vs-open gate (secondary/read-only
aware) while this branch was in flight, so the two designs are merged
here rather than replayed commit-by-commit:

- Keep main's DBKey{identityPath, readOnly, secondaryPath} cross-key
  gate as the identity/correctness foundation instead of this PR's
  original path-only destroyingPaths set.
- Layer this PR's close/destroy quarantine on top: a failed close or
  destroy leaves the registry entry with closeError set instead of
  erasing or leaking it, so shutdown()/destroy() can retry and
  registryStatus()/`database:closeFailed` surface the failure.
  destroyingPaths returns as a path-level gate solely for the window
  between "registry entries erased" and "physical files deleted" --
  that deletion runs without databasesMutex held (it's I/O), so the
  registry alone can't gate a concurrent open once its entries are
  gone.
- Bound every lifecycle wait (open/destroy/shutdown) by a new
  lifecycleWaitSeconds setting (default 30s) instead of waiting
  unboundedly, consistent with this codebase's existing
  ROCKSDB_JS_PARK_TIMEOUT_MS precedent.
- Add DBRegistry::Teardown(), run from the module's last env-cleanup
  hook after Shutdown(): a descriptor quarantined at process exit must
  not survive to the registry singleton's static destructor, which
  runs after RocksDB's own statics and aborts the process.
- Move handle attach/adoption inside DBRegistry::OpenDB() under the
  same lock as its lifecycle waits (was: caller-side, after the lock
  released) -- closes a real race in main's current code where a
  concurrent destroy() could tear down a descriptor between OpenDB()
  returning and the caller's separate attach() call.
- Add close-time compaction cancellation (two tokens: descriptor-wide
  for synchronous callers, per-handle for async, per invariant 6) and
  make async-work admission/cancellation atomic under one mutex so a
  registration can never land after a close has already observed zero
  in-flight work.
- Serialize iterator calls against a foreign forced close and add a
  destroy()-tolerant readOnly-aware destroy() precondition on the JS
  side (Store::readOnly, checked even for a never-opened handle).

AGENTS.md invariants 17, 19 (renumbered from main's 18), 21-23
document the merged design; invariant 6 documents the quarantine and
compaction-cancellation additions in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…on, update docs

Test coverage for the rebased lifecycle work: destroy/close quarantine
and retry, bounded lifecycle waits, the destroy-vs-open path gate,
compaction cancellation on all four close paths (self, foreign close,
destroy, shutdown), async-work admission mutual exclusion, and
iterator/count races against a concurrent forced close.

test/secondary.test.ts's "settle an in-flight catch-up when the
database is torn down" is updated for the merged design: destroy()
now waits out and tolerates a lingering async-state reference the same
way an in-flight checkpoint/backup already does (see
test/checkpoint.test.ts and test/backup-stream.test.ts), rather than
failing on it the way the old refcount-based design did -- there was
no principled reason for catchUpWithPrimary to be the one operation
treated differently here, and the two now agree.

README.md documents the quarantine/retry contract for close(),
destroy(), shutdown(), and registryStatus(); AGENTS.md documents the
new env vars.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ng, destroy flush waste

- CloseDB and the iterator finalizer detached from `closables` before
  close()/Reset(), leaving a concurrent destroy()/shutdown() sweep unable to
  see (and wait for) a handle/iterator still draining async work or mid-Next()
  -- a real use-after-free window on the shared rocksdb::DB. Detach after
  close() returns instead; closeMutex/iteratorMutex already serialize a
  foreign close arriving in that window.
- DBHandle::close() released `logRefs` napi_refs gated on a std::thread::id
  equality check, the same recycled-pthread-id hazard invariant 18 already
  fixed for TransactionHandle. Add DBRegistry::ReleaseLogRefsByEnv, wired into
  the env cleanup hook like CloseTransactionsByEnv, so a dying env's logRefs
  are emptied while it is still alive -- before its thread id could ever be
  reused against the stale guard.
- OpenDB's "still closing"/"retry in progress" waits parked on one matching
  descriptor's condition variable but re-scanned the whole path in their
  predicate, so two descriptors closing on one path (e.g. a writable and a
  secondary) could leave an opener asleep for the full lifecycleWaitSeconds
  even after the path was free. Track the specific selected entry instead.
- finishClose(destroying=true) still ran a full flush + close-time compaction
  before the files are unlinked -- wasted I/O, and with allow_write_stall's
  default a destroy() of a stalled database could hang indefinitely while
  holding the destroyingPaths gate. Skip both when destroying.
- Add a fixture covering a lifecycleWaitSeconds timeout actually firing and
  the path recovering afterward (previously only config validation was
  tested).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01122SfiNfCtiLgWvQTD6wZ7
…eal two-descriptor regression test

- fork-lifecycle-timeout.mts raced the shutdown retry claim against the main
  thread's open() with no barrier, so under load the open could win and throw
  the wrong error instead of timing out -- a flake, not a proof. Expose
  `closeRetrying` on registryStatus() and poll for it before racing the open.
- fork-compact-cancel-destroy.mts lost its discriminating power once destroy()
  skips compactOnClose (previous commit): with nothing left to block on
  compactMutex, an early vs. late cancellation arm became timing-indistinguishable.
  Drive it through shutdown() instead, which still runs compactOnClose.
  Verified: reverting the early arm now makes the fixture fail again (7.7s vs
  the ~500ms bound), confirming this restores the fixture's purpose.
- Added a genuine two-descriptor regression test for the OpenDB
  condition-variable/predicate fix (db_registry.cpp:603/:655): a writable and
  a read-only descriptor quarantine, then retry concurrently under one
  shutdown() call, with an opener racing in. Verified against the pre-fix
  predicate: 4/4 runs stalled to the full 8s deadline instead of the expected
  ~3s, confirming this catches the regression the prior test explicitly could
  not.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01122SfiNfCtiLgWvQTD6wZ7
shutdown() is process-wide, not scoped to one path, and its worker call could
still be re-scanning for anything left to close after the racing open() call
returned but before it posted shutdownResult -- a handle opened in that
window is not safe to read from or hold onto (shutdown() could sweep and
force-close it too). Keep the timing-only open (the actual regression proof)
but defer the data-preservation check to a fresh open, strictly after the
worker's shutdown() call and the worker itself are both confirmed done.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01122SfiNfCtiLgWvQTD6wZ7
Review follow-up on the `shutdown()`-throws thread: the behavior is kept
(db.close() throws the identical quarantine error, and three of the four
throw sites are lifecycle timeouts that `database:closeFailed` cannot
carry), but the only place it was written down was the README. The
exported symbol now carries the same contract, including why a
`process.on('exit')` listener has to wrap the call: a throw from an exit
listener skips every listener registered after it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…eams

Independent pre-push review round 1 on the rebased head (codex + gemini +
harper-domain) found five real items:

- `DBHandle::open()` now drops the previous lifecycle's `logRefs`. The
  owner-thread guard that makes a foreign close napi-safe (AGENTS.md
  invariant 18) also means a cross-env `destroy()`/`shutdown()` leaves the
  cache populated, so a reopened handle handed `useLog()` back a
  `TransactionLog` whose store `weak_ptr` pointed at the unregistered store
  of the closed lifecycle. Only `addEntry` re-resolves, so every read
  accessor reported an empty log — `getLogFileSize()` returned 0 instead of
  31 in the new fixture, which fails 1/1 without the fix.
- `registryStatus()`'s destroy-cleanup tombstone branch omitted
  `transactionDetails`, which `RegistryStatusDB` declares non-optional; a
  monitor reading `entry.transactionDetails.length` threw on exactly the
  entry shape that only appears when a physical destroy failed.
- The three test-delay seams this PR added (`ROCKSDB_JS_BACKUP_DELAY_MS`,
  `ROCKSDB_JS_DESTROY_DELAY_MS`, `ROCKSDB_JS_CLOSE_RETRY_DELAY_MS`) still
  called `::getenv` from a libuv worker / arbitrary teardown thread. They
  are snapshotted in `initializeTestSeams()` now, like the fault flags
  beside them.
- `fork-shutdown-retry.mts` raced the worker's retry claim: the worker posts
  before calling `shutdown()`, so the parent's open could hit the still
  quarantined entry. It now polls `closeRetrying` first and re-reads data
  from a fresh handle after the worker is done — the two barriers the
  two-descriptor fixture already had.
- `fork-compact-cancel-async.mts` had no `'message'` listener attached
  across `await outcome`, so a `{destroyed:true}` posted in that window was
  dropped and the fixture would hang to its timeout.

Declined, with reasons in the PR body: making `isClosing()` a relaxed load
(the reviewer's own estimate is that it is dwarfed by `iterator->Next()`,
and the flag is read under mutexes elsewhere), and the aggregate
comment-narration nit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI turned red on macOS (Bun + Deno) with four quarantine tests failing on
`/private/var/...` vs `/var/...`: a tombstoned `registryStatus()` entry and
every `database:closeFailed` event reported the registry key's resolved
identity, so a caller matching either against the path it opened did not
recognize it. That is exactly what AGENTS.md invariant 19 forbids
("returning only the resolved identity breaks callers that match paths
against the spelling they supplied") and what `registryStatus()` already
does correctly for a live descriptor.

- `emitCloseFailures()` reports `descriptor->path`.
- `DBRegistryEntry::reportedPath` remembers the opening caller's spelling so
  a destroy-cleanup tombstone — which has no descriptor left to ask — can
  still report it, in `registryStatus()` and in its own emit.

Not caused by the rebase (the pre-rebase head carried identical code and was
green on the older macos-26-arm64 runner image); the new image's TMPDIR made
the latent mismatch reachable. `test/destroy.test.ts` now covers it on every
platform with an explicit symlink instead of depending on macOS's `/var`
link — verified to fail with `expected undefined to be true`, the same
assertion macOS CI produced, when either report site is reverted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tombstone

Round 3 of the independent pre-push review (codex) found the one gap in the
previous commit: a second `destroy()` after a failed physical cleanup starts
with `reportedPath = identityPath` and its scan skipped the existing
tombstone, because that entry has no descriptor. So `registryStatus().path`
kept reporting the opened spelling while the retry's `database:closeFailed`
reverted to the resolved identity — the two disagreeing is the same defect
one step later. The scan now falls back to a matching entry's remembered
spelling, still preferring a live descriptor's.

The symlink test asserts the event path on the retry as well, and fails with
`expected [ …(2) ] to match object [ …(2) ]` without the fallback.

Also dropped from that round, verified rather than assumed: Gemini's blocker
on `checkpoint.cpp` leaking `operationsInFlight` when admission or queueing
fails — `CheckpointInFlightClaim` is the RAII equivalent of
`BackupInFlightClaim` and `handedOff` is only set after both succeed
(`src/binding/database/checkpoint.cpp:69-76,114-115,206`).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI's remaining SIGSEGV (`fork-destroy-open.mts`, one Deno/Bun job per run
since the rebase) is a real native use-after-free on the JS thread, not a
flake. Reproduced locally at 2/60 by running that fixture 6-way concurrently,
and gdb put it exactly here:

  __strlen_avx2
  v8::String::NewFromUtf8
  napi_set_named_property
  rocksdb_js::DBRegistry::RegistryStatus

`registryStatus()` holds `databasesMutex`, which covers the registry map but
not a descriptor's own maps. It walked `descriptor->columns` — guarded by
`columnsMutex` — while a cross-env `destroy()`'s `finishClose()` cleared that
map from the worker thread, so `name.c_str()` pointed into a freed map node
and `napi_set_named_property()` `strlen()`ed it. `locks.size()` was read
unguarded the same way (a count, so a torn read rather than a fault).

The column summary is now snapshotted under `columnsMutex` (plus the per-CF
`userSharedBuffersMutex` for its buffer count) and the JS values are built
after releasing it — holding it across the N-API calls would risk a finalizer
re-entering the same non-recursive mutex on this thread, which is why
`transactions` already had this shape under `txnsMutex`. `locks.size()` is
read under `locksMutex`. Neither adds a lock-order inversion: `OpenDB` and
`CollectWriteBufferManagerInventory` already establish
`databasesMutex → columnsMutex`, `getUserSharedBuffer` is a leaf, and no
`locksMutex` region reaches the registry.

Pre-existing on `main` (its `registryStatus()` walks `columns` unguarded too);
this PR is what makes it reachable, because `destroy()` now force-closes every
descriptor and its own fixtures poll `registryStatus()` across that window.

`test/fixtures/fork-registry-status-column-race.mts` makes it deterministic
rather than leaving it to CI luck: a worker churns `dropSync()` against the
shared descriptor while the main thread polls `registryStatus()`, with the new
`ROCKSDB_JS_REGISTRY_STATUS_COLUMNS_DELAY_MS` seam parking the walk per column
family so an erase lands inside it. Column names run past libstdc++'s 15-char
small-string buffer so the erase frees a separate heap allocation (an SSO name
usually survives the free intact and hides the bug). 5/5 abort without the
fix, 3/3 clean with it; the concurrent `fork-destroy-open.mts` stress went
2/60 → 0/132.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TMokk8DsJyGwpHz4Lmts85
…iming

Round 5 of the independent pre-push review (codex, seconded by the domain
adjudicator) caught `test/fixtures/fork-registry-status-destroy-race.mts`
shipping unreferenced: it was the first attempt at the `registryStatus()`
column-walk regression test and does not reproduce (0/7 against the unfixed
walk), because a single close-time `columns.clear()` rarely lands inside a
walk and a small-string column name usually survives the free intact. The
drop-churn fixture that replaced it does reproduce deterministically (5/5),
and it covers the same thing — the fix is the snapshot in the walk, not
anything per-mutator, so reverting that snapshot fails the churn fixture too.

Also reworded the tombstone branch's comment, which claimed it fills "every
non-optional field of RegistryStatusDB": `userSharedBuffers` is declared
non-optional and set by no branch, and `columnFamilies` is typed `string[]`
against an object. Both predate this PR and are noted rather than fixed here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TMokk8DsJyGwpHz4Lmts85
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
@kriszyp
kriszyp force-pushed the kris/serialize-destroy-open branch from 5e8803d to 8784294 Compare September 9, 2026 20:44
Comment thread src/binding/database/db_registry.cpp
kriszyp and others added 3 commits September 9, 2026 15:52
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Co-Authored-By: GPT-5 Codex <noreply@openai.com>

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a big one, but it looks great! I love the OperationGuard!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants