Outcome
Prove the actual Harper RocksDB-backed Tantivy path using existing supported storage APIs before freezing the integration factory.
Acceptance criteria
- Run real Harper registration -> projected batch -> Tantivy index -> RocksDB objects -> commit -> reload -> query -> close -> restart/reopen.
- Resolve the bounded request/response or staging transport from native Directory calls to Harper-owned store workers/views; verify exact APIs and source revisions.
- Keep post-commit hooks nonblocking; bound bytes/requests and prove storage work never waits cyclically on its own JS environment.
- Exercise open_write, immediate readability, repeated flush, writer continuation, immutable slices, delete-while-open, locks/watch and atomic metadata against pinned Tantivy.
- Evaluate existing WAL-enabled writes/transactions and supported durability barriers; prove referenced objects durable before publication acknowledgement.
- Test storage-worker exit, close/drop/recreate, crash at publication, and source-commit-before-delivery replay through the shared Harper protocol.
- Measure event-loop delay, transport wait, copies, storage service time, indexing throughput, query latency and lag with multiple indexes.
- Report any missing correctness guarantee explicitly; do not substitute the unmerged native lease, private rocksdb-js build, whole-index RAM or native files.
Reuse experimental Directory tests where applicable. The previous lease experiment is evidence, not completion of this new gate. Exact source progress/resume remains Harper-owned protocol work.
-
Before representative timing, assert offset-addressable reads, bounded storage operations/bytes per range and MB written, and no directory-wide lock across unrelated host I/O.
-
Automate the real Harper slice in CI, including delayed/reordered/lost storage responses, exact write-option auditing, duplicate worker attachment and second writable-process rejection.
-
Choose fixed-size chunks or a bounded offset index within this first slice. Record metadata lookup bounds, chunk-size sweeps, read/write/space amplification, append/flush behavior, format versioning, offset-metadata atomicity and reclamation reachability before treating timings as representative.
-
Prove a process-wide bound on native threads blocked on storage, including indexing, merge and search threads and any threads waiting for admission permits. Capacity must be established before work starts, with query/control completion progress preserved.
-
Exercise bounded incremental orphan discovery and graceful/forced teardown under real Harper lifecycle ordering; no full-generation blocking startup sweep or release of state still reachable by native tasks.
-
Attribute event-loop delay on storage-serving JS environments, handoff/queue time, storage calls and native blocked time separately; measure blocked-thread occupancy against its configured budget.
-
Identify the supported enforcement point for the process-wide thread budget. Account for caller-sized indexing threads, package search threads and the pinned engine's internal merge pool; if the latter cannot be resized, constrain resident writers and concurrent opens/rebuilds using its verified thread cost. A transport permit does not bound threads waiting outside it.
Design
Tantivy storage through Harper. The Harper integration is planned; the native backend is implemented. No production change to base rocksdb-js is required by this work.
Comment generated by kAIle (GPT-6)
Outcome
Prove the actual Harper RocksDB-backed Tantivy path using existing supported storage APIs before freezing the integration factory.
Acceptance criteria
Reuse experimental Directory tests where applicable. The previous lease experiment is evidence, not completion of this new gate. Exact source progress/resume remains Harper-owned protocol work.
Before representative timing, assert offset-addressable reads, bounded storage operations/bytes per range and MB written, and no directory-wide lock across unrelated host I/O.
Automate the real Harper slice in CI, including delayed/reordered/lost storage responses, exact write-option auditing, duplicate worker attachment and second writable-process rejection.
Choose fixed-size chunks or a bounded offset index within this first slice. Record metadata lookup bounds, chunk-size sweeps, read/write/space amplification, append/flush behavior, format versioning, offset-metadata atomicity and reclamation reachability before treating timings as representative.
Prove a process-wide bound on native threads blocked on storage, including indexing, merge and search threads and any threads waiting for admission permits. Capacity must be established before work starts, with query/control completion progress preserved.
Exercise bounded incremental orphan discovery and graceful/forced teardown under real Harper lifecycle ordering; no full-generation blocking startup sweep or release of state still reachable by native tasks.
Attribute event-loop delay on storage-serving JS environments, handoff/queue time, storage calls and native blocked time separately; measure blocked-thread occupancy against its configured budget.
Identify the supported enforcement point for the process-wide thread budget. Account for caller-sized indexing threads, package search threads and the pinned engine's internal merge pool; if the latter cannot be resized, constrain resident writers and concurrent opens/rebuilds using its verified thread cost. A transport permit does not bound threads waiting outside it.
Design
Tantivy storage through Harper. The Harper integration is planned; the native backend is implemented. No production change to base rocksdb-js is required by this work.
Comment generated by kAIle (GPT-6)