Skip to content

kv: answer entity-type matches at a temporal basis from the TYPE index - #46

Draft
bjacobso wants to merge 1 commit into
mainfrom
worldvm/kv-memory-performance
Draft

bjacobso wants to merge 1 commit into
mainfrom
worldvm/kv-memory-performance

Conversation

@bjacobso

Copy link
Copy Markdown
Owner

Problem

Public Triples reads always resolve a temporal basis before they reach the KV layer (resolveTemporalBasis(basis, yield* runtime.now)). The entity-type fast path in KvTriplesLive.match only ran when basis === undefined, so it never ran for public reads. Every match({ entityType }) became scanCollectTemporalAsync({}, basis): a temporal scan of the whole store, transaction journal included.

ConfigStore.load issues five of these reads (objects, types, revisions, snapshots, refs), so every ConfigRuntime.evaluate scanned every fact five times.

Change

  • KvTripleStore.getByEntityTypeTemporal (and an async twin) read the TYPE index, then keep the datoms visible at the basis. Retraction only rewrites META, so the index still names every datom ever asserted with the type, and visibleAt decides recorded and valid time exactly as the full scan did.
  • KvTriplesLive.match uses them whenever a basis is present.
  • The existing non-temporal lookups share the same helpers.

Evidence

  • New test in KvTriples.store.test.ts: entity-type matches at three bases (before a retraction in recorded time, after it, and in a later valid-time interval) return the right triples. A counting KV backend shows the three reads touch 6 entries. Without the fix they touch 4,548.
  • On an in-memory store with 16,000 facts, ConfigRuntime.evaluate takes 1–3 ms with the fix and 480–985 ms without it.
  • pnpm --filter @triplex-build/triplex typecheck, test (712 tests), pnpm lint and pnpm format:check pass.

Context

Found while running Triplex core in the browser, in memory, behind WorldVM's programmable-operations demo, where requirement statuses become ConfigRuntime decisions. WorldVM currently works around this by caching ref resolutions.

The same exploration measured the in-memory write path, which this PR does not change. transact costs about 165 µs per fact in Chrome and 250–300 µs in Node (16k facts: 2.6 s in Chrome; 82k facts: 14.7 s). The time goes to byte-encoding four or five index keys per triple, a per-key Effect plus a char-by-char latin1 conversion inside transactions (the backend's setAllStr fast path isn't used there), one journal triple per change (which roughly doubles the writes), and GC. That's a candidate for a follow-up.

🤖 Generated with Claude Code

Public Triples reads always resolve a temporal basis before reaching the
KV layer, so the entity-type fast path (basis === undefined) never ran
for them: every match({ entityType }) became a temporal scan of the
whole store, journal included. ConfigStore.load issues five of these,
so each ConfigRuntime.evaluate scanned every fact five times.

Add getByEntityTypeTemporal (and its async twin) to KvTripleStore: read
the TYPE index, then keep the datoms visible at the basis. Retraction
only rewrites META, so the index still names every datom asserted with
the type and visibleAt decides recorded and valid time as before.

On an in-memory store with 16,000 facts, ConfigRuntime.evaluate drops
from 480-985 ms to 1-3 ms. Found while running Triplex in the browser
behind WorldVM's programmable-operations demo.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bjacobso added a commit that referenced this pull request Oct 30, 2026
…ory to the entity

Entity-type matches at a temporal basis now read the TYPE index (the cherry-picked fix from
#46). Attached segments join that path through their own type index, with the same visibility
check, so a segment fact retracted later is still visible at an earlier basis.

`transactionsForEntity` hydrated an attachment's record with every fact in its segment. It now
lists only that entity's facts, found by binary search, so an entity's history page stays small
however large the segment is. `transaction(txId)` still derives the complete change list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant