In-memory bulk loads: segments attached in one transaction, position refs, faster KV writes

.changeset/import-segments.md

@@ -0,0 +1,12 @@
+---
+"@triplex-build/triplex": minor
+---
+
+Add experimental import segments to `@triplex-build/triplex/operational`. `Segment.build` sorts
+and indexes a batch of facts in one pass into an immutable, content-addressed segment;
+`Segment.encode` and `decode` move it as one buffer; `Segment.attach` adds it to an in-memory KV
+store in one transaction that journals a single summary. Reads merge attached segments with
+indexed facts at any basis, retracting a segment fact is ordinary history, and
+`Import.run(..., { store: "segments" })` builds and attaches one segment per chunk before
+publishing a `PositionRef`. `Sha256.create` adds an incremental hasher, and SHA-256 is about
+2.7 times faster.

.changeset/incremental-import.md

@@ -0,0 +1,7 @@
+---
+"@triplex-build/triplex": minor
+"@triplex-build/triplex-testkit": minor
+---
+
+Add `Import` to `@triplex-build/triplex/operational`: commit a large dataset in chunks, yielding to
+the host between them, and publish it atomically by moving a `PositionRef`.

.changeset/kv-batched-writes.md

@@ -0,0 +1,8 @@
+---
+"@triplex-build/triplex": patch
+---
+
+Make KV transactions write their assertions in one batch. The in-memory backend buffers them by
+latin1 string key without a per-entry `Effect`, merges new keys into its sorted index instead of
+re-sorting after every commit, and encodes ASCII keys without per-character concatenation.
+`KvTransaction` gains optional `setAll` and `setAllStr` members.

.changeset/position-refs.md

@@ -0,0 +1,8 @@
+---
+"@triplex-build/triplex": minor
+"@triplex-build/triplex-testkit": minor
+---
+
+Add `PositionRef` to `@triplex-build/triplex/operational`: named, compare-and-swap pointers to
+committed positions, moved in one journaled transaction and read back as a `recordedPosition`
+basis.

.changeset/recorded-position-basis.md

@@ -0,0 +1,8 @@
+---
+"@triplex-build/triplex": minor
+"@triplex-build/triplex-testkit": minor
+---
+
+Accept `recordedPosition` in the public temporal basis. A read at a commit position sees exactly the
+transactions committed at or before it, even when commits share a millisecond. Paged Datalog and
+entity pages clamp it to the latest commit, and derivation runs persist it.

.changeset/summary-journal.md

@@ -0,0 +1,8 @@
+---
+"@triplex-build/triplex": minor
+"@triplex-build/triplex-testkit": minor
+---
+
+Add `journal: "summary"` to `TransactionMeta`. Such a transaction records one `:_tx/summary`
+(counts, changed attributes, and a content digest) instead of one `:_tx/change` per fact; its
+record and result carry a `summary` and its record has no `changes`.

docs/.vitepress/config.ts

@@ -33,6 +33,7 @@ const sidebar: DefaultTheme.SidebarItem[] = [
     items: [
       { text: "Configuration", link: "/configuration" },
       { text: "Operational primitives", link: "/operational-primitives" },
+      { text: "Import segments", link: "/segments" },
       { text: "Configuration HTTP API", link: "/http-api" },
       { text: "Performance", link: "/performance" },
       {

docs/agents.md

@@ -139,8 +139,9 @@ scheduler that reopens work at expiry. Read more in [Derivations](/derivations).
 
 - **What did we believe then about what was true then?** Set both `recordedAt` and `validAt`.
   Setting only one leaves the other at "now".
-- **What did we know when we made a decision?** Use the decision's receipt `instant` as
-  `recordedAt`. Find the receipt with `transactionByCommand`.
+- **What did we know when we made a decision?** Use the decision's receipt `position` as
+  `recordedPosition` (or its `instant` as `recordedAt`, which can't separate commits in the same
+  millisecond). Find the receipt with `transactionByCommand`.
 - **Who changed this entity, and why?** Page `transactionsForEntity(entityId)`, newest first.
   Each record carries its actor, command, correlation, causation, release, and typed changes.
 - **What was retracted?** `history(entityId)` includes retracted assertions. `entity` and `match`

docs/concepts.md

@@ -66,7 +66,10 @@ omitted, it uses the runtime's current time. Setting only one axis does not free
 
 Recorded time is assigned by the Triplex transaction boundary, not supplied as application data.
 Use the journal's ordered commit positions when exact ordering matters; timestamps alone may not
-distinguish commits in the same millisecond.
+distinguish commits in the same millisecond. A basis can cut recorded time by position instead:
+`{ recordedPosition }` reads exactly the transactions committed at or before that position (a
+`TransactionResult.position`). With both `recordedAt` and `recordedPosition`, a fact must be
+visible under both cuts.
 
 ## Datalog asks across relationships
 

docs/current-state.md

@@ -108,6 +108,9 @@ outside the normal test matrix.
   are rejected where a complete explanation cannot be preserved.
 - Config decision verification proves internal content integrity given a trusted decision root. It
   does not independently prove that an external actor chose the right rule or answer.
+- Import segments (`Segment.attach`, `Import.run` with `store: "segments"`) are an experimental
+  prototype for the in-memory KV backend only; SQL backends reject them. See
+  [Import segments](/segments).
 - Constraint enforcement is opt-in and serialized through the commit-position boundary. Direct
   adapter writes, authorization, general Datalog invariants, inbox/outbox delivery, timers, retries,
   and application workflow lifecycle are host responsibilities.

docs/operational-primitives.md

@@ -49,7 +49,18 @@ Every successful `transact` persists an `_Transaction` entity with:
 - one `:_tx/change` JSON fact for every asserted or retracted application fact.
 
 `Triples.transaction(txId)` reconstructs the typed envelope. The same facts remain available to
-Datalog for application-specific audit queries. A command ID is an atomically unique idempotency
+Datalog for application-specific audit queries.
+
+A transaction committed with `journal: "summary"` records one `:_tx/summary` instead of one
+`:_tx/change` per fact: assertion and retraction counts, the sorted changed attributes, and a
+`ContentId` digest of the changes (entity, attribute, value, valid interval and entity type per
+assertion; triple ID per retraction; generated IDs excluded). It keeps the envelope, the commit
+position, and every `:_tx/changed-entity` reference, so the feed, command receipts, and
+`transactionsForEntity` behave as before. Its record has an empty `changes` list and a `summary`.
+The facts keep their full bitemporal history, so what changed is still readable at the
+transaction's position; what is lost is replaying the journal alone to rebuild those facts.
+Derivation relevance uses the summary's attributes, and change emission and entity snapshots fall
+back to the submitted operations for such transactions. A command ID is an atomically unique idempotency
 identity within one Triplex database. Concurrent attempts acquire a backend-local claim inside the
 same transaction as the facts and envelope; exactly one commits. A loser receives
 `CommandAlreadyCommittedError` with the original transaction ID, and
@@ -88,6 +99,33 @@ another cursor update, and repeat forever. These unjournaled system writes still
 positions, so recorded snapshot ordering remains total; application commands remain fully
 journaled.
 
+### Published position refs
+
+`@triplex-build/triplex/operational` also provides `PositionRef`: a named, movable pointer to a
+committed position, stored as a reserved entity. `PositionRef.move` points it at a position in one
+journaled transaction, with compare-and-retract against concurrent moves and an optional
+`expectedPosition` (`null` for "not created yet"). Positions past the latest commit are rejected.
+`PositionRef.basis(triples, name, { validAt })` returns `{ recordedPosition, validAt }`; resolve it
+once and reuse it for every read that must agree, because resolving per read could straddle a
+move. Every move is history, so `PositionRef.get(triples, name, { recordedAt })` says what was
+published at an earlier instant. A move may carry a JSON manifest describing what it published.
+
+### Incremental imports
+
+`Import.run(triples, facts, { ref, chunkSize })` commits a large dataset as many transactions
+and yields to the host's event loop between them, so a browser keeps painting and a server keeps
+serving. Chunks default to the summary journal and share the import ID as their correlation ID.
+With `ref`, one final transaction moves the ref to the last chunk's position and stores the import
+manifest: fact and chunk counts, position range, and a digest over the chunk digests. Readers at
+the ref see the previous publication until that move and the whole import after it. Readers at the
+latest state see chunks as they commit. A failed import leaves its committed chunks in place,
+unpublished.
+
+With `store: "segments"`, each chunk is built into an immutable, content-addressed `Segment` and
+attached in one small transaction instead of being transacted fact by fact. Reads merge attached
+segments with indexed facts at any basis. This is an in-memory-only prototype; see
+[Import segments](/segments) for the format, journal shape, and audit trade-offs.
+
 ### Snapshot-stable Datalog pages
 
 `Triples.queryPage` returns an opaque, versioned keyset cursor. Its content-addressed fingerprints
@@ -100,9 +138,16 @@ transactions share the same recorded millisecond. Cursor decoding and reuse fail
 ### Bitemporal query basis
 
 Facts carry separate recorded and valid intervals. Direct matching, entity reads, batched reads,
-and Datalog accept the same `{ recordedAt?, validAt? }` basis, so every clause in a join,
-negation, or rule sees one coherent cut. Historical corrections append new facts; they do not
-rewrite the recorded history.
+and Datalog accept the same `{ recordedAt?, recordedPosition?, validAt? }` basis, so every clause
+in a join, negation, or rule sees one coherent cut. Historical corrections append new facts; they
+do not rewrite the recorded history.
+
+`recordedPosition` cuts recorded time at a commit position rather than an instant. Positions are
+the total commit order, so a read at position `p` sees every transaction committed at or before
+`p` and nothing after it, even when commits share a millisecond. With `recordedAt` as well, a fact
+must be visible under both. A position past the latest commit reads the latest state, so only a
+committed position is a stable cut. Paged reads clamp a requested position to the latest commit
+and keep it in their cursor.
 
 ### Portable derivations and pure reconciliation