Allow backdating createdAt/updatedAt on unscoped Stack.create()

.changeset/backdated-create.md

@@ -0,0 +1,32 @@
+---
+'@haverstack/core': minor
+---
+
+Add `createdAt`/`updatedAt` options to `Stack.create()`, so an app — or a stack owner, through
+their own server — can import an existing corpus with its real dates instead of every record
+landing stamped with the import moment.
+
+- Unconditional on unscoped `Stack.create()`, like the existing client-minted `id` option.
+  `ScopedStack.create()` accepts the same two fields, but only from the stack owner acting
+  alone (undelegated, authenticated as themselves — the same tier that already gates hard
+  delete, `commitMigration()`, and `includeUnlisted`); a grantee, or a delegated app acting for
+  the owner, is refused with `StackPermissionError`. `POST /records` inherits this rule
+  automatically on a server built on `ScopedStack`: an owner-authenticated request may carry
+  both fields, anyone else's has them ignored, as before.
+- Omit `id` and it is derived from `createdAt`'s timestamp, so the two agree by construction.
+  Supply both, and they are checked against each other using the same `idTimestampSkewMs`
+  tolerance the ordinary `id`-vs-current-time check already uses (default 24 hours; `null`
+  disables this check too) — disagreement beyond that tolerance throws `StackValidationError`
+  rather than silently diverging. An owner's plain `id`-only create through `ScopedStack` is
+  unaffected — it still gets the ordinary `id`-vs-current-time check, not this one.
+- `updatedAt` defaults to `createdAt`, not to the actual current time, so a plain import
+  doesn't fabricate a fake edit and inflate version history. An `updatedAt` earlier than
+  `createdAt` is a validation error, including when `createdAt` defaulted to now.
+- Both fields must be valid Dates within the range a record ID's timestamp prefix can
+  encode (1970-01-01 through 3084-12-12); anything else is a `StackValidationError`. An
+  `Invalid Date` in particular is refused rather than stored, since its `NaN` timestamp
+  would silently switch off the checks above instead of failing them.
+- Dates are copied on the way in, so an import loop that advances and reuses a single
+  `Date` across rows doesn't retro-edit the records it already wrote.
+
+See docs/spec/data-model.md § Record IDs.

docs/spec/data-model.md

@@ -66,6 +66,19 @@ The same rules are enforced locally, so a client-minted ID behaves identically w
 - `Stack.create(typeId, content, { id })` validates charset, length, and the reserved prefix, and throws `StackConflictError` on a duplicate. This is a full-trust context (an embedded single-app stack, or the server's own code) — no clock-skew check.
 - `ScopedStack.create()` — a grantee minting an ID — applies the same validation **plus** the timestamp-skew check, since a grantee is exactly the untrusted actor who could otherwise forge a sort position. The tolerance is configurable per Stack via `Stack.create(adapter, { idTimestampSkewMs })` (default 24 hours; pass `null` to disable).
 
+### Backdating on import
+
+`Stack.create()` also accepts `createdAt`/`updatedAt` (`BackdatableCreateRecordOptions`) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment:
+
+- **Unconditional on unscoped `Stack.create()`** — the same full-trust context as the `id` option above.
+- **Owner-only on `ScopedStack.create()`.** Refused to everyone but the stack owner acting alone (undelegated, authenticated as themselves — the same `ownerActingAlone` tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`): a grantee, or a delegated app acting for the owner, could otherwise forge a sort position through `createdAt` the same way the `id` skew check exists to stop it forging one through `id`. `ScopedStack.create()` refuses both fields outright for anyone else.
+- **Owner-authenticated only, over the wire.** `POST /records` may carry `createdAt`/`updatedAt` when the request authenticates as the stack owner acting alone; a server built on `ScopedStack` inherits this automatically, since it enforces the same rule a local caller does. Anyone else's request has both fields ignored, as every other server-assigned field already is — see [Wire format § Records](./wire-format.md#records).
+- **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `id`-vs-current-time check above uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed — including for the owner, whose plain `id`-only creates through `ScopedStack` still get the ordinary `id`-vs-current-time check, not this one.
+- **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error — including when `createdAt` was left to default to now.
+- **Both fields must be valid, representable Dates.** An `Invalid Date` (what `new Date()` yields for a malformed date string, a common shape for a bad row in an imported corpus) is a `StackValidationError`, not a record: its `getTime()` is `NaN`, and every comparison against `NaN` is false, so an unchecked one would switch off the ordering and skew checks above rather than fail them. The representable range is the range a record ID's 9-character timestamp prefix can encode — `1970-01-01T00:00:00.000Z` through `3084-12-12T12:41:28.831Z` — since a `createdAt` outside it has no ID that can agree with it. Content genuinely dated outside that window belongs in the record's own content fields, not in `createdAt`.
+- **A backdated record's `updatedAt` predates its import**, by construction. A sync process that walks records by `updatedAt` to find what changed since a cursor will not see a backdated import as "recent" — which is the point (it isn't a recent edit), but worth knowing if a consumer expects an import to appear at the head of such a cursor.
+- **Backdated records are invisible to an `updatedAt` cursor.** A consumer syncing incrementally by `filter.updatedAt.after` will not see records imported with historical dates, because they land behind the cursor. Import against a full corpus read, not a change cursor.
+
 ## Associations
 
 Tags, attachments, and relationships are unified under a single **Association** model. All three associate a Record with a labeled payload — the label carries semantic meaning (e.g. `"avatar"`, `"parent"`, `"reply-to"`).

docs/spec/wire-format.md

@@ -270,7 +270,7 @@ When present, the server applies the mutation only if the record's current versi
 
 `POST /records` accepts a full record body, including an optional client-supplied `id` — see [Record IDs](./data-model.md#record-ids) for the validation and duplicate-conflict rules the server applies.
 
-**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, `createdAt`, and `updatedAt`, which the server assigns as it does on any write. `appId` is the deliberate exception — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path.
+**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, which the server always assigns. `createdAt` and `updatedAt` are almost the same — server-assigned and ignored on every request but one: an **owner-authenticated** request (the stack owner acting alone, undelegated — the same tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`) may include them, to backdate an imported record's clock fields instead of stamping the import moment. `ScopedStack` already enforces this — refusing both fields to anyone else, and checking a supplied `id` against `createdAt` rather than against the current time when they're present — so a server built on it inherits the rule automatically, same as `entityId`/`principalId` above; one that maps a request body onto `Stack` directly has to reproduce the owner check itself. See [Data model § Backdating on import](./data-model.md#backdating-on-import). `appId` is the deliberate exception among the rest — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path.
 
 ### Migration commit
 

packages/core/src/id.ts

@@ -147,6 +147,18 @@ export const _resetIdState = (): void => {
 // Public API
 // -------------------------------------------------------
 
+/**
+ * Largest timestamp (ms since epoch) that still encodes to the
+ * 9-character prefix an ID's format requires: 32^9 - 1, i.e.
+ * 3084-12-12T12:41:28.831Z. One millisecond past it the prefix grows to 10
+ * characters and the ID no longer passes isValidIdFormat() — the library
+ * would be minting an ID it rejects on the way back in. generateId() can't
+ * reach this, since it encodes Date.now(); generateIdForTimestamp() takes
+ * whatever timestamp the caller asks for, so it is the one that has to
+ * check.
+ */
+export const MAX_ID_TIMESTAMP = Math.pow(BASE, MIN_TIMESTAMP_LENGTH) - 1;
+
 /**
  * Generate a new Stack record ID. Time-sortable: lexicographic order
  * matches creation order, with same-millisecond IDs monotonically
@@ -167,6 +179,31 @@ export const generateId = (timestamp: number = Date.now()): string => {
   return nowId + randChars;
 };
 
+/**
+ * Mint an ID for an arbitrary (typically past) timestamp — used by
+ * Stack.create() to derive an ID from an explicit `createdAt` when
+ * importing historical records. Deliberately bypasses generateId()'s
+ * monotonic `lastTimestamp` floor: that floor exists to protect *live* ID
+ * generation from a backward clock step (NTP correction, suspend/resume),
+ * and would otherwise clamp a deliberately historical timestamp forward to
+ * "now" the moment the process has minted any live ID past it — silently
+ * defeating the backdate it was asked for. Same-millisecond uniqueness for
+ * a historical timestamp is therefore left to a fresh random suffix each
+ * call rather than the live incrementing scheme; a collision surfaces the
+ * same way any client-supplied id collision does, as StackConflictError
+ * from the adapter.
+ */
+export const generateIdForTimestamp = (timestamp: number): string => {
+  if (!Number.isFinite(timestamp) || timestamp < 0 || timestamp > MAX_ID_TIMESTAMP) {
+    throw new IdGenerationError(
+      `Timestamp ${timestamp} cannot be encoded as an ID: expected 0…${MAX_ID_TIMESTAMP} ` +
+        `(1970-01-01T00:00:00.000Z…${new Date(MAX_ID_TIMESTAMP).toISOString()}).`,
+    );
+  }
+  const nowId = pad(crockford32Encode(timestamp), MIN_TIMESTAMP_LENGTH);
+  return nowId + generateRandChars();
+};
+
 // -------------------------------------------------------
 // Format validation
 // -------------------------------------------------------

packages/core/src/index.ts

@@ -37,6 +37,7 @@ export type {
   StackErrorCode,
   StackClient,
   CreateRecordOptions,
+  BackdatableCreateRecordOptions,
   StackOptions,
   GetRecordOptions,
   DeleteRecordOptions,