diff --git a/.changeset/backdated-create.md b/.changeset/backdated-create.md new file mode 100644 index 0000000..73734ff --- /dev/null +++ b/.changeset/backdated-create.md @@ -0,0 +1,32 @@ +--- +'@haverstack/core': minor +--- + +Add `createdAt`/`updatedAt` options to `Stack.create()`, so an app — or a stack owner, through +their own server — can import an existing corpus with its real dates instead of every record +landing stamped with the import moment. + +- Unconditional on unscoped `Stack.create()`, like the existing client-minted `id` option. + `ScopedStack.create()` accepts the same two fields, but only from the stack owner acting + alone (undelegated, authenticated as themselves — the same tier that already gates hard + delete, `commitMigration()`, and `includeUnlisted`); a grantee, or a delegated app acting for + the owner, is refused with `StackPermissionError`. `POST /records` inherits this rule + automatically on a server built on `ScopedStack`: an owner-authenticated request may carry + both fields, anyone else's has them ignored, as before. +- Omit `id` and it is derived from `createdAt`'s timestamp, so the two agree by construction. + Supply both, and they are checked against each other using the same `idTimestampSkewMs` + tolerance the ordinary `id`-vs-current-time check already uses (default 24 hours; `null` + disables this check too) — disagreement beyond that tolerance throws `StackValidationError` + rather than silently diverging. An owner's plain `id`-only create through `ScopedStack` is + unaffected — it still gets the ordinary `id`-vs-current-time check, not this one. +- `updatedAt` defaults to `createdAt`, not to the actual current time, so a plain import + doesn't fabricate a fake edit and inflate version history. An `updatedAt` earlier than + `createdAt` is a validation error, including when `createdAt` defaulted to now. +- Both fields must be valid Dates within the range a record ID's timestamp prefix can + encode (1970-01-01 through 3084-12-12); anything else is a `StackValidationError`. An + `Invalid Date` in particular is refused rather than stored, since its `NaN` timestamp + would silently switch off the checks above instead of failing them. +- Dates are copied on the way in, so an import loop that advances and reuses a single + `Date` across rows doesn't retro-edit the records it already wrote. + +See docs/spec/data-model.md § Record IDs. diff --git a/docs/spec/data-model.md b/docs/spec/data-model.md index d26e2c7..c0517a1 100644 --- a/docs/spec/data-model.md +++ b/docs/spec/data-model.md @@ -66,6 +66,19 @@ The same rules are enforced locally, so a client-minted ID behaves identically w - `Stack.create(typeId, content, { id })` validates charset, length, and the reserved prefix, and throws `StackConflictError` on a duplicate. This is a full-trust context (an embedded single-app stack, or the server's own code) — no clock-skew check. - `ScopedStack.create()` — a grantee minting an ID — applies the same validation **plus** the timestamp-skew check, since a grantee is exactly the untrusted actor who could otherwise forge a sort position. The tolerance is configurable per Stack via `Stack.create(adapter, { idTimestampSkewMs })` (default 24 hours; pass `null` to disable). +### Backdating on import + +`Stack.create()` also accepts `createdAt`/`updatedAt` (`BackdatableCreateRecordOptions`) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment: + +- **Unconditional on unscoped `Stack.create()`** — the same full-trust context as the `id` option above. +- **Owner-only on `ScopedStack.create()`.** Refused to everyone but the stack owner acting alone (undelegated, authenticated as themselves — the same `ownerActingAlone` tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`): a grantee, or a delegated app acting for the owner, could otherwise forge a sort position through `createdAt` the same way the `id` skew check exists to stop it forging one through `id`. `ScopedStack.create()` refuses both fields outright for anyone else. +- **Owner-authenticated only, over the wire.** `POST /records` may carry `createdAt`/`updatedAt` when the request authenticates as the stack owner acting alone; a server built on `ScopedStack` inherits this automatically, since it enforces the same rule a local caller does. Anyone else's request has both fields ignored, as every other server-assigned field already is — see [Wire format § Records](./wire-format.md#records). +- **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `id`-vs-current-time check above uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed — including for the owner, whose plain `id`-only creates through `ScopedStack` still get the ordinary `id`-vs-current-time check, not this one. +- **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error — including when `createdAt` was left to default to now. +- **Both fields must be valid, representable Dates.** An `Invalid Date` (what `new Date()` yields for a malformed date string, a common shape for a bad row in an imported corpus) is a `StackValidationError`, not a record: its `getTime()` is `NaN`, and every comparison against `NaN` is false, so an unchecked one would switch off the ordering and skew checks above rather than fail them. The representable range is the range a record ID's 9-character timestamp prefix can encode — `1970-01-01T00:00:00.000Z` through `3084-12-12T12:41:28.831Z` — since a `createdAt` outside it has no ID that can agree with it. Content genuinely dated outside that window belongs in the record's own content fields, not in `createdAt`. +- **A backdated record's `updatedAt` predates its import**, by construction. A sync process that walks records by `updatedAt` to find what changed since a cursor will not see a backdated import as "recent" — which is the point (it isn't a recent edit), but worth knowing if a consumer expects an import to appear at the head of such a cursor. +- **Backdated records are invisible to an `updatedAt` cursor.** A consumer syncing incrementally by `filter.updatedAt.after` will not see records imported with historical dates, because they land behind the cursor. Import against a full corpus read, not a change cursor. + ## Associations Tags, attachments, and relationships are unified under a single **Association** model. All three associate a Record with a labeled payload — the label carries semantic meaning (e.g. `"avatar"`, `"parent"`, `"reply-to"`). diff --git a/docs/spec/wire-format.md b/docs/spec/wire-format.md index 5887121..35fa5ce 100644 --- a/docs/spec/wire-format.md +++ b/docs/spec/wire-format.md @@ -270,7 +270,7 @@ When present, the server applies the mutation only if the record's current versi `POST /records` accepts a full record body, including an optional client-supplied `id` — see [Record IDs](./data-model.md#record-ids) for the validation and duplicate-conflict rules the server applies. -**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, `createdAt`, and `updatedAt`, which the server assigns as it does on any write. `appId` is the deliberate exception — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path. +**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, which the server always assigns. `createdAt` and `updatedAt` are almost the same — server-assigned and ignored on every request but one: an **owner-authenticated** request (the stack owner acting alone, undelegated — the same tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`) may include them, to backdate an imported record's clock fields instead of stamping the import moment. `ScopedStack` already enforces this — refusing both fields to anyone else, and checking a supplied `id` against `createdAt` rather than against the current time when they're present — so a server built on it inherits the rule automatically, same as `entityId`/`principalId` above; one that maps a request body onto `Stack` directly has to reproduce the owner check itself. See [Data model § Backdating on import](./data-model.md#backdating-on-import). `appId` is the deliberate exception among the rest — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path. ### Migration commit diff --git a/packages/core/src/id.ts b/packages/core/src/id.ts index c3cb0be..647f8bc 100644 --- a/packages/core/src/id.ts +++ b/packages/core/src/id.ts @@ -147,6 +147,18 @@ export const _resetIdState = (): void => { // Public API // ------------------------------------------------------- +/** + * Largest timestamp (ms since epoch) that still encodes to the + * 9-character prefix an ID's format requires: 32^9 - 1, i.e. + * 3084-12-12T12:41:28.831Z. One millisecond past it the prefix grows to 10 + * characters and the ID no longer passes isValidIdFormat() — the library + * would be minting an ID it rejects on the way back in. generateId() can't + * reach this, since it encodes Date.now(); generateIdForTimestamp() takes + * whatever timestamp the caller asks for, so it is the one that has to + * check. + */ +export const MAX_ID_TIMESTAMP = Math.pow(BASE, MIN_TIMESTAMP_LENGTH) - 1; + /** * Generate a new Stack record ID. Time-sortable: lexicographic order * matches creation order, with same-millisecond IDs monotonically @@ -167,6 +179,31 @@ export const generateId = (timestamp: number = Date.now()): string => { return nowId + randChars; }; +/** + * Mint an ID for an arbitrary (typically past) timestamp — used by + * Stack.create() to derive an ID from an explicit `createdAt` when + * importing historical records. Deliberately bypasses generateId()'s + * monotonic `lastTimestamp` floor: that floor exists to protect *live* ID + * generation from a backward clock step (NTP correction, suspend/resume), + * and would otherwise clamp a deliberately historical timestamp forward to + * "now" the moment the process has minted any live ID past it — silently + * defeating the backdate it was asked for. Same-millisecond uniqueness for + * a historical timestamp is therefore left to a fresh random suffix each + * call rather than the live incrementing scheme; a collision surfaces the + * same way any client-supplied id collision does, as StackConflictError + * from the adapter. + */ +export const generateIdForTimestamp = (timestamp: number): string => { + if (!Number.isFinite(timestamp) || timestamp < 0 || timestamp > MAX_ID_TIMESTAMP) { + throw new IdGenerationError( + `Timestamp ${timestamp} cannot be encoded as an ID: expected 0…${MAX_ID_TIMESTAMP} ` + + `(1970-01-01T00:00:00.000Z…${new Date(MAX_ID_TIMESTAMP).toISOString()}).`, + ); + } + const nowId = pad(crockford32Encode(timestamp), MIN_TIMESTAMP_LENGTH); + return nowId + generateRandChars(); +}; + // ------------------------------------------------------- // Format validation // ------------------------------------------------------- diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index d1b3c63..dc1a07a 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -37,6 +37,7 @@ export type { StackErrorCode, StackClient, CreateRecordOptions, + BackdatableCreateRecordOptions, StackOptions, GetRecordOptions, DeleteRecordOptions, diff --git a/packages/core/src/stack.ts b/packages/core/src/stack.ts index 793af31..1462a5c 100644 --- a/packages/core/src/stack.ts +++ b/packages/core/src/stack.ts @@ -14,7 +14,13 @@ * Apps should never talk to a StackAdapter directly. */ -import { generateId, isValidIdFormat, idTimestamp } from './id.js'; +import { + generateId, + generateIdForTimestamp, + isValidIdFormat, + idTimestamp, + MAX_ID_TIMESTAMP, +} from './id.js'; import { hashSchema, isCompatible, @@ -308,6 +314,37 @@ export type CreateRecordOptions = { unlisted?: boolean; }; +/** + * CreateRecordOptions extended with createdAt/updatedAt, for backdating a + * record's clock fields on import (e.g. migrating an existing archive with + * its original dates). Accepted unconditionally by unscoped Stack.create(). + * ScopedStack.create() accepts the same two fields, but only from the + * stack owner acting alone (undelegated, authenticated as themselves) — a + * grantee, or a delegated app acting for the owner, is refused, since + * either could otherwise forge a sort position the same way a raw `id` + * could. A server built on ScopedStack inherits this automatically: an + * owner-authenticated `POST /records` may carry both fields; anyone else's + * request has them ignored, as before. See docs/spec/data-model.md § + * Record IDs and docs/spec/wire-format.md § Records. + */ +export type BackdatableCreateRecordOptions = CreateRecordOptions & { + /** + * The record's creation time. When `id` is also supplied, its embedded + * timestamp must agree with this within `idTimestampSkewMs` (default 24h; + * see StackOptions.idTimestampSkewMs) or the create throws + * StackValidationError. Omit `id` to have it derived from this timestamp + * instead. Defaults to now. + */ + createdAt?: Date; + /** + * The record's last-modified time. Defaults to `createdAt` (or now, if + * `createdAt` is omitted too) — never to the actual current time — so a + * plain import doesn't fabricate a fake edit. Must not precede + * `createdAt`. + */ + updatedAt?: Date; +}; + export type ScopedStackOptions = { /** * The entity a delegated app acts for. Omit when the principal acts as @@ -325,10 +362,13 @@ export type StackOptions = { */ ownerProfile?: { name: string; handle?: string }; /** - * Clock-skew tolerance (ms) for the timestamp-prefix check - * ScopedStack.create() runs on grantee-supplied IDs; unscoped - * Stack.create() never runs it. Default: 24 hours; null disables. - * See docs/spec/data-model.md § Record IDs. + * Clock-skew tolerance (ms) for two timestamp-prefix checks: the one + * ScopedStack.create() runs on a non-backdated create's client-supplied + * `id` against the current time, and the one Stack.create() runs between + * an explicit `id` and an explicit `createdAt` when both are supplied — + * reached directly when unscoped, or via ScopedStack.create() when the + * requester is the owner acting alone. Default: 24 hours; null disables + * both. See docs/spec/data-model.md § Record IDs. */ idTimestampSkewMs?: number | null; }; @@ -799,18 +839,72 @@ function validateRecordId(id: string): void { } /** - * Timestamp-prefix plausibility check for grantee-minted IDs - * (ScopedStack.create()) — a grantee is untrusted and could otherwise mint - * an ID that forges its sort position. Pass null to disable. + * Validity and range check for the backdating options on unscoped + * Stack.create(). A Date is only as good as what the caller parsed it + * from, and the two failure modes both need catching here rather than + * downstream: + * + * - **Invalid Date** (`new Date('13/45/2020')` off a malformed import row) + * has a NaN getTime(), and every comparison against NaN is false — so an + * unchecked Invalid Date passes the updatedAt/createdAt ordering check + * and the id/createdAt skew check by turning them off, mints the + * epoch-zero ID `000000000xxx`, and persists a record whose + * `createdAt.toISOString()` throws RangeError in serializeRecord() — + * making that record, and any wire response containing it, + * permanently unreadable. + * - **Out of encodable range** — before 1970 crockford32Encode() throws a + * bare RangeError from deep inside the ID encoder, and past + * MAX_ID_TIMESTAMP the derived ID silently grows to 13 characters and + * fails isValidIdFormat(). + * + * Checked at the door for both, as a StackValidationError naming the + * field. Applies whether or not an `id` is supplied: a clock field outside + * this range is unrepresentable regardless of where the ID came from. + */ +function validateClockField(value: Date | undefined, path: string): ValidationError[] { + if (value === undefined) return []; + const ms = value.getTime(); + if (Number.isNaN(ms)) { + return [{ path, message: `${path} is not a valid Date.` }]; + } + if (ms < 0 || ms > MAX_ID_TIMESTAMP) { + return [ + { + path, + message: + `${path} is outside the representable range ` + + `(1970-01-01T00:00:00.000Z…${new Date(MAX_ID_TIMESTAMP).toISOString()}).`, + }, + ]; + } + return []; +} + +/** + * Timestamp-prefix plausibility check, shared by callers that compare an + * ID's embedded millisecond against a different reference each: + * ScopedStack.create() against the current time for any non-backdated + * create (a grantee is untrusted and could otherwise mint an ID that + * forges its sort position, and this is the one check standing between a + * delegated or grantee caller and doing so), and Stack.create() — reached + * directly when unscoped, or via ScopedStack.create() when the requester + * is the owner acting alone with an explicit `createdAt` — against that + * `createdAt` instead (the two must agree, not silently diverge — see + * docs/spec/data-model.md § Record IDs). Pass null to disable. */ -function validateIdTimestampSkew(id: string, toleranceMs: number | null): void { +function validateIdTimestampSkew( + id: string, + toleranceMs: number | null, + referenceMs: number, + referenceLabel: string, +): void { if (toleranceMs === null) return; - const skew = Math.abs(Date.now() - idTimestamp(id)); + const skew = Math.abs(referenceMs - idTimestamp(id)); if (skew > toleranceMs) { throw new StackValidationError([ { path: 'id', - message: `ID "${id}" timestamp is outside the allowed clock-skew tolerance (${toleranceMs}ms).`, + message: `ID "${id}" timestamp disagrees with ${referenceLabel} by more than the allowed clock-skew tolerance (${toleranceMs}ms).`, }, ]); } @@ -1317,12 +1411,17 @@ export class Stack implements StackClient { * Create a new record. Validates content against the type's schema. * `_group` records get their author stamped as the first `admin` roster * association here — the single stamping site for both Stack.create() - * and ScopedStack.create(). See docs/spec/identity.md § Group. + * and ScopedStack.create(). `opts.createdAt`/`updatedAt` let a caller + * backdate an imported record's clock fields — unconditionally here; + * ScopedStack.create() forwards to this same method, but only reaches + * this far with them when the requester is the stack owner acting alone + * — see BackdatableCreateRecordOptions and docs/spec/data-model.md § + * Record IDs. */ async create = Record>( typeId: TypeId, content: T, - opts: CreateRecordOptions = {}, + opts: BackdatableCreateRecordOptions = {}, ): Promise { this.assertOpen(); const type = await this.getTypeCached(typeId); @@ -1330,12 +1429,33 @@ export class Stack implements StackClient { throw new Error(`Unknown type: "${typeId}". Call defineType() first.`); } + // updatedAt defaults to createdAt, not to the actual current time, so a + // plain import doesn't fabricate a fake edit. + // + // Copied, never aliased: the caller keeps its own reference to any Date + // it passed, and an import loop that reuses one Date across rows + // (`d.setTime(...)` per record — the obvious way to write one) would + // otherwise retro-edit every record it had already written, with no + // version bump and no change event. createdAt drives updatedAt and + // unlistedAt below, so one copy taken here covers all three. + const createdAt = + opts.createdAt !== undefined ? new Date(opts.createdAt.getTime()) : new Date(); + const updatedAt = opts.updatedAt !== undefined ? new Date(opts.updatedAt.getTime()) : createdAt; + const errors = [ ...validateReservedKeys(content), ...validateContent(content, type.schema), ...validatePermissions(opts.permissions), ...validateAssociations(opts.associations), + ...validateClockField(opts.createdAt, 'createdAt'), + ...validateClockField(opts.updatedAt, 'updatedAt'), ]; + // Compared against the *effective* createdAt, so an updatedAt supplied + // on its own is caught too: defaulted-createdAt is now, which a + // backdated updatedAt alone would still precede. + if (opts.updatedAt !== undefined && updatedAt.getTime() < createdAt.getTime()) { + errors.push({ path: 'updatedAt', message: 'updatedAt cannot precede createdAt.' }); + } if (errors.length > 0) { throw new StackValidationError(errors); } @@ -1348,19 +1468,44 @@ export class Stack implements StackClient { await this.checkBindingsOnCreate(typeId, content as Record); - if (opts.id !== undefined) validateRecordId(opts.id); + if (opts.id !== undefined) { + validateRecordId(opts.id); + // Only when both are explicit: an `id` alone (no createdAt) stays a + // pure position choice, exactly as before this option existed. + if (opts.createdAt !== undefined) { + validateIdTimestampSkew( + opts.id, + this.idTimestampSkewMsValue, + opts.createdAt.getTime(), + 'createdAt', + ); + } + } const associations = baseIdOf(typeId) === SYSTEM_TYPES.GROUP ? stampGroupAdmin(opts.associations, opts.entityId ?? this.ownerEntityId) : opts.associations; - const now = new Date(); + // createdAt (hoisted above, alongside its validation) drives the ID + // when the caller doesn't supply one, so the two agree by construction + // rather than by coincidence — the same relationship an explicit `id` + // is checked against above. + // An explicit createdAt mints via generateIdForTimestamp(), which never + // clamps to "now" — generateId()'s monotonic floor would otherwise + // silently pull a deliberately historical id forward once this process + // has minted any live id past it. The no-createdAt path keeps + // generateId(), unaffected and still monotonic-safe. + const id = + opts.id ?? + (opts.createdAt !== undefined + ? generateIdForTimestamp(createdAt.getTime()) + : generateId(createdAt.getTime())); const record: StackRecord = { - id: opts.id ?? generateId(), + id, typeId, - createdAt: now, - updatedAt: now, + createdAt, + updatedAt, content, version: 1, ...(opts.parentId && { parentId: opts.parentId }), @@ -1374,7 +1519,7 @@ export class Stack implements StackClient { ...(opts.principalId && { updatedVia: opts.principalId }), ...(opts.permissions?.length && { permissions: opts.permissions }), ...(associations?.length && { associations }), - ...(opts.unlisted && { unlistedAt: now }), + ...(opts.unlisted && { unlistedAt: createdAt }), }; const created = await this.adapter.createRecord(record); @@ -3456,15 +3601,34 @@ export class ScopedStack implements StackClient { * reference-creating options gated, and non-owner `_attachment@1` * creation refused save one carve-out. A scoped create always stamps * authorship — an absent entityId means an unscoped `Stack` wrote it. - * See docs/spec/access-control.md and docs/spec/attachments.md. + * `createdAt`/`updatedAt` are refused to everyone but the owner acting + * alone — see the guard below and docs/spec/data-model.md § Record IDs. + * See also docs/spec/access-control.md and docs/spec/attachments.md. */ async create = Record>( typeId: TypeId, content: T, - opts: CreateRecordOptions = {}, + opts: BackdatableCreateRecordOptions = {}, ): Promise { const principal = this.principalEntityId; if (!principal) throw new StackPermissionError('Anonymous requesters cannot create records'); + // createdAt/updatedAt let a caller backdate a record's clock fields — + // and, without `id` also supplied, its sort position too. Refused to + // everyone but the owner acting alone (undelegated, authenticated as + // themselves): a grantee is exactly the untrusted actor the `id` + // skew check below already exists to stop from forging a sort + // position, and a delegated app acting for the owner inherits none of + // the owner's extra trust — same reasoning as mayGrantAccess() below. + // Refused rather than silently dropped, so an app never believes it + // published something it didn't. This is also the enforcement a + // server built on ScopedStack inherits for `POST /records`: an + // owner-authenticated request may carry both fields, anyone else's + // has them ignored. + if (('createdAt' in opts || 'updatedAt' in opts) && !this.ownerActingAlone) { + throw new StackPermissionError( + 'createdAt/updatedAt can only be set by the stack owner acting alone; a grantee or delegated create always stamps the current time.', + ); + } if (!(await this.checkCreateGrant(typeId))) { throw new StackPermissionError(`No create grant for type "${typeId}"`); } @@ -3492,7 +3656,15 @@ export class ScopedStack implements StackClient { await this.requireAppIdMatchesPrincipal(opts.appId); if (opts.id !== undefined) { validateRecordId(opts.id); - validateIdTimestampSkew(opts.id, this.idTimestampSkewMs); + // Skipped when createdAt is also supplied: only the owner reaches + // here with that combination (checked above), and Stack.create() + // below checks the id against createdAt instead of "now" — the + // check here exists for a live grantee write, and a backdated + // owner create is deliberately not one. See + // docs/spec/data-model.md § Record IDs. + if (opts.createdAt === undefined) { + validateIdTimestampSkew(opts.id, this.idTimestampSkewMs, Date.now(), 'the current time'); + } } if (opts.parentId !== undefined && !(await this.canReadReferent(opts.parentId))) { throw new StackPermissionError(); diff --git a/packages/core/tests/id.test.ts b/packages/core/tests/id.test.ts index f957b14..237d04e 100644 --- a/packages/core/tests/id.test.ts +++ b/packages/core/tests/id.test.ts @@ -1,15 +1,19 @@ import { describe, test, expect, beforeEach, vi } from 'vitest'; import { generateId, + generateIdForTimestamp, crockford32Encode, crockford32Decode, isValidIdFormat, idTimestamp, RAND_SUFFIX_LENGTH, BASE, + IdGenerationError, IdGenerationOverflowError, + MAX_ID_TIMESTAMP, _setLastNowId, _setLastRandChars, + _setLastTimestamp, _resetIdState, } from '../src/id.js'; @@ -226,6 +230,83 @@ describe('generateId', () => { }); }); +// ------------------------------------------------------- +// generateIdForTimestamp +// ------------------------------------------------------- + +describe('generateIdForTimestamp', () => { + beforeEach(() => { + _resetIdState(); + }); + + test('timestamp prefix matches the encoded timestamp', () => { + const t = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(t); + expect(idTimestamp(id)).toBe(t); + }); + + test('produces a well-formed id', () => { + const id = generateIdForTimestamp(new Date('2020-06-15T12:00:00.000Z').valueOf()); + expect(isValidIdFormat(id)).toBe(true); + }); + + test('is not clamped by a later live generateId() call in the same process (regression)', () => { + // A live create() advances the monotonic floor to "now" — a much + // larger timestamp than any historical import date. + generateId(Date.now()); + + const historical = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(historical); + + // generateId() itself would have clamped this forward to "now"; + // generateIdForTimestamp() must not. + expect(idTimestamp(id)).toBe(historical); + }); + + test('does not advance generateId()’s monotonic floor for later live ids', () => { + // Minting a historical id must not make generateId() think the clock + // has already reached that timestamp when live generation resumes. + generateIdForTimestamp(new Date('2099-01-01').valueOf()); + + const now = Date.now(); + const liveId = generateId(now); + expect(idTimestamp(liveId)).toBe(now); + }); + + test('does not consult the live monotonic floor for a backward-looking timestamp', () => { + _setLastTimestamp(new Date('2024-06-01').valueOf()); + const historical = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(historical); + expect(idTimestamp(id)).toBe(historical); + }); + + // generateId() encodes Date.now() and so can never leave the encodable + // range; generateIdForTimestamp() encodes whatever it is handed, so the + // bound has to live here. Without it a timestamp past 32^9-1 overflows + // the 9-char prefix and yields a 13-char id the library itself rejects. + test('accepts the last encodable millisecond and keeps the id well-formed', () => { + const id = generateIdForTimestamp(MAX_ID_TIMESTAMP); + expect(isValidIdFormat(id)).toBe(true); + expect(idTimestamp(id)).toBe(MAX_ID_TIMESTAMP); + }); + + test('throws rather than minting a malformed id one ms past the range', () => { + expect(() => generateIdForTimestamp(MAX_ID_TIMESTAMP + 1)).toThrow(IdGenerationError); + }); + + test('throws on a negative (pre-epoch) timestamp', () => { + expect(() => generateIdForTimestamp(new Date('1969-07-20').valueOf())).toThrow( + IdGenerationError, + ); + }); + + test('throws on NaN rather than silently minting the epoch-zero id', () => { + expect(() => generateIdForTimestamp(new Date('not a date').valueOf())).toThrow( + IdGenerationError, + ); + }); +}); + // ------------------------------------------------------- // ID format validation // ------------------------------------------------------- diff --git a/packages/core/tests/scoped-stack.test.ts b/packages/core/tests/scoped-stack.test.ts index ac94036..edafa64 100644 --- a/packages/core/tests/scoped-stack.test.ts +++ b/packages/core/tests/scoped-stack.test.ts @@ -923,6 +923,102 @@ describe('ScopedStack.create — client-supplied id', () => { }); }); +// ------------------------------------------------------- +// ScopedStack.create — createdAt/updatedAt: owner acting alone only +// ------------------------------------------------------- + +describe('ScopedStack.create — createdAt/updatedAt refused to anyone but the owner acting alone', () => { + beforeEach(async () => { + await stack.defineType(COMMENT, 'Comment', { text: { kind: 'text', required: true } }); + await stack.grant(MEMBER, [{ actions: ['create'], typeId: COMMENT }]); + }); + + test('rejects a grantee-supplied createdAt with StackPermissionError', async () => { + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('rejects a grantee-supplied updatedAt with StackPermissionError', async () => { + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { updatedAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('does not create a record when refused', async () => { + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(); + expect((await stack.query({ filter: { typeId: COMMENT } })).records).toHaveLength(0); + }); + + test('rejects createdAt from a principal delegated to act for the owner (not owner acting alone)', async () => { + // subjectEntityId === OWNER satisfies checkCreateGrant()'s owner-subject + // carve-out, so this exercises the createdAt/updatedAt gate itself + // rather than getting stopped earlier by a missing create grant. + await expect( + stack + .asEntity(MEMBER, { onBehalfOf: OWNER }) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('rejects createdAt when the owner delegates for someone else — the owner’s trust does not transfer to the subject', async () => { + await expect( + stack + .asEntity(OWNER, { onBehalfOf: MEMBER }) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('owner acting alone may set createdAt, and updatedAt defaults to match', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.asEntity(OWNER).create(COMMENT, { text: 'hello' }, { createdAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(createdAt); + }); + + test('owner acting alone may set updatedAt distinct from createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-20T12:00:00.000Z'); + const record = await stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { createdAt, updatedAt }); + expect(record.updatedAt).toEqual(updatedAt); + }); + + test('owner acting alone: an id agreeing with createdAt is accepted, ignoring the "vs. now" skew check', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(createdAt.valueOf()); + const record = await stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('owner acting alone: an id disagreeing with createdAt is rejected against createdAt, not "now"', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect( + stack.asEntity(OWNER).create(COMMENT, { text: 'hello' }, { id, createdAt }), + ).rejects.toThrow(StackValidationError); + }); + + test('owner acting alone still gets the reserved-prefix and format checks on a backdated id', async () => { + await expect( + stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { id: 'too-short', createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackQueryError); + }); +}); + // ------------------------------------------------------- // ScopedStack — grant-based read // ------------------------------------------------------- diff --git a/packages/core/tests/stack.test.ts b/packages/core/tests/stack.test.ts index 15a8058..3d7e909 100644 --- a/packages/core/tests/stack.test.ts +++ b/packages/core/tests/stack.test.ts @@ -13,7 +13,14 @@ import { StackPayloadTooLargeError, StackClosedError, } from '../src/stack.js'; -import { generateId, crockford32Encode, IdGenerationError } from '../src/id.js'; +import { + generateId, + crockford32Encode, + idTimestamp, + isValidIdFormat, + MAX_ID_TIMESTAMP, + IdGenerationError, +} from '../src/id.js'; import { InvalidDidError } from '../src/did.js'; import { MemoryAdapter, IncapableMemoryAdapter } from '../src/testing.js'; import { firstRecordedAttachment } from '../src/attachment-download.js'; @@ -520,6 +527,173 @@ describe('create — client-supplied id', () => { }); }); +// ------------------------------------------------------- +// create — backdating (createdAt / updatedAt) +// ------------------------------------------------------- + +describe('create — backdating (createdAt/updatedAt)', () => { + test('accepts an explicit createdAt and defaults updatedAt to match', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(createdAt); + }); + + test('accepts a distinct updatedAt when supplied', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-20T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt, updatedAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(updatedAt); + }); + + test('rejects an updatedAt preceding createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-10T12:00:00.000Z'); + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt, updatedAt }), + ).rejects.toThrow(StackValidationError); + }); + + test('derives the id from createdAt when no id is supplied', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(idTimestamp(record.id)).toBe(createdAt.valueOf()); + }); + + test('accepts an explicit id whose timestamp agrees with createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(createdAt.valueOf()); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('rejects an explicit id whose timestamp disagrees with createdAt beyond tolerance', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect(stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt })).rejects.toThrow( + StackValidationError, + ); + }); + + test('idTimestampSkewMs: null disables the id/createdAt consistency check too', async () => { + const permissiveAdapter = new MemoryAdapter({ ownerEntityId: 'owner-123', timezone: 'UTC' }); + const permissiveStack = await Stack.create(permissiveAdapter, { idTimestampSkewMs: null }); + await permissiveStack.defineType(NOTE_V1, 'Note', { text: { kind: 'text', required: true } }); + + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + const record = await permissiveStack.create(NOTE_V1, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('an id alone (no createdAt) stays a pure position choice — no consistency check applies', async () => { + const ancientId = idWithTimestamp(new Date('2000-01-01').valueOf()); + const before = new Date(); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { id: ancientId }); + expect(record.id).toBe(ancientId); + expect(record.createdAt.valueOf()).toBeGreaterThanOrEqual(before.valueOf()); + }); + + test('a prior live create() does not clamp a later backdated id forward (regression)', async () => { + // Advance generateId()'s monotonic floor to "now" via an ordinary, + // undated create() — the scenario a long-running import script hits + // once it has written anything live before importing historical data. + await stack.create(NOTE_V1, { text: 'live' }); + + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'backdated' }, { createdAt }); + expect(idTimestamp(record.id)).toBe(createdAt.valueOf()); + }); + + test('unlistedAt stamps from createdAt, not the actual current time', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt, unlisted: true }); + expect(record.unlistedAt).toEqual(createdAt); + }); + + // An Invalid Date's getTime() is NaN, and every comparison against NaN + // is false — so an unchecked Invalid Date doesn't slip past the + // ordering and skew checks, it switches them off, then persists a + // record that throws RangeError the moment anything serializes it. + test('rejects an Invalid Date createdAt rather than minting an epoch-zero id', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects an Invalid Date updatedAt', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { updatedAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('an Invalid Date createdAt cannot switch off the id/createdAt skew check', async () => { + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects a pre-epoch createdAt as a validation error, not a raw RangeError', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('1969-07-20T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects a pre-epoch createdAt even when an explicit id skips ID derivation', async () => { + const id = idWithTimestamp(0); + await expect( + stack.create( + NOTE_V1, + { text: 'hello' }, + { id, createdAt: new Date('1969-07-20T00:00:00.000Z') }, + ), + ).rejects.toThrow(StackValidationError); + }); + + // Past 32^9-1 ms the 9-char timestamp prefix overflows to 10, so the + // derived id would be 13 chars — one the library itself rejects via + // isValidIdFormat(). Year-9999 sentinels are ordinary in imported data. + test('rejects a far-future createdAt rather than minting a malformed 13-char id', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('9999-01-01T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('accepts a createdAt at the last encodable millisecond', async () => { + const createdAt = new Date(MAX_ID_TIMESTAMP); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(isValidIdFormat(record.id)).toBe(true); + expect(idTimestamp(record.id)).toBe(MAX_ID_TIMESTAMP); + }); + + // The one-sided case: createdAt defaults to now, which a backdated + // updatedAt on its own still precedes. + test('rejects an updatedAt preceding a defaulted createdAt', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { updatedAt: new Date('2000-01-01T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('does not alias the caller Date — mutating it after create leaves the record alone', async () => { + const cursor = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create( + NOTE_V1, + { text: 'hello' }, + { createdAt: cursor, unlisted: true }, + ); + + // The shape of a real import loop: one Date advanced per row. + cursor.setFullYear(1990); + + const stored = await stack.get(record.id); + expect(stored?.createdAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + expect(stored?.updatedAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + expect(stored?.unlistedAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + }); +}); + // ------------------------------------------------------- // Type cache — create()/update()/etc. shouldn't pay a getType() // round trip on every write for a value that can't change.