From 609c320728ff47cae3997042685a9fc2f7a12150 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 00:20:06 +0000 Subject: [PATCH 1/3] feat: allow backdating createdAt/updatedAt on unscoped Stack.create() Closes #203. Stack.create() stamped createdAt/updatedAt from new Date() unconditionally, so importing an existing archive collapsed every record to the import moment. Since a full-trust caller already picks a record's creation *position* via a client-minted id, letting createdAt agree with it closes an inconsistency rather than opening a new hole. - UnscopedCreateRecordOptions adds createdAt/updatedAt to Stack.create() only. ScopedStack.create() never accepts them, even past the type system (a grantee could otherwise forge a sort position the same way a raw id could), and the wire format is unchanged. - Omit id and it's derived from createdAt's timestamp, so the two agree by construction; supply both and they're checked against each other with the same idTimestampSkewMs tolerance ScopedStack's grantee check uses. - updatedAt defaults to createdAt, not the actual current time, so a plain import doesn't fabricate a fake edit. - Added generateIdForTimestamp() to id.ts: deriving an id from an explicit createdAt must not go through generateId()'s monotonic "never sort before a live id already minted this process" floor, or a backdated import would silently get clamped forward to "now" the moment any live create() has run first. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01RKowhhMr6TnFEKsgaYtNYm --- .changeset/backdated-create.md | 20 ++++ docs/spec/data-model.md | 9 ++ packages/core/src/id.ts | 19 ++++ packages/core/src/index.ts | 1 + packages/core/src/stack.ts | 127 +++++++++++++++++++---- packages/core/tests/id.test.ts | 53 ++++++++++ packages/core/tests/scoped-stack.test.ts | 39 +++++++ packages/core/tests/stack.test.ts | 88 +++++++++++++++- 8 files changed, 335 insertions(+), 21 deletions(-) create mode 100644 .changeset/backdated-create.md diff --git a/.changeset/backdated-create.md b/.changeset/backdated-create.md new file mode 100644 index 0000000..51901dd --- /dev/null +++ b/.changeset/backdated-create.md @@ -0,0 +1,20 @@ +--- +'@haverstack/core': minor +--- + +Add `createdAt`/`updatedAt` options to unscoped `Stack.create()`, so an app can import an +existing corpus with its real dates instead of every record landing stamped with the import +moment. + +- Full-trust context only, like the existing client-minted `id` option: `ScopedStack.create()` + never accepts `createdAt`/`updatedAt`, and `POST /records` keeps refusing both, unchanged. +- Omit `id` and it is derived from `createdAt`'s timestamp, so the two agree by construction. + Supply both, and they are checked against each other using the same `idTimestampSkewMs` + tolerance `ScopedStack.create()`'s grantee check already uses (default 24 hours; `null` + disables this check too) — disagreement beyond that tolerance throws `StackValidationError` + rather than silently diverging. +- `updatedAt` defaults to `createdAt`, not to the actual current time, so a plain import + doesn't fabricate a fake edit and inflate version history. An `updatedAt` earlier than + `createdAt` is a validation error. + +See docs/spec/data-model.md § Record IDs. diff --git a/docs/spec/data-model.md b/docs/spec/data-model.md index d26e2c7..118e503 100644 --- a/docs/spec/data-model.md +++ b/docs/spec/data-model.md @@ -66,6 +66,15 @@ The same rules are enforced locally, so a client-minted ID behaves identically w - `Stack.create(typeId, content, { id })` validates charset, length, and the reserved prefix, and throws `StackConflictError` on a duplicate. This is a full-trust context (an embedded single-app stack, or the server's own code) — no clock-skew check. - `ScopedStack.create()` — a grantee minting an ID — applies the same validation **plus** the timestamp-skew check, since a grantee is exactly the untrusted actor who could otherwise forge a sort position. The tolerance is configurable per Stack via `Stack.create(adapter, { idTimestampSkewMs })` (default 24 hours; pass `null` to disable). +### Backdating on import + +`Stack.create()` also accepts `createdAt`/`updatedAt` (`UnscopedCreateRecordOptions`) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment. Unscoped only, same as the full-trust `id` option above: + +- **Never on `ScopedStack.create()`.** The same reasoning as the `id` skew check applies: a grantee could otherwise forge a sort position, this time through `createdAt` rather than `id`. `ScopedStack.create()` refuses both fields outright, even if a caller bypasses the type system to supply them. +- **Never over the wire.** `POST /records` keeps refusing client-supplied `createdAt`/`updatedAt`, unchanged — see [Wire format § Records](./wire-format.md#records). +- **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `ScopedStack` grantee check uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed. +- **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error. + ## Associations Tags, attachments, and relationships are unified under a single **Association** model. All three associate a Record with a labeled payload — the label carries semantic meaning (e.g. `"avatar"`, `"parent"`, `"reply-to"`). diff --git a/packages/core/src/id.ts b/packages/core/src/id.ts index c3cb0be..253737b 100644 --- a/packages/core/src/id.ts +++ b/packages/core/src/id.ts @@ -167,6 +167,25 @@ export const generateId = (timestamp: number = Date.now()): string => { return nowId + randChars; }; +/** + * Mint an ID for an arbitrary (typically past) timestamp — used by + * Stack.create() to derive an ID from an explicit `createdAt` when + * importing historical records. Deliberately bypasses generateId()'s + * monotonic `lastTimestamp` floor: that floor exists to protect *live* ID + * generation from a backward clock step (NTP correction, suspend/resume), + * and would otherwise clamp a deliberately historical timestamp forward to + * "now" the moment the process has minted any live ID past it — silently + * defeating the backdate it was asked for. Same-millisecond uniqueness for + * a historical timestamp is therefore left to a fresh random suffix each + * call rather than the live incrementing scheme; a collision surfaces the + * same way any client-supplied id collision does, as StackConflictError + * from the adapter. + */ +export const generateIdForTimestamp = (timestamp: number): string => { + const nowId = pad(crockford32Encode(timestamp), MIN_TIMESTAMP_LENGTH); + return nowId + generateRandChars(); +}; + // ------------------------------------------------------- // Format validation // ------------------------------------------------------- diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index d1b3c63..84530f9 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -37,6 +37,7 @@ export type { StackErrorCode, StackClient, CreateRecordOptions, + UnscopedCreateRecordOptions, StackOptions, GetRecordOptions, DeleteRecordOptions, diff --git a/packages/core/src/stack.ts b/packages/core/src/stack.ts index 793af31..f983f51 100644 --- a/packages/core/src/stack.ts +++ b/packages/core/src/stack.ts @@ -14,7 +14,7 @@ * Apps should never talk to a StackAdapter directly. */ -import { generateId, isValidIdFormat, idTimestamp } from './id.js'; +import { generateId, generateIdForTimestamp, isValidIdFormat, idTimestamp } from './id.js'; import { hashSchema, isCompatible, @@ -308,6 +308,32 @@ export type CreateRecordOptions = { unlisted?: boolean; }; +/** + * Stack.create()-only extension of CreateRecordOptions: backdating a + * record's clock fields for import (e.g. migrating an existing archive with + * its original dates). Never accepted by ScopedStack.create() — a grantee + * could otherwise forge a sort position the same way a raw `id` could — and + * never over the wire, where the server always assigns both fields. See + * docs/spec/data-model.md § Record IDs. + */ +export type UnscopedCreateRecordOptions = CreateRecordOptions & { + /** + * The record's creation time. When `id` is also supplied, its embedded + * timestamp must agree with this within `idTimestampSkewMs` (default 24h; + * see StackOptions.idTimestampSkewMs) or the create throws + * StackValidationError. Omit `id` to have it derived from this timestamp + * instead. Defaults to now. + */ + createdAt?: Date; + /** + * The record's last-modified time. Defaults to `createdAt` (or now, if + * `createdAt` is omitted too) — never to the actual current time — so a + * plain import doesn't fabricate a fake edit. Must not precede + * `createdAt`. + */ + updatedAt?: Date; +}; + export type ScopedStackOptions = { /** * The entity a delegated app acts for. Omit when the principal acts as @@ -325,10 +351,11 @@ export type StackOptions = { */ ownerProfile?: { name: string; handle?: string }; /** - * Clock-skew tolerance (ms) for the timestamp-prefix check - * ScopedStack.create() runs on grantee-supplied IDs; unscoped - * Stack.create() never runs it. Default: 24 hours; null disables. - * See docs/spec/data-model.md § Record IDs. + * Clock-skew tolerance (ms) for two timestamp-prefix checks: the one + * ScopedStack.create() runs on grantee-supplied IDs against the current + * time, and the one unscoped Stack.create() runs between an explicit `id` + * and an explicit `createdAt` when both are supplied. Default: 24 hours; + * null disables both. See docs/spec/data-model.md § Record IDs. */ idTimestampSkewMs?: number | null; }; @@ -799,18 +826,27 @@ function validateRecordId(id: string): void { } /** - * Timestamp-prefix plausibility check for grantee-minted IDs - * (ScopedStack.create()) — a grantee is untrusted and could otherwise mint - * an ID that forges its sort position. Pass null to disable. + * Timestamp-prefix plausibility check, shared by two callers that compare + * an ID's embedded millisecond against a different reference each: + * ScopedStack.create() against the current time (a grantee is untrusted and + * could otherwise mint an ID that forges its sort position), and unscoped + * Stack.create() against an explicit `createdAt` (the two must agree, not + * silently diverge — see docs/spec/data-model.md § Record IDs). Pass null + * to disable. */ -function validateIdTimestampSkew(id: string, toleranceMs: number | null): void { +function validateIdTimestampSkew( + id: string, + toleranceMs: number | null, + referenceMs: number, + referenceLabel: string, +): void { if (toleranceMs === null) return; - const skew = Math.abs(Date.now() - idTimestamp(id)); + const skew = Math.abs(referenceMs - idTimestamp(id)); if (skew > toleranceMs) { throw new StackValidationError([ { path: 'id', - message: `ID "${id}" timestamp is outside the allowed clock-skew tolerance (${toleranceMs}ms).`, + message: `ID "${id}" timestamp disagrees with ${referenceLabel} by more than the allowed clock-skew tolerance (${toleranceMs}ms).`, }, ]); } @@ -1317,12 +1353,15 @@ export class Stack implements StackClient { * Create a new record. Validates content against the type's schema. * `_group` records get their author stamped as the first `admin` roster * association here — the single stamping site for both Stack.create() - * and ScopedStack.create(). See docs/spec/identity.md § Group. + * and ScopedStack.create(). Full-trust context only: unlike + * ScopedStack.create(), `opts.createdAt`/`updatedAt` let a caller backdate + * an imported record's clock fields — see UnscopedCreateRecordOptions and + * docs/spec/data-model.md § Record IDs. */ async create = Record>( typeId: TypeId, content: T, - opts: CreateRecordOptions = {}, + opts: UnscopedCreateRecordOptions = {}, ): Promise { this.assertOpen(); const type = await this.getTypeCached(typeId); @@ -1336,6 +1375,13 @@ export class Stack implements StackClient { ...validatePermissions(opts.permissions), ...validateAssociations(opts.associations), ]; + if ( + opts.createdAt !== undefined && + opts.updatedAt !== undefined && + opts.updatedAt.getTime() < opts.createdAt.getTime() + ) { + errors.push({ path: 'updatedAt', message: 'updatedAt cannot precede createdAt.' }); + } if (errors.length > 0) { throw new StackValidationError(errors); } @@ -1348,19 +1394,47 @@ export class Stack implements StackClient { await this.checkBindingsOnCreate(typeId, content as Record); - if (opts.id !== undefined) validateRecordId(opts.id); + if (opts.id !== undefined) { + validateRecordId(opts.id); + // Only when both are explicit: an `id` alone (no createdAt) stays a + // pure position choice, exactly as before this option existed. + if (opts.createdAt !== undefined) { + validateIdTimestampSkew( + opts.id, + this.idTimestampSkewMsValue, + opts.createdAt.getTime(), + 'createdAt', + ); + } + } const associations = baseIdOf(typeId) === SYSTEM_TYPES.GROUP ? stampGroupAdmin(opts.associations, opts.entityId ?? this.ownerEntityId) : opts.associations; - const now = new Date(); + // createdAt drives the ID when the caller doesn't supply one, so the + // two agree by construction rather than by coincidence — the same + // relationship an explicit `id` is checked against above. updatedAt + // defaults to createdAt, not to the actual current time, so a plain + // import doesn't fabricate a fake edit. + const createdAt = opts.createdAt ?? new Date(); + const updatedAt = opts.updatedAt ?? createdAt; + // An explicit createdAt mints via generateIdForTimestamp(), which never + // clamps to "now" — generateId()'s monotonic floor would otherwise + // silently pull a deliberately historical id forward once this process + // has minted any live id past it. The no-createdAt path keeps + // generateId(), unaffected and still monotonic-safe. + const id = + opts.id ?? + (opts.createdAt !== undefined + ? generateIdForTimestamp(createdAt.getTime()) + : generateId(createdAt.getTime())); const record: StackRecord = { - id: opts.id ?? generateId(), + id, typeId, - createdAt: now, - updatedAt: now, + createdAt, + updatedAt, content, version: 1, ...(opts.parentId && { parentId: opts.parentId }), @@ -1374,7 +1448,7 @@ export class Stack implements StackClient { ...(opts.principalId && { updatedVia: opts.principalId }), ...(opts.permissions?.length && { permissions: opts.permissions }), ...(associations?.length && { associations }), - ...(opts.unlisted && { unlistedAt: now }), + ...(opts.unlisted && { unlistedAt: createdAt }), }; const created = await this.adapter.createRecord(record); @@ -3463,6 +3537,19 @@ export class ScopedStack implements StackClient { content: T, opts: CreateRecordOptions = {}, ): Promise { + // CreateRecordOptions carries no createdAt/updatedAt, so a type-checked + // caller can't reach this — but opts is forwarded whole to the + // unscoped Stack.create() below, which *does* accept them, and this + // method's own signature is exactly what stands between a caller that + // bypasses the type system (a raw JS caller, an `as any`) and a + // grantee backdating its own sort position. Refused rather than + // silently dropped, so an app never believes it published something it + // didn't. See docs/spec/data-model.md § Record IDs. + if ('createdAt' in opts || 'updatedAt' in opts) { + throw new StackPermissionError( + 'createdAt/updatedAt can only be set through an unscoped Stack; a scoped create always stamps the current time.', + ); + } const principal = this.principalEntityId; if (!principal) throw new StackPermissionError('Anonymous requesters cannot create records'); if (!(await this.checkCreateGrant(typeId))) { @@ -3492,7 +3579,7 @@ export class ScopedStack implements StackClient { await this.requireAppIdMatchesPrincipal(opts.appId); if (opts.id !== undefined) { validateRecordId(opts.id); - validateIdTimestampSkew(opts.id, this.idTimestampSkewMs); + validateIdTimestampSkew(opts.id, this.idTimestampSkewMs, Date.now(), 'the current time'); } if (opts.parentId !== undefined && !(await this.canReadReferent(opts.parentId))) { throw new StackPermissionError(); diff --git a/packages/core/tests/id.test.ts b/packages/core/tests/id.test.ts index f957b14..d719a0d 100644 --- a/packages/core/tests/id.test.ts +++ b/packages/core/tests/id.test.ts @@ -1,6 +1,7 @@ import { describe, test, expect, beforeEach, vi } from 'vitest'; import { generateId, + generateIdForTimestamp, crockford32Encode, crockford32Decode, isValidIdFormat, @@ -10,6 +11,7 @@ import { IdGenerationOverflowError, _setLastNowId, _setLastRandChars, + _setLastTimestamp, _resetIdState, } from '../src/id.js'; @@ -226,6 +228,57 @@ describe('generateId', () => { }); }); +// ------------------------------------------------------- +// generateIdForTimestamp +// ------------------------------------------------------- + +describe('generateIdForTimestamp', () => { + beforeEach(() => { + _resetIdState(); + }); + + test('timestamp prefix matches the encoded timestamp', () => { + const t = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(t); + expect(idTimestamp(id)).toBe(t); + }); + + test('produces a well-formed id', () => { + const id = generateIdForTimestamp(new Date('2020-06-15T12:00:00.000Z').valueOf()); + expect(isValidIdFormat(id)).toBe(true); + }); + + test('is not clamped by a later live generateId() call in the same process (regression)', () => { + // A live create() advances the monotonic floor to "now" — a much + // larger timestamp than any historical import date. + generateId(Date.now()); + + const historical = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(historical); + + // generateId() itself would have clamped this forward to "now"; + // generateIdForTimestamp() must not. + expect(idTimestamp(id)).toBe(historical); + }); + + test('does not advance generateId()’s monotonic floor for later live ids', () => { + // Minting a historical id must not make generateId() think the clock + // has already reached that timestamp when live generation resumes. + generateIdForTimestamp(new Date('2099-01-01').valueOf()); + + const now = Date.now(); + const liveId = generateId(now); + expect(idTimestamp(liveId)).toBe(now); + }); + + test('does not consult the live monotonic floor for a backward-looking timestamp', () => { + _setLastTimestamp(new Date('2024-06-01').valueOf()); + const historical = new Date('2020-06-15T12:00:00.000Z').valueOf(); + const id = generateIdForTimestamp(historical); + expect(idTimestamp(id)).toBe(historical); + }); +}); + // ------------------------------------------------------- // ID format validation // ------------------------------------------------------- diff --git a/packages/core/tests/scoped-stack.test.ts b/packages/core/tests/scoped-stack.test.ts index ac94036..25cc72d 100644 --- a/packages/core/tests/scoped-stack.test.ts +++ b/packages/core/tests/scoped-stack.test.ts @@ -923,6 +923,45 @@ describe('ScopedStack.create — client-supplied id', () => { }); }); +// ------------------------------------------------------- +// ScopedStack.create — createdAt/updatedAt refused +// ------------------------------------------------------- + +// CreateRecordOptions carries no createdAt/updatedAt, so a type-checked +// caller can't reach this path — these tests simulate a caller that +// bypasses the type system (raw JS, an `as any`) to confirm the runtime +// guard, not just the type, is what stands between a grantee and +// backdating its own sort position. +describe('ScopedStack.create — createdAt/updatedAt refused even past the type system', () => { + beforeEach(async () => { + await stack.defineType(COMMENT, 'Comment', { text: { kind: 'text', required: true } }); + await stack.grant(MEMBER, [{ actions: ['create'], typeId: COMMENT }]); + }); + + test('rejects a smuggled createdAt with StackPermissionError', async () => { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const opts: any = { createdAt: new Date('2020-01-01') }; + await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow( + StackPermissionError, + ); + }); + + test('rejects a smuggled updatedAt with StackPermissionError', async () => { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const opts: any = { updatedAt: new Date('2020-01-01') }; + await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow( + StackPermissionError, + ); + }); + + test('does not create a record when refused', async () => { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const opts: any = { createdAt: new Date('2020-01-01') }; + await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow(); + expect((await stack.query({ filter: { typeId: COMMENT } })).records).toHaveLength(0); + }); +}); + // ------------------------------------------------------- // ScopedStack — grant-based read // ------------------------------------------------------- diff --git a/packages/core/tests/stack.test.ts b/packages/core/tests/stack.test.ts index 15a8058..c6faabb 100644 --- a/packages/core/tests/stack.test.ts +++ b/packages/core/tests/stack.test.ts @@ -13,7 +13,7 @@ import { StackPayloadTooLargeError, StackClosedError, } from '../src/stack.js'; -import { generateId, crockford32Encode, IdGenerationError } from '../src/id.js'; +import { generateId, crockford32Encode, idTimestamp, IdGenerationError } from '../src/id.js'; import { InvalidDidError } from '../src/did.js'; import { MemoryAdapter, IncapableMemoryAdapter } from '../src/testing.js'; import { firstRecordedAttachment } from '../src/attachment-download.js'; @@ -520,6 +520,92 @@ describe('create — client-supplied id', () => { }); }); +// ------------------------------------------------------- +// create — backdating (createdAt / updatedAt) +// ------------------------------------------------------- + +describe('create — backdating (createdAt/updatedAt)', () => { + test('accepts an explicit createdAt and defaults updatedAt to match', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(createdAt); + }); + + test('accepts a distinct updatedAt when supplied', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-20T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt, updatedAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(updatedAt); + }); + + test('rejects an updatedAt preceding createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-10T12:00:00.000Z'); + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt, updatedAt }), + ).rejects.toThrow(StackValidationError); + }); + + test('derives the id from createdAt when no id is supplied', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(idTimestamp(record.id)).toBe(createdAt.valueOf()); + }); + + test('accepts an explicit id whose timestamp agrees with createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(createdAt.valueOf()); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('rejects an explicit id whose timestamp disagrees with createdAt beyond tolerance', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect(stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt })).rejects.toThrow( + StackValidationError, + ); + }); + + test('idTimestampSkewMs: null disables the id/createdAt consistency check too', async () => { + const permissiveAdapter = new MemoryAdapter({ ownerEntityId: 'owner-123', timezone: 'UTC' }); + const permissiveStack = await Stack.create(permissiveAdapter, { idTimestampSkewMs: null }); + await permissiveStack.defineType(NOTE_V1, 'Note', { text: { kind: 'text', required: true } }); + + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + const record = await permissiveStack.create(NOTE_V1, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('an id alone (no createdAt) stays a pure position choice — no consistency check applies', async () => { + const ancientId = idWithTimestamp(new Date('2000-01-01').valueOf()); + const before = new Date(); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { id: ancientId }); + expect(record.id).toBe(ancientId); + expect(record.createdAt.valueOf()).toBeGreaterThanOrEqual(before.valueOf()); + }); + + test('a prior live create() does not clamp a later backdated id forward (regression)', async () => { + // Advance generateId()'s monotonic floor to "now" via an ordinary, + // undated create() — the scenario a long-running import script hits + // once it has written anything live before importing historical data. + await stack.create(NOTE_V1, { text: 'live' }); + + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'backdated' }, { createdAt }); + expect(idTimestamp(record.id)).toBe(createdAt.valueOf()); + }); + + test('unlistedAt stamps from createdAt, not the actual current time', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt, unlisted: true }); + expect(record.unlistedAt).toEqual(createdAt); + }); +}); + // ------------------------------------------------------- // Type cache — create()/update()/etc. shouldn't pay a getType() // round trip on every write for a value that can't change. From e2799d456b1fc717ff2e855bd881f4d32c41f009 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 15:35:50 +0000 Subject: [PATCH 2/3] fix: validate backdating clock fields on unscoped Stack.create() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hardening follow-up to the createdAt/updatedAt backdating options. The privilege boundary those options draw is sound — ScopedStack.create() refuses both fields before doing anything else, and generateIdForTimestamp() neither reads nor advances generateId()'s monotonic floor, so an import can't perturb live ID generation. What was missing was input validation on the full-trust path, where three inputs a real import can produce were accepted and stored: - An Invalid Date (what a malformed source row parses to) has a NaN getTime(), and every comparison against NaN is false — so it didn't slip past the updatedAt/createdAt ordering check and the id/createdAt skew check, it switched them off. The record then persisted with an epoch-zero ID and a createdAt whose toISOString() throws RangeError in serializeRecord(), leaving that record and any wire response containing it permanently unreadable. - A pre-epoch createdAt surfaced as a bare RangeError ("Not defined for negative numbers!") thrown from inside the ID encoder, naming neither the field nor the record — and was accepted outright when an explicit `id` skipped ID derivation. - A createdAt past 3084-12-12 overflowed the 9-character timestamp prefix, minting a 13-character ID that isValidIdFormat() rejects — the library producing an ID it refuses on the way back in. Year-9999 sentinels are ordinary in imported data. Both fields are now checked for validity and for the range a record ID's timestamp prefix can encode, as a StackValidationError naming the field, whether or not an `id` is supplied. generateIdForTimestamp() carries the same bound itself, since unlike generateId() it encodes a caller-supplied timestamp rather than Date.now(). Also: - The ordering check now compares against the effective createdAt, so an updatedAt supplied on its own is caught too. Previously it required both fields, letting { updatedAt: } store a record modified before it was created. - Caller Dates are copied rather than stored by reference. An import loop that advances and reuses one Date across rows would otherwise retro-edit every record it had already written, with no version bump and no change event. Spec and changeset document the new rules, plus a note that backdated records fall behind an updatedAt sync cursor by construction. --- .changeset/backdated-create.md | 8 ++- docs/spec/data-model.md | 4 +- packages/core/src/id.ts | 18 +++++++ packages/core/src/stack.ts | 85 ++++++++++++++++++++++++----- packages/core/tests/id.test.ts | 28 ++++++++++ packages/core/tests/stack.test.ts | 90 ++++++++++++++++++++++++++++++- 6 files changed, 217 insertions(+), 16 deletions(-) diff --git a/.changeset/backdated-create.md b/.changeset/backdated-create.md index 51901dd..40f0f41 100644 --- a/.changeset/backdated-create.md +++ b/.changeset/backdated-create.md @@ -15,6 +15,12 @@ moment. rather than silently diverging. - `updatedAt` defaults to `createdAt`, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. An `updatedAt` earlier than - `createdAt` is a validation error. + `createdAt` is a validation error, including when `createdAt` defaulted to now. +- Both fields must be valid Dates within the range a record ID's timestamp prefix can + encode (1970-01-01 through 3084-12-12); anything else is a `StackValidationError`. An + `Invalid Date` in particular is refused rather than stored, since its `NaN` timestamp + would silently switch off the checks above instead of failing them. +- Dates are copied on the way in, so an import loop that advances and reuses a single + `Date` across rows doesn't retro-edit the records it already wrote. See docs/spec/data-model.md § Record IDs. diff --git a/docs/spec/data-model.md b/docs/spec/data-model.md index 118e503..a7ee9e7 100644 --- a/docs/spec/data-model.md +++ b/docs/spec/data-model.md @@ -73,7 +73,9 @@ The same rules are enforced locally, so a client-minted ID behaves identically w - **Never on `ScopedStack.create()`.** The same reasoning as the `id` skew check applies: a grantee could otherwise forge a sort position, this time through `createdAt` rather than `id`. `ScopedStack.create()` refuses both fields outright, even if a caller bypasses the type system to supply them. - **Never over the wire.** `POST /records` keeps refusing client-supplied `createdAt`/`updatedAt`, unchanged — see [Wire format § Records](./wire-format.md#records). - **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `ScopedStack` grantee check uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed. -- **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error. +- **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error — including when `createdAt` was left to default to now. +- **Both fields must be valid, representable Dates.** An `Invalid Date` (what `new Date()` yields for a malformed date string, a common shape for a bad row in an imported corpus) is a `StackValidationError`, not a record: its `getTime()` is `NaN`, and every comparison against `NaN` is false, so an unchecked one would switch off the ordering and skew checks above rather than fail them. The representable range is the range a record ID's 9-character timestamp prefix can encode — `1970-01-01T00:00:00.000Z` through `3084-12-12T12:41:28.831Z` — since a `createdAt` outside it has no ID that can agree with it. Content genuinely dated outside that window belongs in the record's own content fields, not in `createdAt`. +- **Backdated records are invisible to an `updatedAt` cursor.** A consumer syncing incrementally by `filter.updatedAt.after` will not see records imported with historical dates, because they land behind the cursor. Import against a full corpus read, not a change cursor. ## Associations diff --git a/packages/core/src/id.ts b/packages/core/src/id.ts index 253737b..647f8bc 100644 --- a/packages/core/src/id.ts +++ b/packages/core/src/id.ts @@ -147,6 +147,18 @@ export const _resetIdState = (): void => { // Public API // ------------------------------------------------------- +/** + * Largest timestamp (ms since epoch) that still encodes to the + * 9-character prefix an ID's format requires: 32^9 - 1, i.e. + * 3084-12-12T12:41:28.831Z. One millisecond past it the prefix grows to 10 + * characters and the ID no longer passes isValidIdFormat() — the library + * would be minting an ID it rejects on the way back in. generateId() can't + * reach this, since it encodes Date.now(); generateIdForTimestamp() takes + * whatever timestamp the caller asks for, so it is the one that has to + * check. + */ +export const MAX_ID_TIMESTAMP = Math.pow(BASE, MIN_TIMESTAMP_LENGTH) - 1; + /** * Generate a new Stack record ID. Time-sortable: lexicographic order * matches creation order, with same-millisecond IDs monotonically @@ -182,6 +194,12 @@ export const generateId = (timestamp: number = Date.now()): string => { * from the adapter. */ export const generateIdForTimestamp = (timestamp: number): string => { + if (!Number.isFinite(timestamp) || timestamp < 0 || timestamp > MAX_ID_TIMESTAMP) { + throw new IdGenerationError( + `Timestamp ${timestamp} cannot be encoded as an ID: expected 0…${MAX_ID_TIMESTAMP} ` + + `(1970-01-01T00:00:00.000Z…${new Date(MAX_ID_TIMESTAMP).toISOString()}).`, + ); + } const nowId = pad(crockford32Encode(timestamp), MIN_TIMESTAMP_LENGTH); return nowId + generateRandChars(); }; diff --git a/packages/core/src/stack.ts b/packages/core/src/stack.ts index f983f51..9a12842 100644 --- a/packages/core/src/stack.ts +++ b/packages/core/src/stack.ts @@ -14,7 +14,13 @@ * Apps should never talk to a StackAdapter directly. */ -import { generateId, generateIdForTimestamp, isValidIdFormat, idTimestamp } from './id.js'; +import { + generateId, + generateIdForTimestamp, + isValidIdFormat, + idTimestamp, + MAX_ID_TIMESTAMP, +} from './id.js'; import { hashSchema, isCompatible, @@ -825,6 +831,48 @@ function validateRecordId(id: string): void { } } +/** + * Validity and range check for the backdating options on unscoped + * Stack.create(). A Date is only as good as what the caller parsed it + * from, and the two failure modes both need catching here rather than + * downstream: + * + * - **Invalid Date** (`new Date('13/45/2020')` off a malformed import row) + * has a NaN getTime(), and every comparison against NaN is false — so an + * unchecked Invalid Date passes the updatedAt/createdAt ordering check + * and the id/createdAt skew check by turning them off, mints the + * epoch-zero ID `000000000xxx`, and persists a record whose + * `createdAt.toISOString()` throws RangeError in serializeRecord() — + * making that record, and any wire response containing it, + * permanently unreadable. + * - **Out of encodable range** — before 1970 crockford32Encode() throws a + * bare RangeError from deep inside the ID encoder, and past + * MAX_ID_TIMESTAMP the derived ID silently grows to 13 characters and + * fails isValidIdFormat(). + * + * Checked at the door for both, as a StackValidationError naming the + * field. Applies whether or not an `id` is supplied: a clock field outside + * this range is unrepresentable regardless of where the ID came from. + */ +function validateClockField(value: Date | undefined, path: string): ValidationError[] { + if (value === undefined) return []; + const ms = value.getTime(); + if (Number.isNaN(ms)) { + return [{ path, message: `${path} is not a valid Date.` }]; + } + if (ms < 0 || ms > MAX_ID_TIMESTAMP) { + return [ + { + path, + message: + `${path} is outside the representable range ` + + `(1970-01-01T00:00:00.000Z…${new Date(MAX_ID_TIMESTAMP).toISOString()}).`, + }, + ]; + } + return []; +} + /** * Timestamp-prefix plausibility check, shared by two callers that compare * an ID's embedded millisecond against a different reference each: @@ -1369,17 +1417,31 @@ export class Stack implements StackClient { throw new Error(`Unknown type: "${typeId}". Call defineType() first.`); } + // updatedAt defaults to createdAt, not to the actual current time, so a + // plain import doesn't fabricate a fake edit. + // + // Copied, never aliased: the caller keeps its own reference to any Date + // it passed, and an import loop that reuses one Date across rows + // (`d.setTime(...)` per record — the obvious way to write one) would + // otherwise retro-edit every record it had already written, with no + // version bump and no change event. createdAt drives updatedAt and + // unlistedAt below, so one copy taken here covers all three. + const createdAt = + opts.createdAt !== undefined ? new Date(opts.createdAt.getTime()) : new Date(); + const updatedAt = opts.updatedAt !== undefined ? new Date(opts.updatedAt.getTime()) : createdAt; + const errors = [ ...validateReservedKeys(content), ...validateContent(content, type.schema), ...validatePermissions(opts.permissions), ...validateAssociations(opts.associations), + ...validateClockField(opts.createdAt, 'createdAt'), + ...validateClockField(opts.updatedAt, 'updatedAt'), ]; - if ( - opts.createdAt !== undefined && - opts.updatedAt !== undefined && - opts.updatedAt.getTime() < opts.createdAt.getTime() - ) { + // Compared against the *effective* createdAt, so an updatedAt supplied + // on its own is caught too: defaulted-createdAt is now, which a + // backdated updatedAt alone would still precede. + if (opts.updatedAt !== undefined && updatedAt.getTime() < createdAt.getTime()) { errors.push({ path: 'updatedAt', message: 'updatedAt cannot precede createdAt.' }); } if (errors.length > 0) { @@ -1413,13 +1475,10 @@ export class Stack implements StackClient { ? stampGroupAdmin(opts.associations, opts.entityId ?? this.ownerEntityId) : opts.associations; - // createdAt drives the ID when the caller doesn't supply one, so the - // two agree by construction rather than by coincidence — the same - // relationship an explicit `id` is checked against above. updatedAt - // defaults to createdAt, not to the actual current time, so a plain - // import doesn't fabricate a fake edit. - const createdAt = opts.createdAt ?? new Date(); - const updatedAt = opts.updatedAt ?? createdAt; + // createdAt (hoisted above, alongside its validation) drives the ID + // when the caller doesn't supply one, so the two agree by construction + // rather than by coincidence — the same relationship an explicit `id` + // is checked against above. // An explicit createdAt mints via generateIdForTimestamp(), which never // clamps to "now" — generateId()'s monotonic floor would otherwise // silently pull a deliberately historical id forward once this process diff --git a/packages/core/tests/id.test.ts b/packages/core/tests/id.test.ts index d719a0d..237d04e 100644 --- a/packages/core/tests/id.test.ts +++ b/packages/core/tests/id.test.ts @@ -8,7 +8,9 @@ import { idTimestamp, RAND_SUFFIX_LENGTH, BASE, + IdGenerationError, IdGenerationOverflowError, + MAX_ID_TIMESTAMP, _setLastNowId, _setLastRandChars, _setLastTimestamp, @@ -277,6 +279,32 @@ describe('generateIdForTimestamp', () => { const id = generateIdForTimestamp(historical); expect(idTimestamp(id)).toBe(historical); }); + + // generateId() encodes Date.now() and so can never leave the encodable + // range; generateIdForTimestamp() encodes whatever it is handed, so the + // bound has to live here. Without it a timestamp past 32^9-1 overflows + // the 9-char prefix and yields a 13-char id the library itself rejects. + test('accepts the last encodable millisecond and keeps the id well-formed', () => { + const id = generateIdForTimestamp(MAX_ID_TIMESTAMP); + expect(isValidIdFormat(id)).toBe(true); + expect(idTimestamp(id)).toBe(MAX_ID_TIMESTAMP); + }); + + test('throws rather than minting a malformed id one ms past the range', () => { + expect(() => generateIdForTimestamp(MAX_ID_TIMESTAMP + 1)).toThrow(IdGenerationError); + }); + + test('throws on a negative (pre-epoch) timestamp', () => { + expect(() => generateIdForTimestamp(new Date('1969-07-20').valueOf())).toThrow( + IdGenerationError, + ); + }); + + test('throws on NaN rather than silently minting the epoch-zero id', () => { + expect(() => generateIdForTimestamp(new Date('not a date').valueOf())).toThrow( + IdGenerationError, + ); + }); }); // ------------------------------------------------------- diff --git a/packages/core/tests/stack.test.ts b/packages/core/tests/stack.test.ts index c6faabb..3d7e909 100644 --- a/packages/core/tests/stack.test.ts +++ b/packages/core/tests/stack.test.ts @@ -13,7 +13,14 @@ import { StackPayloadTooLargeError, StackClosedError, } from '../src/stack.js'; -import { generateId, crockford32Encode, idTimestamp, IdGenerationError } from '../src/id.js'; +import { + generateId, + crockford32Encode, + idTimestamp, + isValidIdFormat, + MAX_ID_TIMESTAMP, + IdGenerationError, +} from '../src/id.js'; import { InvalidDidError } from '../src/did.js'; import { MemoryAdapter, IncapableMemoryAdapter } from '../src/testing.js'; import { firstRecordedAttachment } from '../src/attachment-download.js'; @@ -604,6 +611,87 @@ describe('create — backdating (createdAt/updatedAt)', () => { const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt, unlisted: true }); expect(record.unlistedAt).toEqual(createdAt); }); + + // An Invalid Date's getTime() is NaN, and every comparison against NaN + // is false — so an unchecked Invalid Date doesn't slip past the + // ordering and skew checks, it switches them off, then persists a + // record that throws RangeError the moment anything serializes it. + test('rejects an Invalid Date createdAt rather than minting an epoch-zero id', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects an Invalid Date updatedAt', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { updatedAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('an Invalid Date createdAt cannot switch off the id/createdAt skew check', async () => { + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { id, createdAt: new Date('not a date') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects a pre-epoch createdAt as a validation error, not a raw RangeError', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('1969-07-20T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('rejects a pre-epoch createdAt even when an explicit id skips ID derivation', async () => { + const id = idWithTimestamp(0); + await expect( + stack.create( + NOTE_V1, + { text: 'hello' }, + { id, createdAt: new Date('1969-07-20T00:00:00.000Z') }, + ), + ).rejects.toThrow(StackValidationError); + }); + + // Past 32^9-1 ms the 9-char timestamp prefix overflows to 10, so the + // derived id would be 13 chars — one the library itself rejects via + // isValidIdFormat(). Year-9999 sentinels are ordinary in imported data. + test('rejects a far-future createdAt rather than minting a malformed 13-char id', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { createdAt: new Date('9999-01-01T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('accepts a createdAt at the last encodable millisecond', async () => { + const createdAt = new Date(MAX_ID_TIMESTAMP); + const record = await stack.create(NOTE_V1, { text: 'hello' }, { createdAt }); + expect(isValidIdFormat(record.id)).toBe(true); + expect(idTimestamp(record.id)).toBe(MAX_ID_TIMESTAMP); + }); + + // The one-sided case: createdAt defaults to now, which a backdated + // updatedAt on its own still precedes. + test('rejects an updatedAt preceding a defaulted createdAt', async () => { + await expect( + stack.create(NOTE_V1, { text: 'hello' }, { updatedAt: new Date('2000-01-01T00:00:00.000Z') }), + ).rejects.toThrow(StackValidationError); + }); + + test('does not alias the caller Date — mutating it after create leaves the record alone', async () => { + const cursor = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.create( + NOTE_V1, + { text: 'hello' }, + { createdAt: cursor, unlisted: true }, + ); + + // The shape of a real import loop: one Date advanced per row. + cursor.setFullYear(1990); + + const stored = await stack.get(record.id); + expect(stored?.createdAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + expect(stored?.updatedAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + expect(stored?.unlistedAt).toEqual(new Date('2020-06-15T12:00:00.000Z')); + }); }); // ------------------------------------------------------- From f5ef1e9ecbd32bfc9b75d8ebd109b3ee4824bc27 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 17:04:45 +0000 Subject: [PATCH 3/3] feat: let the stack owner backdate through ScopedStack, and over the wire MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the createdAt/updatedAt backdating options. As scoped, that change only worked for a caller with direct in-process access to an unscoped Stack — a server-hosted deployment had no path at all, since ScopedStack.create() (which every request through a server goes through) refused both fields unconditionally, and the wire spec separately told servers to always ignore them. createdAt/updatedAt now follow the same owner-acting-alone tier already used for hard delete, commitMigration(), and includeUnlisted: - ScopedStack.create() accepts both fields only when the requester is the stack owner, undelegated, authenticated as themselves. A grantee, or a delegated app acting for the owner (either direction — delegating to the owner, or the owner delegating to someone else), is still refused with StackPermissionError. - The existing id-vs-current-time skew check on ScopedStack.create() is skipped when createdAt is also supplied (only reachable by the owner), since Stack.create() below checks the id against createdAt instead — the "vs. now" check is for a live write, and a backdated one deliberately isn't. - No new wire-side code: adapter-api already serializes the full record (Dates included) via JSON.stringify, so a client always sent these fields — what changes is server-side handling. A server built on ScopedStack inherits the owner-only enforcement automatically, the same way it already inherits entityId/principalId assignment. Renamed UnscopedCreateRecordOptions to BackdatableCreateRecordOptions, since ScopedStack.create() now accepts it too (conditionally). Updated docs/spec/data-model.md and wire-format.md, and the changeset. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01RKowhhMr6TnFEKsgaYtNYm --- .changeset/backdated-create.md | 20 +++-- docs/spec/data-model.md | 10 ++- docs/spec/wire-format.md | 2 +- packages/core/src/index.ts | 2 +- packages/core/src/stack.ts | 102 ++++++++++++++--------- packages/core/tests/scoped-stack.test.ts | 101 +++++++++++++++++----- 6 files changed, 164 insertions(+), 73 deletions(-) diff --git a/.changeset/backdated-create.md b/.changeset/backdated-create.md index 40f0f41..73734ff 100644 --- a/.changeset/backdated-create.md +++ b/.changeset/backdated-create.md @@ -2,17 +2,23 @@ '@haverstack/core': minor --- -Add `createdAt`/`updatedAt` options to unscoped `Stack.create()`, so an app can import an -existing corpus with its real dates instead of every record landing stamped with the import -moment. +Add `createdAt`/`updatedAt` options to `Stack.create()`, so an app — or a stack owner, through +their own server — can import an existing corpus with its real dates instead of every record +landing stamped with the import moment. -- Full-trust context only, like the existing client-minted `id` option: `ScopedStack.create()` - never accepts `createdAt`/`updatedAt`, and `POST /records` keeps refusing both, unchanged. +- Unconditional on unscoped `Stack.create()`, like the existing client-minted `id` option. + `ScopedStack.create()` accepts the same two fields, but only from the stack owner acting + alone (undelegated, authenticated as themselves — the same tier that already gates hard + delete, `commitMigration()`, and `includeUnlisted`); a grantee, or a delegated app acting for + the owner, is refused with `StackPermissionError`. `POST /records` inherits this rule + automatically on a server built on `ScopedStack`: an owner-authenticated request may carry + both fields, anyone else's has them ignored, as before. - Omit `id` and it is derived from `createdAt`'s timestamp, so the two agree by construction. Supply both, and they are checked against each other using the same `idTimestampSkewMs` - tolerance `ScopedStack.create()`'s grantee check already uses (default 24 hours; `null` + tolerance the ordinary `id`-vs-current-time check already uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` - rather than silently diverging. + rather than silently diverging. An owner's plain `id`-only create through `ScopedStack` is + unaffected — it still gets the ordinary `id`-vs-current-time check, not this one. - `updatedAt` defaults to `createdAt`, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. An `updatedAt` earlier than `createdAt` is a validation error, including when `createdAt` defaulted to now. diff --git a/docs/spec/data-model.md b/docs/spec/data-model.md index a7ee9e7..c0517a1 100644 --- a/docs/spec/data-model.md +++ b/docs/spec/data-model.md @@ -68,13 +68,15 @@ The same rules are enforced locally, so a client-minted ID behaves identically w ### Backdating on import -`Stack.create()` also accepts `createdAt`/`updatedAt` (`UnscopedCreateRecordOptions`) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment. Unscoped only, same as the full-trust `id` option above: +`Stack.create()` also accepts `createdAt`/`updatedAt` (`BackdatableCreateRecordOptions`) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment: -- **Never on `ScopedStack.create()`.** The same reasoning as the `id` skew check applies: a grantee could otherwise forge a sort position, this time through `createdAt` rather than `id`. `ScopedStack.create()` refuses both fields outright, even if a caller bypasses the type system to supply them. -- **Never over the wire.** `POST /records` keeps refusing client-supplied `createdAt`/`updatedAt`, unchanged — see [Wire format § Records](./wire-format.md#records). -- **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `ScopedStack` grantee check uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed. +- **Unconditional on unscoped `Stack.create()`** — the same full-trust context as the `id` option above. +- **Owner-only on `ScopedStack.create()`.** Refused to everyone but the stack owner acting alone (undelegated, authenticated as themselves — the same `ownerActingAlone` tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`): a grantee, or a delegated app acting for the owner, could otherwise forge a sort position through `createdAt` the same way the `id` skew check exists to stop it forging one through `id`. `ScopedStack.create()` refuses both fields outright for anyone else. +- **Owner-authenticated only, over the wire.** `POST /records` may carry `createdAt`/`updatedAt` when the request authenticates as the stack owner acting alone; a server built on `ScopedStack` inherits this automatically, since it enforces the same rule a local caller does. Anyone else's request has both fields ignored, as every other server-assigned field already is — see [Wire format § Records](./wire-format.md#records). +- **`id` and `createdAt` must agree.** Omit `id` and it's derived from `createdAt`'s timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same `idTimestampSkewMs` tolerance the `id`-vs-current-time check above uses (default 24 hours; `null` disables this check too) — disagreement beyond that tolerance throws `StackValidationError` rather than silently diverging. Supplying `id` alone, with no `createdAt`, is unaffected: that stays a pure position choice, exactly as before this option existed — including for the owner, whose plain `id`-only creates through `ScopedStack` still get the ordinary `id`-vs-current-time check, not this one. - **`updatedAt` defaults to `createdAt`**, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an `updatedAt` earlier than `createdAt` is a validation error — including when `createdAt` was left to default to now. - **Both fields must be valid, representable Dates.** An `Invalid Date` (what `new Date()` yields for a malformed date string, a common shape for a bad row in an imported corpus) is a `StackValidationError`, not a record: its `getTime()` is `NaN`, and every comparison against `NaN` is false, so an unchecked one would switch off the ordering and skew checks above rather than fail them. The representable range is the range a record ID's 9-character timestamp prefix can encode — `1970-01-01T00:00:00.000Z` through `3084-12-12T12:41:28.831Z` — since a `createdAt` outside it has no ID that can agree with it. Content genuinely dated outside that window belongs in the record's own content fields, not in `createdAt`. +- **A backdated record's `updatedAt` predates its import**, by construction. A sync process that walks records by `updatedAt` to find what changed since a cursor will not see a backdated import as "recent" — which is the point (it isn't a recent edit), but worth knowing if a consumer expects an import to appear at the head of such a cursor. - **Backdated records are invisible to an `updatedAt` cursor.** A consumer syncing incrementally by `filter.updatedAt.after` will not see records imported with historical dates, because they land behind the cursor. Import against a full corpus read, not a change cursor. ## Associations diff --git a/docs/spec/wire-format.md b/docs/spec/wire-format.md index 5887121..35fa5ce 100644 --- a/docs/spec/wire-format.md +++ b/docs/spec/wire-format.md @@ -270,7 +270,7 @@ When present, the server applies the mutation only if the record's current versi `POST /records` accepts a full record body, including an optional client-supplied `id` — see [Record IDs](./data-model.md#record-ids) for the validation and duplicate-conflict rules the server applies. -**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, `createdAt`, and `updatedAt`, which the server assigns as it does on any write. `appId` is the deliberate exception — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path. +**`entityId`, `principalId`, `updatedBy` and `updatedVia` are assigned by the server from the authenticated session, and MUST be ignored if a request body carries them.** They are the fields that answer "who did this", so a server that echoes back what it was handed makes every one of them self-reported — and `principalId` exists precisely to be the field that isn't (see [Identity § Attribution and what can be trusted](./identity.md#attribution-and-what-can-be-trusted)). A client naming its own `principalId` could dress any write up as a verified app action, defeating the `_app` cross-check that reads it. `ScopedStack` already overrides both regardless of what a caller passes, so a server built on it inherits this; one that maps a request body onto `Stack` directly has to drop them itself. `updatedBy` and `updatedVia` answer the same question about the mutation that `entityId` and `principalId` answer about the Record, so they are assigned and ignored on identical terms — see [Data model § Authorship and attribution](./data-model.md#authorship-and-attribution). The same applies to `version`, which the server always assigns. `createdAt` and `updatedAt` are almost the same — server-assigned and ignored on every request but one: an **owner-authenticated** request (the stack owner acting alone, undelegated — the same tier that already gates hard delete, `commitMigration()`, and `includeUnlisted`) may include them, to backdate an imported record's clock fields instead of stamping the import moment. `ScopedStack` already enforces this — refusing both fields to anyone else, and checking a supplied `id` against `createdAt` rather than against the current time when they're present — so a server built on it inherits the rule automatically, same as `entityId`/`principalId` above; one that maps a request body onto `Stack` directly has to reproduce the owner check itself. See [Data model § Backdating on import](./data-model.md#backdating-on-import). `appId` is the deliberate exception among the rest — self-reported by design, and never a permission input. For `typeId: "_attachment@1"`, a non-owner requester gets `403` regardless of grants — see [Attachments](./attachments.md#creating-_attachment1-records-directly) for the refusal, its carve-out, and `POST /attachments` as the non-owner-safe combined path. ### Migration commit diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 84530f9..dc1a07a 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -37,7 +37,7 @@ export type { StackErrorCode, StackClient, CreateRecordOptions, - UnscopedCreateRecordOptions, + BackdatableCreateRecordOptions, StackOptions, GetRecordOptions, DeleteRecordOptions, diff --git a/packages/core/src/stack.ts b/packages/core/src/stack.ts index 9a12842..1462a5c 100644 --- a/packages/core/src/stack.ts +++ b/packages/core/src/stack.ts @@ -315,14 +315,19 @@ export type CreateRecordOptions = { }; /** - * Stack.create()-only extension of CreateRecordOptions: backdating a - * record's clock fields for import (e.g. migrating an existing archive with - * its original dates). Never accepted by ScopedStack.create() — a grantee - * could otherwise forge a sort position the same way a raw `id` could — and - * never over the wire, where the server always assigns both fields. See - * docs/spec/data-model.md § Record IDs. + * CreateRecordOptions extended with createdAt/updatedAt, for backdating a + * record's clock fields on import (e.g. migrating an existing archive with + * its original dates). Accepted unconditionally by unscoped Stack.create(). + * ScopedStack.create() accepts the same two fields, but only from the + * stack owner acting alone (undelegated, authenticated as themselves) — a + * grantee, or a delegated app acting for the owner, is refused, since + * either could otherwise forge a sort position the same way a raw `id` + * could. A server built on ScopedStack inherits this automatically: an + * owner-authenticated `POST /records` may carry both fields; anyone else's + * request has them ignored, as before. See docs/spec/data-model.md § + * Record IDs and docs/spec/wire-format.md § Records. */ -export type UnscopedCreateRecordOptions = CreateRecordOptions & { +export type BackdatableCreateRecordOptions = CreateRecordOptions & { /** * The record's creation time. When `id` is also supplied, its embedded * timestamp must agree with this within `idTimestampSkewMs` (default 24h; @@ -358,10 +363,12 @@ export type StackOptions = { ownerProfile?: { name: string; handle?: string }; /** * Clock-skew tolerance (ms) for two timestamp-prefix checks: the one - * ScopedStack.create() runs on grantee-supplied IDs against the current - * time, and the one unscoped Stack.create() runs between an explicit `id` - * and an explicit `createdAt` when both are supplied. Default: 24 hours; - * null disables both. See docs/spec/data-model.md § Record IDs. + * ScopedStack.create() runs on a non-backdated create's client-supplied + * `id` against the current time, and the one Stack.create() runs between + * an explicit `id` and an explicit `createdAt` when both are supplied — + * reached directly when unscoped, or via ScopedStack.create() when the + * requester is the owner acting alone. Default: 24 hours; null disables + * both. See docs/spec/data-model.md § Record IDs. */ idTimestampSkewMs?: number | null; }; @@ -874,13 +881,16 @@ function validateClockField(value: Date | undefined, path: string): ValidationEr } /** - * Timestamp-prefix plausibility check, shared by two callers that compare - * an ID's embedded millisecond against a different reference each: - * ScopedStack.create() against the current time (a grantee is untrusted and - * could otherwise mint an ID that forges its sort position), and unscoped - * Stack.create() against an explicit `createdAt` (the two must agree, not - * silently diverge — see docs/spec/data-model.md § Record IDs). Pass null - * to disable. + * Timestamp-prefix plausibility check, shared by callers that compare an + * ID's embedded millisecond against a different reference each: + * ScopedStack.create() against the current time for any non-backdated + * create (a grantee is untrusted and could otherwise mint an ID that + * forges its sort position, and this is the one check standing between a + * delegated or grantee caller and doing so), and Stack.create() — reached + * directly when unscoped, or via ScopedStack.create() when the requester + * is the owner acting alone with an explicit `createdAt` — against that + * `createdAt` instead (the two must agree, not silently diverge — see + * docs/spec/data-model.md § Record IDs). Pass null to disable. */ function validateIdTimestampSkew( id: string, @@ -1401,15 +1411,17 @@ export class Stack implements StackClient { * Create a new record. Validates content against the type's schema. * `_group` records get their author stamped as the first `admin` roster * association here — the single stamping site for both Stack.create() - * and ScopedStack.create(). Full-trust context only: unlike - * ScopedStack.create(), `opts.createdAt`/`updatedAt` let a caller backdate - * an imported record's clock fields — see UnscopedCreateRecordOptions and - * docs/spec/data-model.md § Record IDs. + * and ScopedStack.create(). `opts.createdAt`/`updatedAt` let a caller + * backdate an imported record's clock fields — unconditionally here; + * ScopedStack.create() forwards to this same method, but only reaches + * this far with them when the requester is the stack owner acting alone + * — see BackdatableCreateRecordOptions and docs/spec/data-model.md § + * Record IDs. */ async create = Record>( typeId: TypeId, content: T, - opts: UnscopedCreateRecordOptions = {}, + opts: BackdatableCreateRecordOptions = {}, ): Promise { this.assertOpen(); const type = await this.getTypeCached(typeId); @@ -3589,28 +3601,34 @@ export class ScopedStack implements StackClient { * reference-creating options gated, and non-owner `_attachment@1` * creation refused save one carve-out. A scoped create always stamps * authorship — an absent entityId means an unscoped `Stack` wrote it. - * See docs/spec/access-control.md and docs/spec/attachments.md. + * `createdAt`/`updatedAt` are refused to everyone but the owner acting + * alone — see the guard below and docs/spec/data-model.md § Record IDs. + * See also docs/spec/access-control.md and docs/spec/attachments.md. */ async create = Record>( typeId: TypeId, content: T, - opts: CreateRecordOptions = {}, + opts: BackdatableCreateRecordOptions = {}, ): Promise { - // CreateRecordOptions carries no createdAt/updatedAt, so a type-checked - // caller can't reach this — but opts is forwarded whole to the - // unscoped Stack.create() below, which *does* accept them, and this - // method's own signature is exactly what stands between a caller that - // bypasses the type system (a raw JS caller, an `as any`) and a - // grantee backdating its own sort position. Refused rather than - // silently dropped, so an app never believes it published something it - // didn't. See docs/spec/data-model.md § Record IDs. - if ('createdAt' in opts || 'updatedAt' in opts) { + const principal = this.principalEntityId; + if (!principal) throw new StackPermissionError('Anonymous requesters cannot create records'); + // createdAt/updatedAt let a caller backdate a record's clock fields — + // and, without `id` also supplied, its sort position too. Refused to + // everyone but the owner acting alone (undelegated, authenticated as + // themselves): a grantee is exactly the untrusted actor the `id` + // skew check below already exists to stop from forging a sort + // position, and a delegated app acting for the owner inherits none of + // the owner's extra trust — same reasoning as mayGrantAccess() below. + // Refused rather than silently dropped, so an app never believes it + // published something it didn't. This is also the enforcement a + // server built on ScopedStack inherits for `POST /records`: an + // owner-authenticated request may carry both fields, anyone else's + // has them ignored. + if (('createdAt' in opts || 'updatedAt' in opts) && !this.ownerActingAlone) { throw new StackPermissionError( - 'createdAt/updatedAt can only be set through an unscoped Stack; a scoped create always stamps the current time.', + 'createdAt/updatedAt can only be set by the stack owner acting alone; a grantee or delegated create always stamps the current time.', ); } - const principal = this.principalEntityId; - if (!principal) throw new StackPermissionError('Anonymous requesters cannot create records'); if (!(await this.checkCreateGrant(typeId))) { throw new StackPermissionError(`No create grant for type "${typeId}"`); } @@ -3638,7 +3656,15 @@ export class ScopedStack implements StackClient { await this.requireAppIdMatchesPrincipal(opts.appId); if (opts.id !== undefined) { validateRecordId(opts.id); - validateIdTimestampSkew(opts.id, this.idTimestampSkewMs, Date.now(), 'the current time'); + // Skipped when createdAt is also supplied: only the owner reaches + // here with that combination (checked above), and Stack.create() + // below checks the id against createdAt instead of "now" — the + // check here exists for a live grantee write, and a backdated + // owner create is deliberately not one. See + // docs/spec/data-model.md § Record IDs. + if (opts.createdAt === undefined) { + validateIdTimestampSkew(opts.id, this.idTimestampSkewMs, Date.now(), 'the current time'); + } } if (opts.parentId !== undefined && !(await this.canReadReferent(opts.parentId))) { throw new StackPermissionError(); diff --git a/packages/core/tests/scoped-stack.test.ts b/packages/core/tests/scoped-stack.test.ts index 25cc72d..edafa64 100644 --- a/packages/core/tests/scoped-stack.test.ts +++ b/packages/core/tests/scoped-stack.test.ts @@ -924,42 +924,99 @@ describe('ScopedStack.create — client-supplied id', () => { }); // ------------------------------------------------------- -// ScopedStack.create — createdAt/updatedAt refused +// ScopedStack.create — createdAt/updatedAt: owner acting alone only // ------------------------------------------------------- -// CreateRecordOptions carries no createdAt/updatedAt, so a type-checked -// caller can't reach this path — these tests simulate a caller that -// bypasses the type system (raw JS, an `as any`) to confirm the runtime -// guard, not just the type, is what stands between a grantee and -// backdating its own sort position. -describe('ScopedStack.create — createdAt/updatedAt refused even past the type system', () => { +describe('ScopedStack.create — createdAt/updatedAt refused to anyone but the owner acting alone', () => { beforeEach(async () => { await stack.defineType(COMMENT, 'Comment', { text: { kind: 'text', required: true } }); await stack.grant(MEMBER, [{ actions: ['create'], typeId: COMMENT }]); }); - test('rejects a smuggled createdAt with StackPermissionError', async () => { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - const opts: any = { createdAt: new Date('2020-01-01') }; - await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow( - StackPermissionError, - ); + test('rejects a grantee-supplied createdAt with StackPermissionError', async () => { + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); }); - test('rejects a smuggled updatedAt with StackPermissionError', async () => { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - const opts: any = { updatedAt: new Date('2020-01-01') }; - await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow( - StackPermissionError, - ); + test('rejects a grantee-supplied updatedAt with StackPermissionError', async () => { + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { updatedAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); }); test('does not create a record when refused', async () => { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - const opts: any = { createdAt: new Date('2020-01-01') }; - await expect(stack.asEntity(MEMBER).create(COMMENT, { text: 'hello' }, opts)).rejects.toThrow(); + await expect( + stack + .asEntity(MEMBER) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(); expect((await stack.query({ filter: { typeId: COMMENT } })).records).toHaveLength(0); }); + + test('rejects createdAt from a principal delegated to act for the owner (not owner acting alone)', async () => { + // subjectEntityId === OWNER satisfies checkCreateGrant()'s owner-subject + // carve-out, so this exercises the createdAt/updatedAt gate itself + // rather than getting stopped earlier by a missing create grant. + await expect( + stack + .asEntity(MEMBER, { onBehalfOf: OWNER }) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('rejects createdAt when the owner delegates for someone else — the owner’s trust does not transfer to the subject', async () => { + await expect( + stack + .asEntity(OWNER, { onBehalfOf: MEMBER }) + .create(COMMENT, { text: 'hello' }, { createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackPermissionError); + }); + + test('owner acting alone may set createdAt, and updatedAt defaults to match', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const record = await stack.asEntity(OWNER).create(COMMENT, { text: 'hello' }, { createdAt }); + expect(record.createdAt).toEqual(createdAt); + expect(record.updatedAt).toEqual(createdAt); + }); + + test('owner acting alone may set updatedAt distinct from createdAt', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const updatedAt = new Date('2020-06-20T12:00:00.000Z'); + const record = await stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { createdAt, updatedAt }); + expect(record.updatedAt).toEqual(updatedAt); + }); + + test('owner acting alone: an id agreeing with createdAt is accepted, ignoring the "vs. now" skew check', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(createdAt.valueOf()); + const record = await stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { id, createdAt }); + expect(record.id).toBe(id); + }); + + test('owner acting alone: an id disagreeing with createdAt is rejected against createdAt, not "now"', async () => { + const createdAt = new Date('2020-06-15T12:00:00.000Z'); + const id = idWithTimestamp(new Date('2000-01-01').valueOf()); + await expect( + stack.asEntity(OWNER).create(COMMENT, { text: 'hello' }, { id, createdAt }), + ).rejects.toThrow(StackValidationError); + }); + + test('owner acting alone still gets the reserved-prefix and format checks on a backdated id', async () => { + await expect( + stack + .asEntity(OWNER) + .create(COMMENT, { text: 'hello' }, { id: 'too-short', createdAt: new Date('2020-01-01') }), + ).rejects.toThrow(StackQueryError); + }); }); // -------------------------------------------------------