diff --git a/.changeset/lazy-hounds-search.md b/.changeset/lazy-hounds-search.md new file mode 100644 index 0000000..ba55624 --- /dev/null +++ b/.changeset/lazy-hounds-search.md @@ -0,0 +1,21 @@ +--- +'@haverstack/record-adapter-sqlite': patch +--- + +Association query filters read through their indexes + +`tags`, `hasAttachment`, `attachmentFileId` and `relatedTo` were correlated +`EXISTS` subqueries, which make SQLite scan every record and probe the association +primary key for each. Phrased as semi-joins, the planner drives from the association +side instead — reading the matching rows through `idx_assoc_kind_label`, +`idx_assoc_kind_file_id`, `idx_file_refs_file_id` or `idx_assoc_related`, then looking +up those records. The cost of an association filter becomes proportional to how many +records match it rather than to how many the stack holds, so the gain grows with +selectivity: the more precisely you ask, the more you save. + +Measured on 20k records with 4k associations, none of the four now needs a full table +scan. `attachmentFileId` benefits most, at 8.8ms to 0.14ms, because SQLite resolves +its two-sided condition as a multi-index OR across both indexes rather than scanning +once and probing twice. + +Results are unchanged; this is the same set of records, found a different way. diff --git a/.changeset/quiet-pandas-tickle.md b/.changeset/quiet-pandas-tickle.md new file mode 100644 index 0000000..17bcfd6 --- /dev/null +++ b/.changeset/quiet-pandas-tickle.md @@ -0,0 +1,51 @@ +--- +'@haverstack/core': minor +'@haverstack/record-adapter-sqlite': minor +'@haverstack/conformance-fixtures': minor +'@haverstack/adapter-api': minor +--- + +Relationship associations carry a discriminated `target` instead of a bare `recordId` + +A relationship's target now names which identifier space its value belongs to: +`{ scope: 'record', recordId, stackUrl? }` for a Record here or in another stack, +`{ scope: 'entity', entityId }` for a DID, and `{ scope: 'external', ns, id }` for +anything outside the stack — an ATProto post, an ActivityPub actor, an email address, +a URL. Core expresses the reference and never dereferences it, so no protocol is +privileged. + +The `entity` arm closes a gap in the identity model rather than only enabling external +references: group rosters stored member DIDs in a field typed `RecordId`, and the +permission path compared the two as plain strings. A roster entry carrying a `record` +target now confers nothing, even when its value equals a member's DID. + +`RecordFilter.relatedTo` moves with it. It names a label, a target, or both, and each +is a pattern: a bare `label` matches every target under it, and an external target with +no `id` matches a whole namespace. A `record` target with no `stackUrl` matches only +local targets — absence names this stack rather than acting as a wildcard. Label-only +and namespace-wide queries were not expressible before. "Carries any relationship at +all" is deliberately not expressible, in line with `tags` and `hasAttachment`, which +have no match-any form either. + +Reference-creation gating now applies only to a relationship naming a Record in this +stack; the other arms name nothing core can resolve, so there is no access for the +gate to protect. The SQLite association table gains `related_scope`, `related_ns` and +`related_stack` columns, all part of the primary key — so two copies of one record on +two networks are two associations rather than a silent no-op. Existing stack files +predate those columns and must be recreated. + +Over the wire, the relationship filter's scope is implied by which parameters appear +(`relatedTo`/`relatedToStack`, `relatedToEntity`, or `relatedToNs`/`relatedToId`), and +a request mixing scopes is rejected with 400. At least one is always present, so the +filter cannot encode to an empty query string and widen the query it meant to narrow. + +A target names exactly one thing, exactly one way, and both halves are enforced at +runtime rather than only by the type — a target reaching a server in a request body, +or a filter decoded from query parameters, is a plain object the type never saw. A +`scope` outside the three, or an empty string where a target names something, is +rejected with `StackValidationError`; a `relatedTo` naming neither a label nor a target +is rejected with `StackQueryError` instead of matching every Record carrying a +relationship. This stack is named by omitting `stackUrl`, never by sending an empty +one: storage, association identity and the filter all read absent and empty as this +stack, and reference-creation gating now reads them that way too, so both spellings of +a local Record require read access to it. diff --git a/AGENTS.md b/AGENTS.md index 9afbe51..a68a782 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,6 +52,7 @@ Design lives in [`docs/spec.md`](./docs/spec.md) and [`docs/spec/`](./docs/spec/ - A change to observable behavior (API contract, permission rule, wire shape, error mapping) **updates the spec in the same change**. - Section names are load-bearing — code comments reference them as `docs/spec/.md § Section`. Verify a section exists before linking to it, and update inbound references when renaming a heading. - Needing a long comment to explain a rule that isn't in the spec means the spec is missing a section. Add it. +- **Spec prose states the system as it is, never how it got there.** The no-previous-implementations rule above applies to prose: "is in core now", "this used to be flat", "when #16 lands" date the document and describe a system the reader can't see. Say which layer carries what instead. This covers `docs/commons/` too — a design guide, not a changelog. ## No backward compatibility diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 92c118d..adc2ba8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -187,6 +187,23 @@ A change to observable behavior — an API contract, a permission rule, a wire s When you link to a spec section from a comment, use the `docs/spec/.md § Section` form and make sure the section actually exists. Section names are load-bearing; renaming a heading means updating the references to it. +**Prose describes the system as it is, not how it got there.** The [no-references-to-previous-implementations](#comments) rule is not only about code comments — it is the same rule, and it bites harder in the spec, because a spec is read by people deciding what to build against. "Relationship targets are in core now", "`relatedTo` used to require a `recordId`", "reconciliation is expected when #15 lands" all date the document and describe a system the reader cannot see. They also rot silently: the sentence stays true-sounding long after the change it refers to stopped being news. + +```md + + +The reference fabric this rests on is in core now. What remains outstanding is +the bridge. + + + +Core carries the reference fabric: relationship targets that name a record in +someone else's stack, and a `relatedTo` filter that queries them. A bridge +carries its own translation table. +``` + +This applies to the Schema Commons ([`docs/commons/`](./docs/commons/)) as much as to `docs/spec/` — a design guide describing what a type means, not a record of how the type arrived at that meaning. Issue numbers in prose are usually the same mistake wearing a different hat: `#204` tells a reader nothing they can act on, and a document that sequences itself against open issues stops being true the moment one closes. Where the reasoning matters, state it. + --- ## No backward compatibility yet diff --git a/README.md b/README.md index f9df571..ae5fd46 100644 --- a/README.md +++ b/README.md @@ -201,7 +201,14 @@ Tags, attachments, and relationships are unified under a single model: ```ts { kind: 'tag', label: 'favourite' } { kind: 'attachment', label: 'avatar', fileId: '...' } -{ kind: 'relationship', label: 'reply-to', recordId: '...' } +{ kind: 'relationship', label: 'reply-to', target: { scope: 'record', recordId: '...' } } +``` + +A relationship's `target` says which identifier space its value lives in — a Record here or in another stack (`{ scope: 'record', recordId, stackUrl? }`), a "who" as a DID (`{ scope: 'entity', entityId }`), or something outside the stack entirely (`{ scope: 'external', ns, id }`). That last one is how a record points at an ATProto post, an ActivityPub actor, an email address or a plain URL: Haverstack expresses the reference and never dereferences it, so no protocol is privileged. + +```ts +{ kind: 'relationship', label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'at://did:plc:abc/app.bsky.feed.post/3k4' } } ``` ### Migrations diff --git a/docs/commons/README.md b/docs/commons/README.md index 0f0b651..7089da3 100644 --- a/docs/commons/README.md +++ b/docs/commons/README.md @@ -131,9 +131,12 @@ these. ## Cross-type conventions Some semantics belong to no single type. These association labels carry commons -authority on **any** record, of any type: +authority on **any** record, of any type. Relationship labels are orthogonal to the +target arm they ride on: a label says what the reference _means_, a target says which +identifier space it lives in, and every label below works with any arm that makes +sense for it. -- **`location`** — `{ kind: 'relationship', label: 'location', recordId: }` +- **`location`** — `{ kind: 'relationship', label: 'location', target: { scope: 'record', recordId: } }` points at a [`place`](./place.md) record: the geotagged photo, the note written at a café, the check-in. Apps that understand places understand every located record for free, whatever its type. @@ -141,17 +144,35 @@ authority on **any** record, of any type: referenced from a record's body text (`note`, `article`, `page`, `message`). How the body refers to the embed is app territory in v1; a commons syntax is an expected follow-up proposal. -- **`series`** — `{ kind: 'relationship', label: 'series', recordId }` groups records +- **`series`** — `{ kind: 'relationship', label: 'series', target: { scope: 'record', recordId } }` groups records that are occurrences of one recurring thing (materialized [`event`](./event.md) occurrences are the motivating case). Reserved now so recurrence proposals build on it rather than around it. -- **`author`** — `{ kind: 'relationship', label: 'author', recordId: }` - attributes any record to a person the stack knows, pointing at a - [`contact`](./contact.md) (or `_entity`) record. Complements, never replaces, a - displayed-byline string like `article.author`: the string is what the work says - about itself (a property, faithful to the work as published); the association is - what _you_ know about the world. Multiple `author` associations express - co-authorship. Prior art: ActivityStreams `attributedTo`. +- **`author`** — attributes any record to a person the stack knows. Two forms, and the + target arm is what distinguishes them: an `entity` target + (`{ scope: 'entity', entityId: }`) names the identity itself, while a `record` + target pointing at a [`contact`](./contact.md) or `_entity` record names your card + about them. Complements, never replaces, a displayed-byline string like + `article.author`: the string is what the work says about itself (a property, faithful + to the work as published); the association is what _you_ know about the world. + Multiple `author` associations express co-authorship. Prior art: ActivityStreams + `attributedTo`. +- **`alias`** — `{ kind: 'relationship', label: 'alias', target: { scope: 'external', ns, id } }` + on an `_entity` record records that this identity is also known by that identifier + elsewhere: `{ ns: 'atproto', id: 'did:plc:…' }`, `{ ns: 'activitypub', id: }`, + `{ ns: 'email', id: 'alice@example.com' }`. It is the resolution primitive — an + inbound record whose author arrives as a foreign identifier is matched back to a + known entity with one indexed `relatedTo` query — which is why it is an association + and not a content field: array fields are opaque to the query engine, so the same + list inside `_entity.content` would force a scan of every entity record. Machine + identifiers only; a person's own words about someone belong on a `contact`. +- **`syndicated-to`** — `{ kind: 'relationship', label: 'syndicated-to', target: { scope: 'external', ns, id } }` + records that a copy of this record was published elsewhere. The canonical copy stays + in the stack; a bridge publishes and stamps the copy's address here. Asking a + namespace without an id ("everything already on ATProto") is the inventory query a + syndication tool runs. Where a copy has a lifecycle of its own — retraction state, a + remote content address, which account it went out from — that is a record of the + bridge's own type, related back to this one; the label alone carries only the link. Cross-type conventions are governed like fields: proposing one is proposing it for every record in every stack, so the bar is correspondingly higher. @@ -233,9 +254,8 @@ concrete intended writer exists — the group cluster graduates when a group-too or demo is real, building on the grant/group primitives (`_group`, type-level grants) documented in the identity and access-control specs. -Deliberately absent from the initial set: `post` (public social shapes are reconciled -with the ATProto-compat RFC, #15 — `message` is the group-scoped shape, not the social -one), recurrence rules (see `event`: occurrences are materialized in @1), +Deliberately absent from the initial set: `post` (the broadcast contract — `message` is +the group-scoped shape, not the social one), recurrence rules (see `event`: occurrences are materialized in @1), `file`/`document` (a first-class `file` type is expected to follow `photo`'s pattern; until a real writer needs it, a record plus attachment covers it), and `checkin` (subsumed by the `location` cross-type convention plus any record). diff --git a/docs/commons/article.md b/docs/commons/article.md index 12a58c5..bbdeb14 100644 --- a/docs/commons/article.md +++ b/docs/commons/article.md @@ -51,8 +51,9 @@ await stack.defineType('org.haverstack/article@1', 'Article', { the canonical location back to the record). `url` present therefore means "this work lives somewhere on the web" — symmetrically for authored and captured articles — and any app can render a "view published" link. Syndicated copies elsewhere never - overwrite it: canonical means canonical; other locations are sidecar territory (or - future syndication relationships, #15). + overwrite it: canonical means canonical, and a copy's location is a `syndicated-to` + relationship with an external target — see + [Cross-type conventions](./README.md#cross-type-conventions). - **Cover image**: attachment association with label `cover` (JSON Feed `image`, h-entry `u-featured`). - **Embedded media** in `text`: attachment associations with label `embed`, per the @@ -101,9 +102,8 @@ keep the paired bookmark record described above. - Layout, theme, rendering hints — presentation is the publishing app's concern. - `slug` and site-structure fields — that is [`page`](./page.md) territory; an article is a work in a feed, a page is a node in a site tree. -- Social/microblog `post` semantics (replies, reposts, mentions) — deferred to - reconciliation with the ATProto-compat RFC (#15). An article is a work; a post is an - utterance. +- Social/microblog `post` semantics (replies, reposts, mentions) — a separate contract. + An article is a work; a post is an utterance. - Comments — already covered, no new type: a comment is a [`message`](./message.md) whose `parentId` is the article (sent into a shared space, moment-indexed, tamper-evident), while a reader's private marginalia is a `note` with an `about` diff --git a/docs/commons/contact.md b/docs/commons/contact.md index 11dd497..2ea8147 100644 --- a/docs/commons/contact.md +++ b/docs/commons/contact.md @@ -45,9 +45,13 @@ await stack.defineType('org.haverstack/contact@1', 'Contact', { **Labels** (vCard's `TYPE` parameter, humanized): an open vocabulary of user-facing words. Well-known values: `"home"`, `"work"`, `"mobile"`. Unknown labels are displayed verbatim — they are the user's words ("boat phone" is legal and correct), not a -machine namespace. Absent means unspecified. This is deliberately the same -labeled-multi-value shape as #15's revised `externalIds` on `EntityContent`, differing -only in that `label` is for humans where `ns` is for machines. +machine namespace. Absent means unspecified. + +These are **directory data the user typed**, not machine identifiers. The resemblance +to a relationship's `{ scope: 'external', ns, id }` target is superficial and the two +do different jobs: an `alias` relationship is what an app resolves an inbound record's +author _through_, while `contact.emails` is what a person reads. A contact that is also +a known principal carries both — see the conventions below. Note that array fields are opaque to the query engine; apps filter contacts by `name` or via associations, not by address. @@ -56,6 +60,10 @@ or via associations, not by address. - **Avatar**: attachment association with label `avatar` — the spec's own example of an attachment association, honored here. +- **Alias identifiers**: a machine identifier for someone in another system — + `did:plc:…`, an ActivityPub actor URL, an npub — is an `alias` relationship with an + external target, on the `_entity` record rather than here. See + [Cross-type conventions](./README.md#cross-type-conventions). - **Contact ↔ entity linkage**: when a contact _is_ a known principal (they appear in grants, groups, or authorship), link the records with `{ kind: 'relationship', label: 'entity', recordId: <_entity record id> }` on the diff --git a/docs/commons/message.md b/docs/commons/message.md index d8cd5cd..86bc49f 100644 --- a/docs/commons/message.md +++ b/docs/commons/message.md @@ -12,9 +12,8 @@ concerns and have no representation here. **This is not the social post.** A `message` is addressed to a group by living in that group's stack; a social post is broadcast to the world under a public identity. The -social shape (reposts, mentions, public replies) belongs to the ATProto-compat RFC -(#15) and is still deferred; this type must not foreclose it, and reconciliation is -expected when #15 lands. +broadcast shape is its own type — see [Choosing a text type](./text-types.md) — and +nothing here forecloses it: the two differ in contract, not in fields. Messages are **sent** — speech addressed to others, whose meaning is indexed to its moment and thread. For the boundary with `note` (kept) and `article` (published), see @@ -84,7 +83,7 @@ discussion. ## Deliberately excluded -- Social-post semantics — #15 territory (see above). +- Social-post semantics — the broadcast contract's territory (see above). - Reactions — a future micro-proposal (likely tag associations by non-authors — which needs finer-grained association permissions than today's `update-own`/`update-any` grant actions provide). diff --git a/docs/commons/place.md b/docs/commons/place.md index 3df06e2..45ccb2e 100644 --- a/docs/commons/place.md +++ b/docs/commons/place.md @@ -45,7 +45,7 @@ await stack.defineType('org.haverstack/place@1', 'Place', { "I was here," addressed to no one — is a (possibly empty-text) `note` with a `location` association, its `createdAt` the check-in time; "I'm at the café, come join me" sent to a group is a `message` with one; the Foursquare-style public - check-in is the future broadcast `post` (#15) with one. The `location` association + check-in is the future broadcast `post` with one. The `location` association is the invariant; the contract varies with the act. - **Deduplication** is app-side: two records with the same coordinates are two records. An app importing venues should query by exact coordinates before creating. diff --git a/docs/commons/text-types.md b/docs/commons/text-types.md index c2d5e5f..ccffb19 100644 --- a/docs/commons/text-types.md +++ b/docs/commons/text-types.md @@ -74,8 +74,8 @@ that being read as a finished work is its purpose. | Sharing a link into the group ("read this!") | `message` (+ rel.) | The commentary is speech; the shared bookmark/article stays an artifact, linked. | | Check-in, private location diary | `note` (+ `location`) | A journal entry with coordinates — addressed to no one. | | "I'm at the café — come join me" (group) | `message` (+ `location`) | Speech; the `location` association is the invariant across every check-in contract. | -| Foursquare-style public check-in | _deferred (#15)_ | Broadcast speech — `post` + `location` once the fourth contract lands. | -| Social media post | _deferred (#15)_ | A fourth contract — public broadcast — see below. | +| Foursquare-style public check-in | _not yet in the commons_ | Broadcast speech — `post` + `location` once the fourth contract lands. | +| Social media post | _not yet in the commons_ | A fourth contract — public broadcast — see below. | ## Comments are messages; marginalia are notes @@ -108,16 +108,17 @@ utterance — _their_ note is our `post`, not our `note`.) `post` must be its own type rather than a `message` with `{ access: 'public' }`, for four mechanical reasons, not just taxonomy: -1. **Sync blast radius.** Outbound bridges map _types_ to external shapes (#15's - `lexiconId` maps a type to an ATProto `$type`), so the type is the sync boundary. +1. **Sync blast radius.** Outbound bridges map _types_ to external shapes — a bridge's + own translation table maps a typeId to an ATProto `$type` — so the type is the sync + boundary. "Everything of type `post` is meant for the world" is an invariant a bridge can enforce; "messages whose permissions happen to be public" puts a group's private thread one permission bug away from the public firehose. 2. **Different threading fabric.** A message thread is `parentId` — within-stack, one trust domain, one indexed query. A post's conversation is inherently cross-stack: a reply lives in the replier's stack, referencing a record in someone else's, which - `parentId` cannot express and #16's `target` union - (`{ scope: 'internal', recordId, stackUrl }` / `{ scope: 'external', id, ns }`) + `parentId` cannot express and a relationship's `target` union + (`{ scope: 'record', recordId, stackUrl }` / `{ scope: 'external', ns, id }`) exists to express. Posts-as-messages would hand board apps threads whose parents they structurally cannot traverse. 3. **Different deletion physics.** Inside a stack, recoverability is real: "anything a @@ -127,24 +128,26 @@ four mechanical reasons, not just taxonomy: 4. **Different authorship requirements.** In-stack, `entityId` means author because the stack is a trust domain. A broadcast utterance travels _without_ its stack, so authorship must be self-certifying — the DID identity model used for `entityId` - generally, surfaced in #15's revised `externalIds` on `EntityContent`. `message` - needs none of it; `post` can't exist without it. - -The dependency chain is therefore: **#16** (cross-stack/cross-protocol reference -fabric, plus its `relatedTo`/capability follow-up so external references are -queryable) → **#15 as revised** (protocol-neutral core hooks, building on the -self-certifying DID identity model; ATProto-specific machinery in `adapter-atproto`) → -a `post@1` proposal here, as the _protocol-neutral_ broadcast -utterance: the canonical copy lives in your stack; bridges syndicate it -(`adapter-atproto` maps it to `app.bsky.feed.post`, an ActivityPub bridge to a `Note`) -and replies come home as external-target relationships. That is the IndieWeb's POSSE -pattern — publish on your own site, syndicate elsewhere — with real primitives -underneath: Bluesky and Mastodon become views of a record you own. - -One forward-compatibility note: `message`'s quote-reply convention uses today's flat -relationship shape (`recordId`). Commons labels (`reply-to`, `about`, `location`, -`series`, …) are orthogonal to #16's `target` union and ride on it unchanged — no -commons redesign is implied by that RFC landing. + generally, plus the `alias` relationships that resolve a foreign identifier back to + a known entity. `message` needs none of it; `post` can't exist without it. + +**Where the work sits.** Core carries the reference fabric a broadcast utterance +rests on: relationship targets that name a record in someone else's stack or an +identifier in another protocol, and a `relatedTo` filter that queries them. A +_bridge_ carries the rest — its typeId → `$type` translation table, and the content +addressing and tombstone machinery that describes a copy rather than the record it was +made from. Those are separate tracks from the content type: a `post` with no replies +and no bridge is fully expressible with what core provides. + +The shape this enables is the IndieWeb's POSSE pattern — publish on your own site, +syndicate elsewhere — with real primitives underneath: the canonical copy lives in your +stack, a bridge stamps a `syndicated-to` relationship for each copy it publishes, and +replies come home as external-target relationships. Bluesky and Mastodon are views of a +record you own. + +Commons labels (`reply-to`, `about`, `location`, `series`, …) are orthogonal to a +target: a label says what a reference means, a target says which identifier space it +lives in, and any label rides on any arm. ## On the names @@ -174,7 +177,7 @@ arrives with fediverse reflexes: Microformats never put a type name on the wire (post-type is discovered from properties), so there is no IndieWeb wire collision at all. The AS2 collision surfaces exactly once — a future ActivityPub bridge maps `post` → AS2 `Note` in its - translation table, the same mechanism as #15's `lexiconId`. + translation table, which is where every such mapping belongs. - **The word was never stable anyway.** Facebook "Notes" was a _long-form articles_ feature — a third, opposite usage. There is no uncontested name to find; the defense is precise contracts here and explicit mappings at the bridges. diff --git a/docs/spec/access-control.md b/docs/spec/access-control.md index c361d07..cd55826 100644 --- a/docs/spec/access-control.md +++ b/docs/spec/access-control.md @@ -150,7 +150,7 @@ A revocation is a soft delete like any other mutation — the owner can `undelet - **The grantee may be an app**: `granteeEntityId` is a DID, and an app that holds its own key has one — so granting an installed app the types it needs is the existing model applied, not new machinery (see [App](./identity.md#app)). When such an app acts for a person, the `-own`/`-any` distinction on _its_ grant collapses to the bare verb; see [Delegation](#delegation-principal-and-subject). - **Group-targeted grants match any roster role.** A `granteeGroupId` grant is satisfied by any entity holding a `member` or `admin` association on the named `_group` Record — unlike record-level `access: 'group'` permissions, there's no `role: 'admin'` narrowing on the grant side; a set of grantees is undifferentiated by role, so the narrower record-level shape doesn't carry over. `granteeEntityId` and `granteeGroupId` are mutually exclusive on grants written through `grant()`; a `_grant` Record naming both (only reachable by writing around it) requires both to be satisfied, consistent with the refuse-again-at-evaluation posture above. The named Record must be in the `_group` family: any Record's `relationship` associations would otherwise serve as a roster, and a group migrated out of the family would keep resolving after it had stopped being a group. - **Group-targeted grants do not count on the principal's side** of a delegated request, for the same reason default grants don't (see [Delegation](#delegation-principal-and-subject)) — one step removed. A `_group` roster is editable by any of its admins, not only by the stack owner, so a grant reaching a principal through a roster would let someone other than the owner name an app to a type the owner never named it to. The rule is about **how the authority arrived, not who holds it**, which is what makes it enforceable: a roster entry is an opaque DID and an `_app` Record's `did` is optional, so nothing can reliably tell an app's DID from a person's. An owner who means to grant an app names it directly, one grant at a time — the same shape as any other capability system that has to name software. Group grants still apply to the **subject** under delegation; only the principal half refuses them. -- **A grant target must name someone.** `grant()`, `revoke()` and `listGrants()` reject an empty `entityId` or an empty or absent `groupId` with `StackQueryError`. `null` is the only way to say "default grant": both an empty string and an absent field are falsy, so a target that names nobody would otherwise be stored as — and evaluated as — a grant to every authenticated entity. Evaluation refuses the same shape again, so a `_grant` Record carrying an empty `granteeEntityId` or `granteeGroupId` confers nothing however it came to exist. The `groupId` itself is **not** format-checked: it is a reference to an existing Record, like `parentId` or an association's `recordId`, and one that resolves to nothing simply denies. +- **A grant target must name someone.** `grant()`, `revoke()` and `listGrants()` reject an empty `entityId` or an empty or absent `groupId` with `StackQueryError`. `null` is the only way to say "default grant": both an empty string and an absent field are falsy, so a target that names nobody would otherwise be stored as — and evaluated as — a grant to every authenticated entity. Evaluation refuses the same shape again, so a `_grant` Record carrying an empty `granteeEntityId` or `granteeGroupId` confers nothing however it came to exist. The `groupId` itself is **not** format-checked: it is a reference to an existing Record, like `parentId` or a relationship association's `record` target, and one that resolves to nothing simply denies. - **Group roster resolution is memoized per operation.** Resolving `granteeGroupId` re-fetches the `_group` Record the same way `access: 'group'` permission resolution does (walking `relationship` associations), so it costs the same per-group lookup. The resolved roles are cached for the lifetime of one operation and threaded alongside `prefetchedGrants` — exactly the lifetime that has — so a `query()` examining many Records resolves a given roster once instead of once per candidate. Deliberately **not** cached for the life of a `ScopedStack`: `asEntity()`/`forSession()` return an object a caller may hold for as long as it likes, and a cache outliving the operation would let removal from a group go unnoticed by that instance. Revocation is the direction an authorization cache must never fail in. **Granting a group grants everyone its admins ever add.** A `_group` roster is managed by the stack owner _and_ by any entity holding an `admin` association on it, and an admin may appoint further admins. So a group-targeted grant is a standing delegation, not a fixed list: whoever holds `admin` on that group decides, from then on, who the grant reaches. This is what delegating group management means, and it is bounded in two ways — the owner outranks the roster, so ownership can never be locked out of a group and pruning is always available; and roster-derived authority stops at the principal boundary (above), so it can never reach an app acting for someone. An owner who wants a roster only they can change appoints no other admins: a group begins with exactly one admin, its creator, and plain members hold no roster authority at all. Where one group would need two levels of trust, use two groups. @@ -256,9 +256,10 @@ Two consequences worth stating plainly rather than leaving to be discovered: A `create` grant on a type authorizes writing Records of that type — it does not, by itself, authorize referencing arbitrary other Records or files through that Record. `ScopedStack.create()` and `ScopedStack.associate()` both additionally check that the requester may create the specific reference being written, since a reference elsewhere confers access (an `attachment` association or file-ref content field makes the referenced file downloadable via `getAttachment()`): - **`attachment` associations and file-ref content fields** require file access: the requester is the owner, uploaded the file themselves (holds an `_attachment@1` Record for it), or can already read some Record referencing it. This is exactly `getAttachment()`'s own access rule — reference creation requires what reference possession would grant. -- **`relationship` associations and `parentId`** require read access to the target Record. +- **`parentId`, and a `relationship` whose target is a Record in this stack** (`{ scope: 'record' }` naming no `stackUrl`), require read access to that Record. The gate tests `stackUrl` for a value rather than for presence, because absent and empty are one target everywhere else — storage, association identity, and the filter all read both as this stack — so both spellings meet the same check. This gate runs before the target's shape is validated, so a reference is refused for the access it names rather than reporting what was wrong with it. - **`tag` associations** carry no reference and are never gated. -- **`_group` roster associations are exempt** from the `relationship` check — a roster association's `recordId` names an Entity, not a readable Record (see [Group](./identity.md#group)), and roster mutation is already gated by the stricter admin-or-owner rule there. +- **The other relationship target arms are never gated** — a `record` target carrying a `stackUrl`, an `entity` target, and an `external` target alike. This is not a gap in the check but the absence of anything for it to protect: the gate exists so that creating a reference cannot convey access to, or confirm the existence of, a Record the requester may not read, and none of these three names a Record in this stack. Core never dereferences them, so no access flows through one, and an accepted write reports back only what the requester already supplied. Gating a target in another stack would additionally require dereferencing that stack at write time, which core does not do — and a stack cannot even recognize its own URL, since `_config` holds no `stackUrl`. +- **`_group` roster associations are exempt** from the `relationship` check — a roster association names an Entity rather than a readable Record (see [Group](./identity.md#group)), and roster mutation is already gated by the stricter admin-or-owner rule there. A missing target and an existing-but-inaccessible one **always produce the same `StackPermissionError`**, with no distinguishing detail — otherwise the check itself becomes a confirmation oracle (e.g. for a guessed file hash: content-addressed `fileId`s mean a successful attach-then-read round-trip would otherwise confirm the stack holds those exact bytes). On `update()`, only file-ref fields actually present in the patch are checked — untouched fields carry no new reference. diff --git a/docs/spec/data-model.md b/docs/spec/data-model.md index 0e7a7e2..8d310d2 100644 --- a/docs/spec/data-model.md +++ b/docs/spec/data-model.md @@ -73,18 +73,35 @@ Tags, attachments, and relationships are unified under a single **Association** type Association = | { kind: 'tag'; label: string } | { kind: 'attachment'; label: string; fileId: string } - | { kind: 'relationship'; label: string; recordId: string }; + | { kind: 'relationship'; label: string; target: RelationshipTarget }; + +type RelationshipTarget = + | { scope: 'record'; recordId: string; stackUrl?: string } + | { scope: 'entity'; entityId: string } + | { scope: 'external'; ns: string; id: string }; ``` **Examples:** - A `contact` type uses `{ kind: "attachment", label: "avatar", fileId: "..." }` as a profile picture. -- A `tweet` type uses `{ kind: "relationship", label: "reply-to", recordId: "..." }` to reference another tweet. +- A `tweet` type uses `{ kind: "relationship", label: "reply-to", target: { scope: "record", recordId: "..." } }` to reference another tweet. - Any record can use `{ kind: "tag", label: "starred" }` for user-defined labels. `parentId` is a separate native field (not an Association) because hierarchical containment is fundamental enough to warrant indexing at the library level. Associations are for metadata and cross-references. -**Reference creation is gated on `ScopedStack`:** an `attachment` association or file-ref content field requires file access, and a `relationship` association or `parentId` requires read access to the target — see [Reference-creation gating](./access-control.md#reference-creation-gating). Plain `Stack` is unscoped and does not apply this. +### Relationship targets + +A relationship's `scope` names **which identifier space its value belongs to**. The three are not interchangeable: the same string can be a Record ID in one and a DID in another, and matching across them would make a group roster look like a record reference. + +- **`record`** — a Record. `recordId` is unique within one stack only, so a reference to a Record in a _different_ stack carries that stack's `stackUrl` alongside it. An absent `stackUrl` means this stack; it is not a wildcard, and a filter that omits it does not match a target that carries one. +- **`entity`** — a "who", as a DID. This is what [group rosters](./identity.md#group) are made of: membership names identities, which mean the same thing in every stack, rather than the local Records that happen to describe them. +- **`external`** — something outside the stack entirely. `ns` names the scheme that interprets `id` — `"atproto"`, `"activitypub"`, `"email"`, `"url"`, anything — and is part of the association's identity, so the same `id` under two namespaces is two associations. Haverstack expresses the reference; the app or adapter interprets it. No protocol is privileged, and nothing in core dereferences one. + +**A target names exactly one thing, exactly one way.** Every part of a target that names something must be a non-empty string, and a target whose `scope` is outside these three is rejected with `StackValidationError` — a discriminated union is a compile-time promise, and a Record arriving from a request body or a foreign server has made no such promise. Where absence is meaningful it is the only way to say so: this stack is named by omitting `stackUrl`, never by sending an empty one, and a whole namespace by omitting an external `id`. An empty string would claim a name while carrying none, and be stored and matched as though it were absent. + +Association identity is `(kind, label)` plus the payload: `fileId` for an attachment, and the whole target for a relationship. Two relationships differing in any target field are two associations, and `dissociate()` removes only the one it names exactly. + +**Reference creation is gated on `ScopedStack`:** an `attachment` association or file-ref content field requires file access, and a `relationship` naming a Record _in this stack_ — plus `parentId` — requires read access to the target. The other target arms are ungated; see [Reference-creation gating](./access-control.md#reference-creation-gating) for why that is safe rather than a hole. Plain `Stack` is unscoped and does not apply this. ## Types @@ -256,7 +273,9 @@ type Filter = { // Association filters tags?: string[]; // records that have ALL of these tags hasAttachment?: string; // records with an attachment of this label - relatedTo?: { recordId: string; label?: string }; + relatedTo?: + | { label: string; target?: RelationshipTargetPattern } + | { label?: string; target: RelationshipTargetPattern }; attachmentFileId?: string; // records that reference a specific attachment file ID, via an `attachment` Association or a top-level `file-ref` content field // Content fields (exact match on top-level keys) @@ -272,6 +291,10 @@ type DateRange = { }; ``` +**`relatedTo` names a label, a target, or both — never neither**, and a filter naming neither is rejected with `StackQueryError` rather than widening to every Record carrying a relationship. Its target follows the same naming rules as a stored one (above), checked the same way. A `RelationshipTargetPattern` is the association's own target shape with the parts a query may leave open. Each half is a pattern: a bare `label` matches every target under it; an `external` target with no `id` matches the whole namespace, which is how a syndication tool asks what it has already published. A `record` target with no `stackUrl` matches only local targets — absence names this stack rather than acting as a wildcard, so a Record referenced in someone else's stack is reachable only by naming that stack. Matching is exact within a scope and never across scopes. + +**"Carries any relationship at all" is deliberately not expressible.** `tags` and `hasAttachment` have no match-any form either, so a relationship one would be the odd exception rather than a missing convenience — and a filter that can encode to nothing is a filter that can silently widen a query when it crosses the wire. The type refuses the empty filter rather than defining it, and `Stack.query()` refuses it again at runtime, where a filter decoded from query parameters is a plain object the type never saw. + **A `content` filter value of `null` means "the field is absent or stored as `null`"** — not "match nothing." Plain equality (SQL `= NULL`, or JS `===` against a possibly-absent key) is never true for a missing field, which would make `{ content: { x: null } }` silently return an empty result. Every adapter, including test doubles, implements `IS NULL` / missing-path semantics for a `null` filter value: it matches a record whose content omits the key entirely and one that stores a literal `null` alike, since from the caller's side both mean "no value here." `baseId` matches every version of a type family — resolved against registered Types (via `listTypes()`), not string-parsed from `typeId`, so it works regardless of which versions happen to exist. This is what keeps `typeId`-filtered queries from silently missing not-yet-migrated older-version records under [explicit, owner-driven migration](#type-migrations): filter by `baseId` to see the whole family, or `typeId` for an exact version. Given both, they intersect. `Stack.query()` resolves `baseId` client-side before dispatching to the adapter — adapters and the wire protocol only ever see a concrete `typeId` set. An unknown `baseId` returns an empty result set rather than throwing. diff --git a/docs/spec/identity.md b/docs/spec/identity.md index 4e5e809..f357420 100644 --- a/docs/spec/identity.md +++ b/docs/spec/identity.md @@ -150,11 +150,11 @@ A group's `handle` is a label on the same terms as an entity's — unenforced, n **Membership** is expressed via associations on the `_group` Record, using the existing Association model: ```ts -{ kind: "relationship", label: "member", recordId: "" } -{ kind: "relationship", label: "admin", recordId: "" } +{ kind: "relationship", label: "member", target: { scope: "entity", entityId: "" } } +{ kind: "relationship", label: "admin", target: { scope: "entity", entityId: "" } } ``` -(`recordId` here names an Entity by DID, not a Record within the target stack — the field is reused rather than duplicated.) +The `entity` scope is what makes a roster a roster: membership names an identity, not a Record. A roster entry carrying a `record` target confers nothing, even if its `recordId` happens to equal a member's DID — see [Relationship targets](./data-model.md#relationship-targets). This gives roles for free via association labels, and membership is queryable and versioned like any other Record data. There is no role hierarchy beyond this single distinction — matching the scale a Group actually serves (a small, cohesive set of Entities), not a general-purpose permissions system: diff --git a/docs/spec/wire-format.md b/docs/spec/wire-format.md index 15f98f1..7c856b4 100644 --- a/docs/spec/wire-format.md +++ b/docs/spec/wire-format.md @@ -235,8 +235,12 @@ This is what lets a client report a mutation's outcome without a second read, an ?tag= (repeatable: ?tag=starred&tag=important) ?hasAttachment= ?attachmentFileId= -?relatedTo= -?relatedToLabel= (only meaningful alongside ?relatedTo; narrows to that label) +?relatedTo= (a Record id — the `record` scope) +?relatedToStack= (only alongside ?relatedTo; that Record's stack URL) +?relatedToEntity= (a DID — the `entity` scope) +?relatedToNs= (a namespace — the `external` scope) +?relatedToId= (only alongside ?relatedToNs; omit to match the whole namespace) +?relatedToLabel= (narrows any of the above to one label; valid alone) ?search= ?sort=createdAt|updatedAt|version ?direction=asc|desc @@ -245,6 +249,8 @@ This is what lets a client report a mutation's outcome without a second read, an ?includeDeleted= ``` +**The relationship filter's scope is implied by which parameters appear**, and the three sets are mutually exclusive: a request mixing `relatedTo`, `relatedToEntity` or `relatedToNs` is rejected with `400`, since there is no correct way to guess which the caller meant. At least one of these parameters is always present when the filter is used — [`relatedTo` names a label, a target, or both](./data-model.md#filter), never neither — so the filter cannot encode to an empty query string and silently widen the query. Omitting `relatedToStack` means the target has no `stackUrl`; the server MUST NOT treat it as a wildcard matching targets that carry one. `relatedToStack` MUST be omitted rather than sent empty — this stack is named one way — and a server MUST reject an empty one with `400` rather than reading it as either a local target or a wildcard. The same holds for `relatedToId`: omit it to match a whole namespace. + `GET /records` covers all native field queries and is usable from a browser or simple HTTP client without a JSON body. `POST /records/query` is a superset — it accepts the full `Query` object as a JSON body and additionally supports `content` field filtering. A server that declares `contentFieldQuery: false` in discovery does not support the POST query endpoint. **Filters gated by a capability fail loudly, not silently.** A `content` filter has no representation in `GET /records`' query params, and `search` behaves however the server does with an unsupported param — so `APIAdapter` checks `capabilities.contentFieldQuery`/`capabilities.fullTextSearch` before dispatching and throws `APIAdapterCapabilityError` locally, without sending a request, when the corresponding filter is used against a server that hasn't declared the capability. The alternative — degrading to an unfiltered or partially filtered result presented as the requested query — is worse than an error for anything that trusts the filter (dedup checks, existence checks, selection-sensitive logic). @@ -356,7 +362,25 @@ Both endpoints accept the same optional `If-Match` precondition described under "associations": [ { "kind": "tag", "label": "starred" }, { "kind": "attachment", "label": "avatar", "fileId": "abc123" }, - { "kind": "relationship", "label": "reply-to", "recordId": "xyz789" } + { + "kind": "relationship", + "label": "reply-to", + "target": { "scope": "record", "recordId": "xyz789" } + }, + { + "kind": "relationship", + "label": "author", + "target": { "scope": "entity", "entityId": "did:key:z6Mk..." } + }, + { + "kind": "relationship", + "label": "syndicated-to", + "target": { + "scope": "external", + "ns": "atproto", + "id": "at://did:plc:abc/app.bsky.feed.post/3k4" + } + } ] } ``` diff --git a/packages/adapter-api/src/index.ts b/packages/adapter-api/src/index.ts index bea2b9c..c9fc939 100644 --- a/packages/adapter-api/src/index.ts +++ b/packages/adapter-api/src/index.ts @@ -34,7 +34,7 @@ import type { ChangeFilter, RecordChange, } from '@haverstack/core'; -import { assertQueryCapabilities } from '@haverstack/core/adapter'; +import { assertQueryCapabilities, assertValidRelatedTo } from '@haverstack/core/adapter'; import type { AdapterCapabilities, SubscribeChangesOptions } from '@haverstack/core/adapter'; import { buildAuthChallengePayload, base64urlEncode } from '@haverstack/core/wire'; import type { DidCredential } from '@haverstack/core/wire'; @@ -325,8 +325,21 @@ const buildQueryParams = (query: StackQuery): URLSearchParams => { if (f.hasAttachment) p.set('hasAttachment', f.hasAttachment); if (f.attachmentFileId) p.set('attachmentFileId', f.attachmentFileId); if (f.relatedTo) { - p.set('relatedTo', f.relatedTo.recordId); - if (f.relatedTo.label) p.set('relatedToLabel', f.relatedTo.label); + // The scope is implied by which qualifier appears, and the type + // guarantees at least one of these branches sets something — so the + // filter can never encode to nothing and silently widen the query. + // The server rejects a mix of scopes. + const t = f.relatedTo.target; + if (t?.scope === 'record') { + p.set('relatedTo', t.recordId); + if (t.stackUrl !== undefined) p.set('relatedToStack', t.stackUrl); + } else if (t?.scope === 'entity') { + p.set('relatedToEntity', t.entityId); + } else if (t?.scope === 'external') { + p.set('relatedToNs', t.ns); + if (t.id !== undefined) p.set('relatedToId', t.id); + } + if (f.relatedTo.label !== undefined) p.set('relatedToLabel', f.relatedTo.label); } if (f.search) p.set('search', f.search); if (f.includeDeleted) p.set('includeDeleted', 'true'); @@ -923,6 +936,11 @@ export class APIAdapter implements StackAdapter { : 'contentFieldQuery'; throw new APIAdapterCapabilityError(capability, err.message); } + // A malformed relationship filter is a caller error, not a missing + // capability, so this one travels as the StackQueryError it is — + // refused here rather than encoded into query params a server would + // have to reject. + assertValidRelatedTo(query.filter?.relatedTo); let raw: WireQueryResponse; if (this.capabilities.contentFieldQuery) { diff --git a/packages/adapter-api/tests/api.test.ts b/packages/adapter-api/tests/api.test.ts index 67abace..ab94b39 100644 --- a/packages/adapter-api/tests/api.test.ts +++ b/packages/adapter-api/tests/api.test.ts @@ -947,7 +947,9 @@ describe('queryRecords', () => { }; const adapter = await openAdapter(limitedDiscovery); mockFetch.mockResolvedValueOnce(jsonResponse(queryEnvelope)); - await adapter.queryRecords({ filter: { relatedTo: { recordId: 'rec-1', label: 'author' } } }); + await adapter.queryRecords({ + filter: { relatedTo: { label: 'author', target: { scope: 'record', recordId: 'rec-1' } } }, + }); const [url] = mockFetch.mock.lastCall as [string]; expect(url).toContain('relatedTo=rec-1'); expect(url).toContain('relatedToLabel=author'); @@ -960,12 +962,35 @@ describe('queryRecords', () => { }; const adapter = await openAdapter(limitedDiscovery); mockFetch.mockResolvedValueOnce(jsonResponse(queryEnvelope)); - await adapter.queryRecords({ filter: { relatedTo: { recordId: 'rec-1' } } }); + await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'record', recordId: 'rec-1' } } }, + }); const [url] = mockFetch.mock.lastCall as [string]; expect(url).toContain('relatedTo=rec-1'); expect(url).not.toContain('relatedToLabel'); }); + // A malformed relationship filter is a caller error, not a missing + // capability, so it travels as StackQueryError — and is refused before a + // request the server would only have to reject goes out. + test('refuses a relatedTo naming neither a label nor a target without sending', async () => { + const adapter = await openAdapter(); + await expect(adapter.queryRecords({ filter: { relatedTo: {} as never } })).rejects.toThrow( + StackQueryError, + ); + expect(mockFetch).toHaveBeenCalledTimes(1); // only the discovery call — no request sent + }); + + test('refuses an empty stackUrl rather than encoding relatedToStack=', async () => { + const adapter = await openAdapter(); + await expect( + adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'record', recordId: 'rec-1', stackUrl: '' } } }, + }), + ).rejects.toThrow(StackQueryError); + expect(mockFetch).toHaveBeenCalledTimes(1); // only the discovery call — no request sent + }); + test('throws APIAdapterCapabilityError for filter.content without contentFieldQuery', async () => { const limitedDiscovery = { ...DISCOVERY, @@ -1053,7 +1078,11 @@ describe('dissociate', () => { // ------------------------------------------------------- describe('a mutation that bumps a version must answer with a Record', () => { - const ASSOC: Association = { kind: 'relationship', label: 'author', recordId: 'rec-other' }; + const ASSOC: Association = { + kind: 'relationship', + label: 'author', + target: { scope: 'record', recordId: 'rec-other' }, + }; test('associate reports an empty body as a protocol error', async () => { const adapter = await openAdapter(); diff --git a/packages/adapter-api/tests/conformance.test.ts b/packages/adapter-api/tests/conformance.test.ts index 4245a1b..9b33feb 100644 --- a/packages/adapter-api/tests/conformance.test.ts +++ b/packages/adapter-api/tests/conformance.test.ts @@ -319,6 +319,41 @@ describe('queryRecords fixtures', () => { expect(result.total).toBeNull(); }); + // The scope is implied by which parameters appear, so a client that + // dropped one would silently widen the query rather than fail. + for (const [name, filter] of [ + [ + 'query-related-to-record-target', + { + relatedTo: { + label: 'series', + target: { scope: 'record' as const, recordId: '1hk153x00001' }, + }, + }, + ], + [ + 'query-related-to-entity-target', + { relatedTo: { target: { scope: 'entity' as const, entityId: 'did:key:z6MkAlice' } } }, + ], + [ + 'query-related-to-external-namespace', + { relatedTo: { target: { scope: 'external' as const, ns: 'atproto' } } }, + ], + ] as const) { + test(name, async () => { + const fixture = queryRecordsFixtures.find((f) => f.name === name)!; + mockFetch.mockResolvedValueOnce(jsonResponse(NATIVE_ONLY_DISCOVERY)); + const adapter = await APIAdapter.open({ url: BASE_URL }); + mockFetch.mockResolvedValueOnce(jsonResponse(fixture.responseBody, fixture.responseStatus)); + + await adapter.queryRecords({ filter }); + + const [url, init] = mockFetch.mock.lastCall as [string, RequestInit]; + expect(url).toBe(`${BASE_URL}${fixture.path}`); + expect(init.method).toBe('GET'); + }); + } + test('discards a total a non-conforming server populates anyway', async () => { const adapter = await openAdapter(); mockFetch.mockResolvedValueOnce(jsonResponse({ records: [], cursor: null, total: 142 }, 200)); diff --git a/packages/conformance-fixtures/src/index.ts b/packages/conformance-fixtures/src/index.ts index 8f81332..daea010 100644 --- a/packages/conformance-fixtures/src/index.ts +++ b/packages/conformance-fixtures/src/index.ts @@ -500,6 +500,48 @@ export const queryRecordsFixtures: ConformanceFixture< total: null, }, }, + { + name: 'query-related-to-record-target', + description: + 'A relationship filter naming a Record in this Stack travels as relatedTo, with ' + + "relatedToStack carrying another Stack's URL when the target has one. An absent " + + 'relatedToStack means this Stack — it is not a wildcard, so a server MUST NOT match a ' + + 'target that carries a stackUrl. This Stack is named that one way: a server MUST ' + + 'reject an empty relatedToStack with 400 rather than read it as local or as a ' + + 'wildcard, and likewise an empty relatedToId, which omission already expresses as the ' + + 'whole namespace. ' + + 'See docs/spec/wire-format.md § Query parameters.', + method: 'GET', + path: '/records?relatedTo=1hk153x00001&relatedToLabel=series', + responseStatus: 200, + responseBody: { records: [], cursor: null, total: null }, + }, + { + name: 'query-related-to-entity-target', + description: + 'A relationship filter naming an identity travels as relatedToEntity, distinct from ' + + 'relatedTo: a DID and a Record id are different reference spaces, and a server that ' + + 'matched one against the other would report group rosters as record references. ' + + 'See docs/spec/wire-format.md § Query parameters.', + method: 'GET', + path: '/records?relatedToEntity=did%3Akey%3Az6MkAlice', + responseStatus: 200, + responseBody: { records: [], cursor: null, total: null }, + }, + { + name: 'query-related-to-external-namespace', + description: + 'A relationship filter naming something outside the Stack travels as relatedToNs plus an ' + + 'optional relatedToId. Omitting relatedToId matches every target in the namespace, which ' + + 'is how a bridge asks what it has already syndicated. A server MUST reject a request ' + + 'mixing parameters from two scopes with 400, and can rely on at least one relatedTo ' + + 'parameter being present whenever the filter is used — the filter never encodes to ' + + 'nothing. See docs/spec/wire-format.md § Query parameters.', + method: 'GET', + path: '/records?relatedToNs=atproto', + responseStatus: 200, + responseBody: { records: [], cursor: null, total: null }, + }, ]; // ------------------------------------------------------- @@ -647,6 +689,41 @@ export const associateFixtures: ConformanceFixture, Wire associations: [{ kind: 'tag', label: 'starred' }], }, }, + { + name: 'associate-relationship-external-target', + description: + 'A relationship association carries its target as a discriminated union — the scope names ' + + 'which identifier space the value belongs to, so a server stores and returns it verbatim ' + + 'rather than flattening the arms into one id column. See docs/spec/data-model.md ' + + '§ Associations.', + method: 'POST', + path: '/records/1hk153x00001/associations', + requestBody: { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'at://did:plc:abc/app.bsky.feed.post/3k4' }, + }, + responseStatus: 200, + responseBody: { + id: '1hk153x00001', + typeId: 'com.example/note@1', + createdAt: '2024-01-01T00:00:00.000Z', + updatedAt: '2024-01-02T00:00:00.000Z', + content: { title: 'Hello', body: 'World' }, + version: 2, + associations: [ + { + kind: 'relationship', + label: 'syndicated-to', + target: { + scope: 'external', + ns: 'atproto', + id: 'at://did:plc:abc/app.bsky.feed.post/3k4', + }, + }, + ], + }, + }, ]; export const dissociateFixtures: ConformanceFixture, WireRecord>[] = [ diff --git a/packages/core/src/access.ts b/packages/core/src/access.ts index 4bc9267..f6ae56b 100644 --- a/packages/core/src/access.ts +++ b/packages/core/src/access.ts @@ -124,7 +124,11 @@ export function groupRoleFromAssociations( ): GroupRole | null { let role: GroupRole | null = null; for (const a of associations ?? []) { - if (a.kind === 'relationship' && a.recordId === entityId) { + if ( + a.kind === 'relationship' && + a.target.scope === 'entity' && + a.target.entityId === entityId + ) { if (a.label === 'admin') return 'admin'; if (a.label === 'member') role = 'member'; } diff --git a/packages/core/src/adapter-entry.ts b/packages/core/src/adapter-entry.ts index c721bd0..fb6542f 100644 --- a/packages/core/src/adapter-entry.ts +++ b/packages/core/src/adapter-entry.ts @@ -16,4 +16,4 @@ export type { SubscribeChangesOptions, } from './types.js'; export { combineAdapters } from './combine.js'; -export { assertQueryCapabilities, assertValidSort } from './stack.js'; +export { assertQueryCapabilities, assertValidSort, assertValidRelatedTo } from './stack.js'; diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 854bd10..d1b3c63 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -75,9 +75,15 @@ export type { TagAssociation, AttachmentAssociation, RelationshipAssociation, + RelationshipTarget, + RecordTarget, + EntityTarget, + ExternalTarget, Permission, StackQuery, RecordFilter, + RelatedToFilter, + RelationshipTargetPattern, QuerySort, QueryResult, DateRange, diff --git a/packages/core/src/stack.ts b/packages/core/src/stack.ts index 03da8a4..a16e5c4 100644 --- a/packages/core/src/stack.ts +++ b/packages/core/src/stack.ts @@ -50,6 +50,8 @@ import type { RecordFilter, QueryResult, Association, + RelationshipTarget, + RelationshipTargetPattern, Permission, Migration, MigrationFn, @@ -543,6 +545,88 @@ export function assertValidSort(sort: QuerySort | undefined): void { } } +/** The identifier spaces a relationship target may name. */ +const TARGET_SCOPES = new Set(['record', 'entity', 'external']); + +/** + * Collect what makes a relationship target malformed. Absence is + * meaningful on `stackUrl` and an external `id` — this stack, and the + * whole namespace — so every part that names something must be non-empty: + * an empty string stores and matches as though it were absent. + * See docs/spec/data-model.md § Relationship targets. + */ +function targetErrors( + target: RelationshipTarget | RelationshipTargetPattern, + path: string, + opts: { externalIdOptional?: boolean } = {}, +): ValidationError[] { + const fail = (message: string): ValidationError[] => [{ path, message }]; + if (!target || typeof target !== 'object') + return fail('A relationship target must be an object.'); + if (!TARGET_SCOPES.has(target.scope)) { + return fail( + `Unknown relationship target scope "${target.scope}": expected "record", "entity" or "external".`, + ); + } + if (target.scope === 'record') { + if (!target.recordId) return fail('A record target requires a non-empty recordId.'); + if (target.stackUrl !== undefined && !target.stackUrl) { + return fail("A record target's stackUrl must be non-empty; omit it to name this stack."); + } + return []; + } + if (target.scope === 'entity') { + return target.entityId ? [] : fail('An entity target requires a non-empty entityId.'); + } + if (!target.ns) return fail('An external target requires a non-empty ns.'); + if (target.id === undefined) { + return opts.externalIdOptional ? [] : fail('An external target requires an id.'); + } + return target.id ? [] : fail("An external target's id must be non-empty when present."); +} + +/** + * Reject a relationship target outside the closed set the types promise. + * A discriminated union is not a runtime guard — a server mapping a + * request body onto an association supplies raw JSON — and an + * unrecognized scope would otherwise be stored under the one arm that + * names a Record in this stack. See docs/spec/data-model.md + * § Relationship targets. + */ +function validateAssociation(association: Association, path = 'association'): ValidationError[] { + if (association?.kind !== 'relationship') return []; + return targetErrors(association.target, `${path}.target`); +} + +/** validateAssociation() over a create's `associations` array. */ +function validateAssociations( + associations: Association[] | undefined, + path = 'associations', +): ValidationError[] { + return (associations ?? []).flatMap((a, i) => validateAssociation(a, `${path}[${i}]`)); +} + +/** + * Reject a relationship filter that names neither a label nor a target, + * or whose target is malformed. `RelatedToFilter` promises one half is + * always present; without the runtime check a filter decoded from a + * request could arrive empty and match every record carrying any + * relationship. See docs/spec/data-model.md § Filter. + */ +export function assertValidRelatedTo(relatedTo: RecordFilter['relatedTo']): void { + if (!relatedTo) return; + if (relatedTo.label === undefined && relatedTo.target === undefined) { + throw new StackQueryError( + 'filter.relatedTo must name a label, a target, or both — "any relationship at all" is not a filter.', + ); + } + if (relatedTo.target === undefined) return; + const errors = targetErrors(relatedTo.target, 'filter.relatedTo.target', { + externalIdOptional: true, + }); + if (errors.length > 0) throw new StackQueryError(errors[0].message); +} + /** * Thrown when an attachment upload exceeds the adapter's declared * `maxAttachmentBytes` ceiling — checked client-side before any bytes are @@ -1241,6 +1325,7 @@ export class Stack implements StackClient { ...validateReservedKeys(content), ...validateContent(content, type.schema), ...validatePermissions(opts.permissions), + ...validateAssociations(opts.associations), ]; if (errors.length > 0) { throw new StackValidationError(errors); @@ -1423,6 +1508,8 @@ export class Stack implements StackClient { opts: IfVersionOptions & ActorOptions = {}, ): Promise { this.assertOpen(); + const errors = validateAssociation(association); + if (errors.length > 0) throw new StackValidationError(errors); const existing = await this.adapter.getRecord(id); if (!existing) { throw new StackNotFoundError(`Record not found: "${id}"`); @@ -1449,6 +1536,8 @@ export class Stack implements StackClient { opts: IfVersionOptions & ActorOptions = {}, ): Promise { this.assertOpen(); + const errors = validateAssociation(association); + if (errors.length > 0) throw new StackValidationError(errors); const existing = await this.adapter.getRecord(id); if (!existing) { throw new StackNotFoundError(`Record not found: "${id}"`); @@ -1575,6 +1664,7 @@ export class Stack implements StackClient { const { presentAt, filter, limit: rawLimit, ...rest } = query; assertQueryCapabilities(filter, this.adapter.capabilities); assertValidSort(query.sort); + assertValidRelatedTo(filter?.relatedTo); const limit = rawLimit !== undefined ? Math.min(rawLimit, MAX_QUERY_LIMIT) : undefined; const resolvedFilter = await this.resolveBaseIdFilter(filter); @@ -2649,15 +2739,28 @@ export class Stack implements StackClient { /** * Matches the SQLite adapter's association primary key (kind, label, - * file_id, related_id). + * file_id, related_scope, related_id, related_ns, related_stack). */ function associationEqual(a: Association, b: Association): boolean { if (a.kind !== b.kind || a.label !== b.label) return false; if (a.kind === 'attachment' && b.kind === 'attachment') return a.fileId === b.fileId; - if (a.kind === 'relationship' && b.kind === 'relationship') return a.recordId === b.recordId; + if (a.kind === 'relationship' && b.kind === 'relationship') { + return targetEqual(a.target, b.target); + } return true; } +/** Structural equality per target arm — what dissociate() matches on. */ +export function targetEqual(a: RelationshipTarget, b: RelationshipTarget): boolean { + if (a.scope !== b.scope) return false; + if (a.scope === 'record' && b.scope === 'record') { + return a.recordId === b.recordId && (a.stackUrl ?? '') === (b.stackUrl ?? ''); + } + if (a.scope === 'entity' && b.scope === 'entity') return a.entityId === b.entityId; + if (a.scope === 'external' && b.scope === 'external') return a.ns === b.ns && a.id === b.id; + return false; +} + function permissionEqual(a: Permission, b: Permission): boolean { if (a.access !== b.access) return false; if (a.access === 'public') return true; @@ -2687,13 +2790,23 @@ const MAX_QUERY_LIMIT = 1000; * if it's not already present. Used to bootstrap a `_group` record's first * admin at create time. */ -function stampGroupAdmin(associations: Association[] | undefined, creator: string): Association[] { +function stampGroupAdmin( + associations: Association[] | undefined, + creator: EntityId, +): Association[] { const list = associations ?? []; const alreadyAdmin = list.some( - (a) => a.kind === 'relationship' && a.label === 'admin' && a.recordId === creator, + (a) => + a.kind === 'relationship' && + a.label === 'admin' && + a.target.scope === 'entity' && + a.target.entityId === creator, ); if (alreadyAdmin) return list; - return [...list, { kind: 'relationship', label: 'admin', recordId: creator }]; + return [ + ...list, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: creator } }, + ]; } /** @@ -3263,16 +3376,27 @@ export class ScopedStack implements StackClient { /** * Reference-creation gate for one association: `attachment` requires - * file access, `relationship` read access to its target; `tag` is - * unchecked, and `_group` roster associations are gated by the stricter - * isGroupManager() instead. See docs/spec/access-control.md - * § Reference-creation gating. + * file access, and a `relationship` naming a record in this stack + * requires read access to it. `tag` is unchecked, and `_group` roster + * associations are gated by the stricter isGroupManager() instead. + * + * The other target arms are ungated because the gate's purpose — + * refusing a reference that would convey access to, or confirm the + * existence of, an unreadable record — has nothing to bite on: core + * never resolves them, so no access flows through one. + * See docs/spec/access-control.md § Reference-creation gating. */ private async requireAssociationAccess(typeId: TypeId, association: Association): Promise { if (association.kind === 'attachment') { if (!(await this.canAccessFile(association.fileId))) throw new StackPermissionError(); } else if (association.kind === 'relationship' && baseIdOf(typeId) !== SYSTEM_TYPES.GROUP) { - if (!(await this.canReadReferent(association.recordId))) throw new StackPermissionError(); + const { target } = association; + // `stackUrl` is tested for a value, not for presence: absent and + // empty are one target — storage, targetEqual() and the filter all + // read them as this stack — so a check on presence alone would + // leave one spelling of a local Record ungated. + if (target.scope !== 'record' || target.stackUrl) return; + if (!(await this.canReadReferent(target.recordId))) throw new StackPermissionError(); } } @@ -3375,6 +3499,7 @@ export class ScopedStack implements StackClient { */ async query(query: StackQuery = {}): Promise { assertValidSort(query.sort); + assertValidRelatedTo(query.filter?.relatedTo); const limit = Math.min(query.limit ?? DEFAULT_QUERY_LIMIT, MAX_QUERY_LIMIT); const records: StackRecord[] = []; const maxFetched = limit * 10; diff --git a/packages/core/src/testing.ts b/packages/core/src/testing.ts index 3b734a2..7247c29 100644 --- a/packages/core/src/testing.ts +++ b/packages/core/src/testing.ts @@ -7,6 +7,7 @@ import type { ActorOptions, StackQuery, QueryResult, + RecordFilter, Association, Permission, AdapterCapabilities, @@ -14,7 +15,12 @@ import type { } from './types.js'; import { SYSTEM_TYPES } from './types.js'; import { applyMergePatch } from './merge.js'; -import { StackVersionConflictError, StackConflictError, StackNotFoundError } from './stack.js'; +import { + StackVersionConflictError, + StackConflictError, + StackNotFoundError, + targetEqual, +} from './stack.js'; /** * In-memory StackAdapter with offset-based cursor pagination. Implements @@ -221,13 +227,8 @@ export class MemoryAdapter implements StackAdapter { ); } if (f.relatedTo) { - const { recordId, label } = f.relatedTo; - results = results.filter((r) => - (r.associations ?? []).some( - (a) => - a.kind === 'relationship' && a.recordId === recordId && (!label || a.label === label), - ), - ); + const relatedTo = f.relatedTo; + results = results.filter((r) => matchesRelatedTo(r.associations, relatedTo)); } // A `null` filter value means "field absent or null" — not "match // nothing". Plain `===` would miss an absent field; treat both as @@ -463,11 +464,40 @@ function withAssociations(record: StackRecord, associations: Association[]): Sta /** * Mirrors Stack's private associationEqual(): identity is (kind, label) plus - * fileId for attachments / recordId for relationships. + * fileId for attachments / the target for relationships. */ function associationEqual(a: Association, b: Association): boolean { if (a.kind !== b.kind || a.label !== b.label) return false; if (a.kind === 'attachment' && b.kind === 'attachment') return a.fileId === b.fileId; - if (a.kind === 'relationship' && b.kind === 'relationship') return a.recordId === b.recordId; + if (a.kind === 'relationship' && b.kind === 'relationship') { + return targetEqual(a.target, b.target); + } return true; } + +/** + * Mirrors sqlite-shared's relatedTo predicate: a bare label matches every + * target under it, and an external target with no `id` matches its whole + * namespace. + */ +function matchesRelatedTo( + associations: Association[] | undefined, + filter: NonNullable, +): boolean { + return (associations ?? []).some((a) => { + if (a.kind !== 'relationship') return false; + if (filter.label !== undefined && a.label !== filter.label) return false; + const want = filter.target; + if (!want) return true; + const got = a.target; + if (got.scope !== want.scope) return false; + if (want.scope === 'record' && got.scope === 'record') { + return got.recordId === want.recordId && (got.stackUrl ?? '') === (want.stackUrl ?? ''); + } + if (want.scope === 'entity' && got.scope === 'entity') return got.entityId === want.entityId; + if (want.scope === 'external' && got.scope === 'external') { + return got.ns === want.ns && (want.id === undefined || got.id === want.id); + } + return false; + }); +} diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index d688c7c..5ab464a 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -54,10 +54,46 @@ export type AttachmentAssociation = { fileId: FileId; }; +/** + * A Record in this stack, or in another one. `stackUrl` is what makes a + * RecordId meaningful outside the stack that minted it — absent means this + * stack. See docs/spec/data-model.md § Associations. + */ +export type RecordTarget = { + scope: 'record'; + recordId: RecordId; + stackUrl?: string; +}; + +/** + * A "who" — a DID, which means the same thing in every stack. Group + * rosters are the canonical use: membership names identities, not records. + * See docs/spec/identity.md § Group. + */ +export type EntityTarget = { + scope: 'entity'; + entityId: EntityId; +}; + +/** + * Something outside the stack entirely. `ns` names the scheme that + * interprets `id` — e.g. "atproto", "activitypub", "email", "url" — so an + * app selects on it rather than sniffing the identifier. + * See docs/spec/data-model.md § External targets. + */ +export type ExternalTarget = { + scope: 'external'; + ns: string; + id: string; +}; + +/** What a relationship association points at. */ +export type RelationshipTarget = RecordTarget | EntityTarget | ExternalTarget; + export type RelationshipAssociation = { kind: 'relationship'; label: string; - recordId: RecordId; + target: RelationshipTarget; }; export type Association = TagAssociation | AttachmentAssociation | RelationshipAssociation; @@ -353,6 +389,26 @@ export const SYSTEM_TYPES = { // Queries // ------------------------------------------------------- +/** + * A target pattern in `RecordFilter.relatedTo` — the association shape with + * the parts a query may leave open: an `external` target without `id` + * matches its whole namespace. + */ +export type RelationshipTargetPattern = + | { scope: 'record'; recordId: RecordId; stackUrl?: string } + | { scope: 'entity'; entityId: EntityId } + | { scope: 'external'; ns: string; id?: string }; + +/** + * A relationship query names a label, a target, or both — never neither. + * "Carries any relationship at all" is deliberately not expressible, in + * line with `tags` and `hasAttachment`, which likewise have no match-any + * form. See docs/spec/data-model.md § Filter. + */ +export type RelatedToFilter = + | { label: string; target?: RelationshipTargetPattern } + | { label?: string; target: RelationshipTargetPattern }; + export type DateRange = { before?: Date; after?: Date; @@ -379,10 +435,14 @@ export type RecordFilter = { // Association filters tags?: string[]; // Records that have ALL of these tags hasAttachment?: string; // Records with an attachment of this label - relatedTo?: { - recordId: RecordId; - label?: string; - }; + /** + * Records carrying a matching relationship association. Either half may + * be given alone and each is a pattern: a bare `label` matches every + * target under it, and an `external` target with no `id` matches the + * whole namespace. An absent `stackUrl` on a `record` target matches + * only local targets. See docs/spec/data-model.md § Filter. + */ + relatedTo?: RelatedToFilter; attachmentFileId?: FileId; // Records that reference this attachment file ID // Content fields — exact match on top-level keys (POST /query only) diff --git a/packages/core/tests/scoped-stack.test.ts b/packages/core/tests/scoped-stack.test.ts index c835beb..0677cba 100644 --- a/packages/core/tests/scoped-stack.test.ts +++ b/packages/core/tests/scoped-stack.test.ts @@ -95,7 +95,9 @@ describe('ScopedStack — read access', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); const record = await adapter.createRecord( @@ -106,11 +108,33 @@ describe('ScopedStack — read access', () => { expect((await stack.asEntity(MEMBER).get(record.id))?.id).toBe(record.id); }); + // Membership names an identity, not a record. A roster entry pointing at + // a record whose id happens to equal the DID confers nothing — the arms + // are what keep the two apart now that both hold plain strings. + test('a record-scoped roster entry does not confer membership', async () => { + const group = await adapter.createRecord( + makeRecord({ + typeId: '_group', + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'record', recordId: MEMBER } }, + ], + }), + ); + const record = await adapter.createRecord( + makeRecord({ + permissions: [{ access: 'group', groupId: group.id, read: true, write: false }], + }), + ); + expect(await stack.asEntity(MEMBER).get(record.id)).toBeNull(); + }); + test('non-member cannot read via a group read grant', async () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); const record = await adapter.createRecord( @@ -470,8 +494,8 @@ describe('ScopedStack — versions', () => { makeRecord({ typeId: '_group@1', associations: [ - { kind: 'relationship', label: 'admin', recordId: ADMIN }, - { kind: 'relationship', label: 'member', recordId: MEMBER }, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: ADMIN } }, + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, ], }), ); @@ -930,7 +954,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['create'], typeId: COMMENT }]); @@ -942,7 +968,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'admin', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: COMMENT }]); @@ -954,7 +982,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: COMMENT }]); @@ -966,7 +996,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: COMMENT }]); @@ -980,7 +1012,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['create'], typeId: COMMENT }]); @@ -998,7 +1032,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: COMMENT }]); @@ -1023,7 +1059,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: APP }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: APP } }, + ], }), ); // Both halves of the intersection are otherwise satisfied: the subject @@ -1058,7 +1096,9 @@ describe('ScopedStack — group-targeted grants', () => { const group = await adapter.createRecord( makeRecord({ typeId: '_group', - associations: [{ kind: 'relationship', label: 'member', recordId: MEMBER }], + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, + ], }), ); // Only the principal side refuses roster-derived authority; the subject @@ -1079,7 +1119,7 @@ describe('ScopedStack — group-targeted grants', () => { await stack.associate(notAGroup.id, { kind: 'relationship', label: 'member', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); await stack.grant({ groupId: notAGroup.id }, [{ actions: ['read-any'], typeId: COMMENT }]); @@ -1121,7 +1161,7 @@ describe('ScopedStack — group-targeted grants', () => { await stack.associate(group.id, { kind: 'relationship', label: 'member', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: COMMENT }]); const record = await stack.create(COMMENT, { text: 'secret' }); @@ -1132,7 +1172,7 @@ describe('ScopedStack — group-targeted grants', () => { await stack.dissociate(group.id, { kind: 'relationship', label: 'member', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); expect(await view.get(record.id)).toBeNull(); }); @@ -1279,7 +1319,7 @@ describe('ScopedStack — write implies read', () => { await stack.associate(group.id, { kind: 'relationship', label: 'member', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); const record = await recordWithHistory([ { access: 'group', groupId: group.id, read: false, write: true }, @@ -1945,8 +1985,8 @@ describe('ScopedStack — group role gating', () => { makeRecord({ typeId: '_group@1', associations: [ - { kind: 'relationship', label: 'admin', recordId: ADMIN }, - { kind: 'relationship', label: 'member', recordId: MEMBER }, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: ADMIN } }, + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, ], ...overrides, }), @@ -1962,14 +2002,18 @@ describe('ScopedStack — group role gating', () => { test('plain member cannot add or remove roster associations', async () => { const group = await makeGroup(); - const newMember: Association = { kind: 'relationship', label: 'member', recordId: STRANGER }; + const newMember: Association = { + kind: 'relationship', + label: 'member', + target: { scope: 'entity', entityId: STRANGER }, + }; await expect(stack.asEntity(MEMBER).associate(group.id, newMember)).rejects.toThrow( StackNotFoundError, ); const existingMember: Association = { kind: 'relationship', label: 'member', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }; await expect(stack.asEntity(MEMBER).dissociate(group.id, existingMember)).rejects.toThrow( StackNotFoundError, @@ -1987,7 +2031,11 @@ describe('ScopedStack — group role gating', () => { const updated = await stack.asEntity(ADMIN).update(group.id, { name: 'renamed' }); expect(updated.content.name).toBe('renamed'); - const newMember: Association = { kind: 'relationship', label: 'member', recordId: STRANGER }; + const newMember: Association = { + kind: 'relationship', + label: 'member', + target: { scope: 'entity', entityId: STRANGER }, + }; await stack.asEntity(ADMIN).associate(group.id, newMember); expect((await adapter.getRecord(group.id))?.associations).toContainEqual(newMember); @@ -2022,7 +2070,11 @@ describe('ScopedStack — group role gating', () => { await expect(stack.asEntity(STRANGER).update(group.id, { name: 'renamed' })).rejects.toThrow( StackPermissionError, ); - const newMember: Association = { kind: 'relationship', label: 'member', recordId: STRANGER }; + const newMember: Association = { + kind: 'relationship', + label: 'member', + target: { scope: 'entity', entityId: STRANGER }, + }; await expect(stack.asEntity(STRANGER).associate(group.id, newMember)).rejects.toThrow( StackPermissionError, ); @@ -2045,7 +2097,7 @@ describe('ScopedStack — group role gating', () => { expect(group.associations).toContainEqual({ kind: 'relationship', label: 'admin', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); const updated = await stack.asEntity(MEMBER).update(group.id, { name: 'renamed' }); @@ -2054,15 +2106,21 @@ describe('ScopedStack — group role gating', () => { test('create-time bootstrap does not duplicate an explicitly supplied admin association', async () => { await stack.grant(MEMBER, [{ actions: ['create'], typeId: '_group@1' }]); - const group = await stack - .asEntity(MEMBER) - .create( - '_group@1', - { name: 'New Group' }, - { associations: [{ kind: 'relationship', label: 'admin', recordId: MEMBER }] }, - ); + const group = await stack.asEntity(MEMBER).create( + '_group@1', + { name: 'New Group' }, + { + associations: [ + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: MEMBER } }, + ], + }, + ); const adminAssociations = (group.associations ?? []).filter( - (a) => a.kind === 'relationship' && a.label === 'admin' && a.recordId === MEMBER, + (a) => + a.kind === 'relationship' && + a.label === 'admin' && + a.target.scope === 'entity' && + a.target.entityId === MEMBER, ); expect(adminAssociations).toHaveLength(1); }); @@ -2073,7 +2131,11 @@ describe('ScopedStack — group role gating', () => { const group = await stack.asEntity(OWNER).create('_group@1', { name: 'New Group' }); expect(group.entityId).toBe(OWNER); const adminAssociations = (group.associations ?? []).filter( - (a) => a.kind === 'relationship' && a.label === 'admin' && a.recordId === OWNER, + (a) => + a.kind === 'relationship' && + a.label === 'admin' && + a.target.scope === 'entity' && + a.target.entityId === OWNER, ); expect(adminAssociations).toHaveLength(1); }); @@ -2089,8 +2151,12 @@ describe('Permission — group role restriction', () => { makeRecord({ typeId: '_group', associations: [ - { kind: 'relationship', label: 'admin', recordId: 'group-admin-2' }, - { kind: 'relationship', label: 'member', recordId: MEMBER }, + { + kind: 'relationship', + label: 'admin', + target: { scope: 'entity', entityId: 'group-admin-2' }, + }, + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, ], }), ); @@ -2111,8 +2177,8 @@ describe('Permission — group role restriction', () => { makeRecord({ typeId: '_group', associations: [ - { kind: 'relationship', label: 'admin', recordId: admin }, - { kind: 'relationship', label: 'member', recordId: MEMBER }, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: admin } }, + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: MEMBER } }, ], }), ); @@ -2425,24 +2491,119 @@ describe('ScopedStack.create — relationship association and parentId gating', COMMENT, { text: 'hi' }, { - associations: [{ kind: 'relationship', label: 'related', recordId: unreadableNote.id }], + associations: [ + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: unreadableNote.id }, + }, + ], }, ), ).rejects.toThrow(StackPermissionError); }); + // An absent and an empty stackUrl are one target everywhere else, so + // both spellings of a local Record meet the same gate — the reference is + // refused for the access it names, before its shape is judged. + test('a record target naming this stack with an empty stackUrl is gated', async () => { + await expect( + stack.asEntity(MEMBER).create( + COMMENT, + { text: 'hi' }, + { + associations: [ + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: unreadableNote.id, stackUrl: '' }, + }, + ], + }, + ), + ).rejects.toThrow(StackPermissionError); + }); + + // The gate refuses a reference that would convey access to, or confirm + // the existence of, an unreadable record. The other arms name nothing in + // this stack, so there is nothing for it to protect and no check to make. + test('a record target in another stack is not gated', async () => { + const record = await stack.asEntity(MEMBER).create( + COMMENT, + { text: 'hi' }, + { + associations: [ + { + kind: 'relationship', + label: 'reply-to', + target: { + scope: 'record', + recordId: unreadableNote.id, + stackUrl: 'https://alice.example/stack', + }, + }, + ], + }, + ); + expect(record.associations).toHaveLength(1); + }); + + test('an entity target is not gated', async () => { + const record = await stack.asEntity(MEMBER).create( + COMMENT, + { text: 'hi' }, + { + associations: [ + { + kind: 'relationship', + label: 'author', + target: { scope: 'entity', entityId: 'did:key:z6MkAlice' }, + }, + ], + }, + ); + expect(record.associations).toHaveLength(1); + }); + + test('an external target is not gated', async () => { + const record = await stack.asEntity(MEMBER).create( + COMMENT, + { text: 'hi' }, + { + associations: [ + { + kind: 'relationship', + label: 'syndicated-to', + target: { + scope: 'external', + ns: 'atproto', + id: 'at://did:plc:abc/app.bsky.feed.post/3k4', + }, + }, + ], + }, + ); + expect(record.associations).toHaveLength(1); + }); + test('relationship association targeting a readable record is allowed', async () => { const record = await stack.asEntity(MEMBER).create( COMMENT, { text: 'hi' }, { - associations: [{ kind: 'relationship', label: 'related', recordId: readableNote.id }], + associations: [ + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: readableNote.id }, + }, + ], }, ); expect(record.associations).toContainEqual({ kind: 'relationship', label: 'related', - recordId: readableNote.id, + target: { scope: 'record', recordId: readableNote.id }, }); }); @@ -2455,7 +2616,11 @@ describe('ScopedStack.create — relationship association and parentId gating', { text: 'hi' }, { associations: [ - { kind: 'relationship', label: 'related', recordId: 'nonexistent-record' }, + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: 'nonexistent-record' }, + }, ], }, ); @@ -2467,7 +2632,13 @@ describe('ScopedStack.create — relationship association and parentId gating', COMMENT, { text: 'hi' }, { - associations: [{ kind: 'relationship', label: 'related', recordId: unreadableNote.id }], + associations: [ + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: unreadableNote.id }, + }, + ], }, ); } catch (e) { @@ -2485,7 +2656,7 @@ describe('ScopedStack.create — relationship association and parentId gating', expect(group.associations).toContainEqual({ kind: 'relationship', label: 'admin', - recordId: MEMBER, + target: { scope: 'entity', entityId: MEMBER }, }); }); @@ -2506,7 +2677,13 @@ describe('ScopedStack.create — relationship association and parentId gating', { text: 'hi' }, { parentId: unreadableNote.id, - associations: [{ kind: 'relationship', label: 'related', recordId: unreadableNote.id }], + associations: [ + { + kind: 'relationship', + label: 'related', + target: { scope: 'record', recordId: unreadableNote.id }, + }, + ], }, ); expect(record.parentId).toBe(unreadableNote.id); @@ -2552,7 +2729,7 @@ describe('ScopedStack.associate — reference-creation gating', () => { stack.asEntity(MEMBER).associate(ownedRecord.id, { kind: 'relationship', label: 'related', - recordId: unreadableNote.id, + target: { scope: 'record', recordId: unreadableNote.id }, }), ).rejects.toThrow(StackPermissionError); }); @@ -2929,7 +3106,11 @@ describe('ScopedStack — delegation', () => { test('an app delegated for a group admin cannot manage the group', async () => { const group = await stack.create('_group@1', { name: 'Book Club' }); - await stack.associate(group.id, { kind: 'relationship', label: 'admin', recordId: MEMBER }); + await stack.associate(group.id, { + kind: 'relationship', + label: 'admin', + target: { scope: 'entity', entityId: MEMBER }, + }); expect(await stack.asEntity(MEMBER).update(group.id, { name: 'Renamed' })).toBeTruthy(); await expect( stack.asEntity(APP, { onBehalfOf: MEMBER }).update(group.id, { name: 'Hijacked' }), @@ -2999,7 +3180,11 @@ describe('ScopedStack — delegation', () => { ).rejects.toThrow(StackNotFoundError); // An admin subject reaches it, since both identities then manage it. - await stack.associate(group.id, { kind: 'relationship', label: 'admin', recordId: MEMBER }); + await stack.associate(group.id, { + kind: 'relationship', + label: 'admin', + target: { scope: 'entity', entityId: MEMBER }, + }); expect( await stack.asEntity(OWNER, { onBehalfOf: MEMBER }).update(group.id, { name: 'Renamed' }), ).toBeTruthy(); diff --git a/packages/core/tests/scoped-subscribe.test.ts b/packages/core/tests/scoped-subscribe.test.ts index 6079167..65b3fcc 100644 --- a/packages/core/tests/scoped-subscribe.test.ts +++ b/packages/core/tests/scoped-subscribe.test.ts @@ -194,7 +194,11 @@ describe('a revocation takes effect on the next event, not the next subscription const group = await stack.create( '_group@1', { name: 'Team' }, - { associations: [{ kind: 'relationship', label: 'member', recordId: READER }] }, + { + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: READER } }, + ], + }, ); // A grant naming the group, not the entity: reachability now depends on // the roster, which is the lookup a subscription caches. @@ -207,7 +211,11 @@ describe('a revocation takes effect on the next event, not the next subscription await settle(); expect(reader.seen).toHaveLength(1); - await stack.dissociate(group.id, { kind: 'relationship', label: 'member', recordId: READER }); + await stack.dissociate(group.id, { + kind: 'relationship', + label: 'member', + target: { scope: 'entity', entityId: READER }, + }); await stack.update(note.id, { text: 'after removal' }); await settle(); @@ -251,7 +259,11 @@ describe('a revocation takes effect on the next event, not the next subscription const group = await stack.create( '_group@1', { name: 'Team' }, - { associations: [{ kind: 'relationship', label: 'member', recordId: READER }] }, + { + associations: [ + { kind: 'relationship', label: 'member', target: { scope: 'entity', entityId: READER } }, + ], + }, ); const note = await stack.create( NOTE, @@ -265,7 +277,11 @@ describe('a revocation takes effect on the next event, not the next subscription await settle(); expect(reader.seen).toHaveLength(1); - await stack.dissociate(group.id, { kind: 'relationship', label: 'member', recordId: READER }); + await stack.dissociate(group.id, { + kind: 'relationship', + label: 'member', + target: { scope: 'entity', entityId: READER }, + }); await stack.update(note.id, { text: 'after removal' }); await settle(); diff --git a/packages/core/tests/stack.test.ts b/packages/core/tests/stack.test.ts index ba463bd..b9f65e9 100644 --- a/packages/core/tests/stack.test.ts +++ b/packages/core/tests/stack.test.ts @@ -17,7 +17,14 @@ import { generateId, crockford32Encode, IdGenerationError } from '../src/id.js'; import { InvalidDidError } from '../src/did.js'; import { MemoryAdapter, IncapableMemoryAdapter } from '../src/testing.js'; import { firstRecordedAttachment } from '../src/attachment-download.js'; -import type { AttachmentContent, BlobFileInfo, StackAdapter, StackRecord } from '../src/types.js'; +import type { + AttachmentContent, + BlobFileInfo, + RecordFilter, + RelationshipTarget, + StackAdapter, + StackRecord, +} from '../src/types.js'; // ------------------------------------------------------- // Test setup @@ -412,14 +419,14 @@ describe('create — _group admin bootstrap', () => { test('owner-created group via plain Stack.create stamps the owner as first admin', async () => { const group = await stack.create('_group@1', { name: 'New Group' }); expect(group.associations).toEqual([ - { kind: 'relationship', label: 'admin', recordId: 'owner-123' }, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: 'owner-123' } }, ]); }); test('stamps the supplied entityId, not the owner, when one is provided', async () => { const group = await stack.create('_group@1', { name: 'New Group' }, { entityId: 'other-456' }); expect(group.associations).toEqual([ - { kind: 'relationship', label: 'admin', recordId: 'other-456' }, + { kind: 'relationship', label: 'admin', target: { scope: 'entity', entityId: 'other-456' } }, ]); }); @@ -427,10 +434,22 @@ describe('create — _group admin bootstrap', () => { const group = await stack.create( '_group@1', { name: 'New Group' }, - { associations: [{ kind: 'relationship', label: 'admin', recordId: 'owner-123' }] }, + { + associations: [ + { + kind: 'relationship', + label: 'admin', + target: { scope: 'entity', entityId: 'owner-123' }, + }, + ], + }, ); const adminAssociations = (group.associations ?? []).filter( - (a) => a.kind === 'relationship' && a.label === 'admin' && a.recordId === 'owner-123', + (a) => + a.kind === 'relationship' && + a.label === 'admin' && + a.target.scope === 'entity' && + a.target.entityId === 'owner-123', ); expect(adminAssociations).toHaveLength(1); }); @@ -2367,7 +2386,7 @@ describe('listGrants', () => { await stack.associate(group.id, { kind: 'relationship', label: 'member', - recordId: 'entity-abc', + target: { scope: 'entity', entityId: 'entity-abc' }, }); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: NOTE_V1 }]); await stack.grant('entity-xyz', [{ actions: ['read-own', 'delete-own'], typeId: NOTE_V1 }]); @@ -2386,7 +2405,7 @@ describe('listGrants', () => { await stack.associate(notAGroup.id, { kind: 'relationship', label: 'member', - recordId: 'entity-abc', + target: { scope: 'entity', entityId: 'entity-abc' }, }); await stack.grant({ groupId: notAGroup.id }, [{ actions: ['read-any'], typeId: NOTE_V1 }]); @@ -2403,7 +2422,7 @@ describe('listGrants', () => { await stack.associate(group.id, { kind: 'relationship', label: 'member', - recordId: 'entity-xyz', + target: { scope: 'entity', entityId: 'entity-xyz' }, }); await stack.grant({ groupId: group.id }, [{ actions: ['read-any'], typeId: NOTE_V1 }]); @@ -3854,3 +3873,287 @@ describe('ungrantable families are refused at evaluation', () => { ).rejects.toThrow(StackPermissionError); }); }); + +// ------------------------------------------------------- +// Relationship targets +// ------------------------------------------------------- + +describe('relationship targets', () => { + // Storage, targetEqual() and the filter all read an absent and an empty + // stackUrl as this stack, so a target names this stack exactly one way. + // Every other part that names something is required for the same reason: + // an empty string would claim a name while carrying none. + test('a record target names this stack by omitting stackUrl, never by emptying it', async () => { + const note = await stack.create(NOTE_V1, { text: 'host' }); + await expect( + stack.associate(note.id, { + kind: 'relationship', + label: 'series', + target: { scope: 'record', recordId: 'somerecordid', stackUrl: '' }, + }), + ).rejects.toThrow(StackValidationError); + }); + + test('a target outside the three scopes is refused', async () => { + const note = await stack.create(NOTE_V1, { text: 'host' }); + await expect( + stack.associate(note.id, { + kind: 'relationship', + label: 'series', + target: { scope: 'Record', recordId: 'somerecordid' } as unknown as RelationshipTarget, + }), + ).rejects.toThrow(StackValidationError); + }); + + test('a target outside the three scopes is refused at create too', async () => { + await expect( + stack.create( + NOTE_V1, + { text: 'host' }, + { + associations: [ + { + kind: 'relationship', + label: 'series', + target: { + scope: 'Record', + recordId: 'somerecordid', + } as unknown as RelationshipTarget, + }, + ], + }, + ), + ).rejects.toThrow(StackValidationError); + }); + + test.each([ + ['a record target', { scope: 'record', recordId: '' }], + ['an entity target', { scope: 'entity', entityId: '' }], + ['an external target namespace', { scope: 'external', ns: '', id: 'x' }], + ['an external target id', { scope: 'external', ns: 'atproto', id: '' }], + ])('%s requires a non-empty identifier', async (_name, target) => { + const note = await stack.create(NOTE_V1, { text: 'host' }); + await expect( + stack.associate(note.id, { + kind: 'relationship', + label: 'series', + target: target as RelationshipTarget, + }), + ).rejects.toThrow(StackValidationError); + }); + + test('two targets differing only by namespace are two associations', async () => { + const note = await stack.create(NOTE_V1, { text: 'crossposted' }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }); + + const stored = await stack.get(note.id); + expect(stored?.associations).toHaveLength(2); + }); + + test('dissociate removes only the target it names', async () => { + const note = await stack.create(NOTE_V1, { text: 'crossposted' }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }); + await stack.dissociate(note.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + + const stored = await stack.get(note.id); + expect(stored?.associations).toEqual([ + { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }, + ]); + }); + + // A record target and an entity target carrying the same string are + // different references — which is the distinction group rosters rest on. + test('a record target does not match an entity target with the same value', async () => { + const note = await stack.create(NOTE_V1, { text: 'ambiguous' }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'about', + target: { scope: 'record', recordId: 'did:key:z6MkAlice' }, + }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'about', + target: { scope: 'entity', entityId: 'did:key:z6MkAlice' }, + }); + + const stored = await stack.get(note.id); + expect(stored?.associations).toHaveLength(2); + }); + + test('a record target in another stack is stored with its stackUrl', async () => { + const note = await stack.create(NOTE_V1, { text: 'reply' }); + await stack.associate(note.id, { + kind: 'relationship', + label: 'reply-to', + target: { scope: 'record', recordId: 'abc123', stackUrl: 'https://alice.example/stack' }, + }); + + const stored = await stack.get(note.id); + expect(stored?.associations?.[0]).toEqual({ + kind: 'relationship', + label: 'reply-to', + target: { scope: 'record', recordId: 'abc123', stackUrl: 'https://alice.example/stack' }, + }); + }); +}); + +// ------------------------------------------------------- +// query — relatedTo filter +// ------------------------------------------------------- + +describe('query — relatedTo filter', () => { + let subject: StackRecord; + + beforeEach(async () => { + subject = await stack.create(NOTE_V1, { text: 'target' }); + const withSeries = await stack.create(NOTE_V1, { text: 'in a series' }); + await stack.associate(withSeries.id, { + kind: 'relationship', + label: 'series', + target: { scope: 'record', recordId: subject.id }, + }); + const syndicated = await stack.create(NOTE_V1, { text: 'crossposted' }); + await stack.associate(syndicated.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'at://did:plc:abc/app.bsky.feed.post/3k4' }, + }); + const authored = await stack.create(NOTE_V1, { text: 'by someone' }); + await stack.associate(authored.id, { + kind: 'relationship', + label: 'author', + target: { scope: 'entity', entityId: 'did:key:z6MkAlice' }, + }); + await stack.create(NOTE_V1, { text: 'unrelated' }); + }); + + // "Carries any relationship at all" is refused by the type, not defined — + // the wire encoding has no way to say it, and `tags`/`hasAttachment` have + // no match-any form either. @ts-expect-error fails typecheck if this ever + // starts compiling. + test('a filter naming neither a label nor a target does not typecheck', () => { + // @ts-expect-error — relatedTo requires a label, a target, or both + const filter: RecordFilter = { relatedTo: {} }; + expect(filter.relatedTo).toEqual({}); + }); + + // A type is not a runtime guard: a server maps query params onto a + // filter and supplies a plain object. Refusing the empty filter here is + // what keeps it from widening to every record carrying a relationship. + test('a filter naming neither a label nor a target is refused', async () => { + await expect( + stack.query({ filter: { relatedTo: {} as NonNullable } }), + ).rejects.toThrow(StackQueryError); + }); + + test('a filter target outside the three scopes is refused', async () => { + await expect( + stack.query({ + filter: { + relatedTo: { + target: { scope: 'Record', recordId: subject.id } as unknown as NonNullable< + NonNullable['target'] + >, + }, + }, + }), + ).rejects.toThrow(StackQueryError); + }); + + test('a filter naming this stack omits stackUrl rather than emptying it', async () => { + await expect( + stack.query({ + filter: { relatedTo: { target: { scope: 'record', recordId: subject.id, stackUrl: '' } } }, + }), + ).rejects.toThrow(StackQueryError); + }); + + test('matches a record target', async () => { + const { records } = await stack.query({ + filter: { relatedTo: { target: { scope: 'record', recordId: subject.id } } }, + }); + expect(records.map((r) => r.content.text)).toEqual(['in a series']); + }); + + test('matches an entity target', async () => { + const { records } = await stack.query({ + filter: { relatedTo: { target: { scope: 'entity', entityId: 'did:key:z6MkAlice' } } }, + }); + expect(records.map((r) => r.content.text)).toEqual(['by someone']); + }); + + test('an external target without an id matches the whole namespace', async () => { + const { records } = await stack.query({ + filter: { relatedTo: { target: { scope: 'external', ns: 'atproto' } } }, + }); + expect(records.map((r) => r.content.text)).toEqual(['crossposted']); + }); + + test('an external target with an id matches exactly', async () => { + const miss = await stack.query({ + filter: { relatedTo: { target: { scope: 'external', ns: 'atproto', id: 'other' } } }, + }); + expect(miss.records).toHaveLength(0); + }); + + test('a bare label matches every target under it', async () => { + const { records } = await stack.query({ filter: { relatedTo: { label: 'series' } } }); + expect(records.map((r) => r.content.text)).toEqual(['in a series']); + }); + + // An absent stackUrl is not a wildcard: it names this stack. + test('a local record target does not match the same id in another stack', async () => { + const remote = await stack.create(NOTE_V1, { text: 'remote reply' }); + await stack.associate(remote.id, { + kind: 'relationship', + label: 'reply-to', + target: { scope: 'record', recordId: subject.id, stackUrl: 'https://alice.example/stack' }, + }); + + const local = await stack.query({ + filter: { + relatedTo: { label: 'reply-to', target: { scope: 'record', recordId: subject.id } }, + }, + }); + expect(local.records).toHaveLength(0); + + const scoped = await stack.query({ + filter: { + relatedTo: { + target: { + scope: 'record', + recordId: subject.id, + stackUrl: 'https://alice.example/stack', + }, + }, + }, + }); + expect(scoped.records.map((r) => r.content.text)).toEqual(['remote reply']); + }); +}); diff --git a/packages/record-adapter-sqlite/tests/record.test.ts b/packages/record-adapter-sqlite/tests/record.test.ts index 7b1da55..c51f152 100644 --- a/packages/record-adapter-sqlite/tests/record.test.ts +++ b/packages/record-adapter-sqlite/tests/record.test.ts @@ -927,6 +927,190 @@ describe('associations', () => { adapter.associate('does-not-exist', { kind: 'tag', label: 'starred' }), ).rejects.toThrow(StackNotFoundError); }); + + test('every target arm round-trips through storage', async () => { + const adapter = await initAdapter(); + const record = makeRecord(); + await adapter.createRecord(record); + const targets = [ + { scope: 'record' as const, recordId: 'rec-other' }, + { scope: 'record' as const, recordId: 'rec-other', stackUrl: 'https://alice.example/stack' }, + { scope: 'entity' as const, entityId: 'did:key:z6MkAlice' }, + { scope: 'external' as const, ns: 'atproto', id: 'at://did:plc:abc/app.bsky.feed.post/3k4' }, + ]; + for (const target of targets) { + await adapter.associate(record.id, { kind: 'relationship', label: 'ref', target }); + } + + const retrieved = await adapter.getRecord(record.id); + const stored = (retrieved?.associations ?? []).flatMap((a) => + a.kind === 'relationship' ? [a.target] : [], + ); + expect(stored).toEqual(expect.arrayContaining(targets)); + expect(stored).toHaveLength(targets.length); + }); + + // The primary key includes the namespace, so two copies of one utterance + // on two networks are two associations rather than a silent no-op. + test('targets differing only by namespace are distinct associations', async () => { + const adapter = await initAdapter(); + const record = makeRecord(); + await adapter.createRecord(record); + await adapter.associate(record.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + await adapter.associate(record.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }); + + const retrieved = await adapter.getRecord(record.id); + expect(retrieved?.associations).toHaveLength(2); + }); + + test('dissociate removes only the target it names', async () => { + const adapter = await initAdapter(); + const record = makeRecord(); + await adapter.createRecord(record); + await adapter.associate(record.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + await adapter.associate(record.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }); + await adapter.dissociate(record.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'copy-1' }, + }); + + const retrieved = await adapter.getRecord(record.id); + expect(retrieved?.associations).toEqual([ + { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'activitypub', id: 'copy-1' }, + }, + ]); + }); +}); + +// ------------------------------------------------------- +// relatedTo filter +// ------------------------------------------------------- + +describe('records — relatedTo filter', () => { + const seed = async (adapter: Awaited>) => { + const series = makeRecord({ id: 'rec-series', content: { text: 'in a series' } }); + const syndicated = makeRecord({ id: 'rec-syndicated', content: { text: 'crossposted' } }); + const authored = makeRecord({ id: 'rec-authored', content: { text: 'by someone' } }); + const bare = makeRecord({ id: 'rec-bare', content: { text: 'unrelated' } }); + for (const r of [series, syndicated, authored, bare]) await adapter.createRecord(r); + await adapter.associate(series.id, { + kind: 'relationship', + label: 'series', + target: { scope: 'record', recordId: 'rec-subject' }, + }); + await adapter.associate(syndicated.id, { + kind: 'relationship', + label: 'syndicated-to', + target: { scope: 'external', ns: 'atproto', id: 'at://did:plc:abc/app.bsky.feed.post/3k4' }, + }); + await adapter.associate(authored.id, { + kind: 'relationship', + label: 'author', + target: { scope: 'entity', entityId: 'did:key:z6MkAlice' }, + }); + }; + + const ids = (result: { records: StackRecord[] }) => result.records.map((r) => r.id).sort(); + + test('matches a record target', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const result = await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'record', recordId: 'rec-subject' } } }, + }); + expect(ids(result)).toEqual(['rec-series']); + }); + + test('matches an entity target', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const result = await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'entity', entityId: 'did:key:z6MkAlice' } } }, + }); + expect(ids(result)).toEqual(['rec-authored']); + }); + + test('an external target without an id matches the whole namespace', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const result = await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'external', ns: 'atproto' } } }, + }); + expect(ids(result)).toEqual(['rec-syndicated']); + }); + + test('a bare label matches every target under it', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const result = await adapter.queryRecords({ filter: { relatedTo: { label: 'author' } } }); + expect(ids(result)).toEqual(['rec-authored']); + }); + + // An entity target and a record target holding the same string are + // different references — the distinction group rosters rest on. + test('a record target does not match an entity target with the same value', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const result = await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'record', recordId: 'did:key:z6MkAlice' } } }, + }); + expect(result.records).toHaveLength(0); + }); + + // An absent stackUrl names this stack rather than acting as a wildcard. + test('a local record target does not match the same id in another stack', async () => { + const adapter = await initAdapter(); + await seed(adapter); + const remote = makeRecord({ id: 'rec-remote' }); + await adapter.createRecord(remote); + await adapter.associate(remote.id, { + kind: 'relationship', + label: 'reply-to', + target: { + scope: 'record', + recordId: 'rec-elsewhere', + stackUrl: 'https://alice.example/stack', + }, + }); + + const local = await adapter.queryRecords({ + filter: { relatedTo: { target: { scope: 'record', recordId: 'rec-elsewhere' } } }, + }); + expect(local.records).toHaveLength(0); + + const scoped = await adapter.queryRecords({ + filter: { + relatedTo: { + target: { + scope: 'record', + recordId: 'rec-elsewhere', + stackUrl: 'https://alice.example/stack', + }, + }, + }, + }); + expect(ids(scoped)).toEqual(['rec-remote']); + }); }); // ------------------------------------------------------- diff --git a/packages/sqlite-shared/src/mappers.ts b/packages/sqlite-shared/src/mappers.ts index 8cf644d..d78d38a 100644 --- a/packages/sqlite-shared/src/mappers.ts +++ b/packages/sqlite-shared/src/mappers.ts @@ -4,7 +4,13 @@ * contract; keeping one copy means the adapters can't drift on them. */ -import type { StackRecord, StackType, RecordVersion, Association } from '@haverstack/core'; +import type { + StackRecord, + StackType, + RecordVersion, + Association, + RelationshipTarget, +} from '@haverstack/core'; export const toMs = (d: Date): number => d.getTime(); export const fromMs = (ms: number): Date => new Date(ms); @@ -48,10 +54,35 @@ export const rowToAssociation = (row: Record): Association => { return { kind: 'relationship', label: row.label as string, - recordId: row.related_id as string, + target: rowToTarget(row), }; }; +/** + * The five columns that identify an association, in the order every + * INSERT and DELETE below binds them. One helper because the two must + * agree exactly — a dissociate that bound them differently would delete + * nothing and report success. + */ +export const associationKeyColumns = (a: Association): [string, string, string, string, string] => { + if (a.kind === 'attachment') return [a.fileId, '', '', '', '']; + if (a.kind !== 'relationship') return ['', '', '', '', '']; + const t = a.target; + if (t.scope === 'entity') return ['', 'entity', t.entityId, '', '']; + if (t.scope === 'external') return ['', 'external', t.id, t.ns, '']; + return ['', 'record', t.recordId, '', t.stackUrl ?? '']; +}; + +const rowToTarget = (row: Record): RelationshipTarget => { + const id = row.related_id as string; + if (row.related_scope === 'entity') return { scope: 'entity', entityId: id }; + if (row.related_scope === 'external') { + return { scope: 'external', ns: row.related_ns as string, id }; + } + const stackUrl = row.related_stack as string; + return { scope: 'record', recordId: id, ...(stackUrl && { stackUrl }) }; +}; + export const rowToType = (row: Record): StackType => { const t: StackType = { id: row.id as string, diff --git a/packages/sqlite-shared/src/query.ts b/packages/sqlite-shared/src/query.ts index bb930ea..ee56859 100644 --- a/packages/sqlite-shared/src/query.ts +++ b/packages/sqlite-shared/src/query.ts @@ -88,11 +88,18 @@ export const buildWhereClause = (query: StackQuery): { sql: string; params: unkn params.push(f.updatedAt.before.getTime()); } + // Every association filter below is a semi-join rather than a correlated + // EXISTS, so the planner drives from the association side — reading the + // matching rows through idx_assoc_kind_label / idx_assoc_kind_file_id / + // idx_file_refs_file_id and looking up those records — instead of + // scanning every record and probing for each. The work is proportional + // to how many records match, not to how many the stack holds. + // Tag filter — record must have ALL specified tags if (f.tags?.length) { for (const tag of f.tags) { conditions.push( - `EXISTS (SELECT 1 FROM associations a WHERE a.record_id = r.id AND a.kind = 'tag' AND a.label = ?)`, + `r.id IN (SELECT a.record_id FROM associations a WHERE a.kind = 'tag' AND a.label = ?)`, ); params.push(tag); } @@ -101,7 +108,7 @@ export const buildWhereClause = (query: StackQuery): { sql: string; params: unkn // Attachment label filter if (f.hasAttachment) { conditions.push( - `EXISTS (SELECT 1 FROM associations a WHERE a.record_id = r.id AND a.kind = 'attachment' AND a.label = ?)`, + `r.id IN (SELECT a.record_id FROM associations a WHERE a.kind = 'attachment' AND a.label = ?)`, ); params.push(f.hasAttachment); } @@ -110,21 +117,47 @@ export const buildWhereClause = (query: StackQuery): { sql: string; params: unkn // either via an attachment association or a top-level file-ref content field if (f.attachmentFileId) { conditions.push( - `(EXISTS (SELECT 1 FROM associations a WHERE a.record_id = r.id AND a.kind = 'attachment' AND a.file_id = ?) - OR EXISTS (SELECT 1 FROM file_refs fr WHERE fr.record_id = r.id AND fr.file_id = ?))`, + `(r.id IN (SELECT a.record_id FROM associations a WHERE a.kind = 'attachment' AND a.file_id = ?) + OR r.id IN (SELECT fr.record_id FROM file_refs fr WHERE fr.file_id = ?))`, ); params.push(f.attachmentFileId, f.attachmentFileId); } - // Relationship filter + // Relationship filter — each clause is an optional pattern, so a bare + // label matches every target under it and an external target with no + // `id` matches its whole namespace (docs/spec/data-model.md § Filter). + // A target-bearing pattern reads through idx_assoc_related; a bare label + // through idx_assoc_kind_label. if (f.relatedTo) { + const clauses: string[] = []; + const target = f.relatedTo.target; + if (f.relatedTo.label !== undefined) { + clauses.push('a.label = ?'); + params.push(f.relatedTo.label); + } + if (target) { + clauses.push('a.related_scope = ?'); + params.push(target.scope); + if (target.scope === 'record') { + clauses.push('a.related_id = ?', 'a.related_stack = ?'); + params.push(target.recordId, target.stackUrl ?? ''); + } else if (target.scope === 'entity') { + clauses.push('a.related_id = ?'); + params.push(target.entityId); + } else { + clauses.push('a.related_ns = ?'); + params.push(target.ns); + if (target.id !== undefined) { + clauses.push('a.related_id = ?'); + params.push(target.id); + } + } + } conditions.push( - `EXISTS (SELECT 1 FROM associations a WHERE a.record_id = r.id AND a.kind = 'relationship' AND a.related_id = ?` + - (f.relatedTo.label ? ` AND a.label = ?` : '') + + `r.id IN (SELECT a.record_id FROM associations a WHERE a.kind = 'relationship'` + + clauses.map((c) => ` AND ${c}`).join('') + `)`, ); - params.push(f.relatedTo.recordId); - if (f.relatedTo.label) params.push(f.relatedTo.label); } // Content field filters (top-level scalar exact match). A `null` value diff --git a/packages/sqlite-shared/src/record-logic.ts b/packages/sqlite-shared/src/record-logic.ts index b0b7851..ea0d22d 100644 --- a/packages/sqlite-shared/src/record-logic.ts +++ b/packages/sqlite-shared/src/record-logic.ts @@ -31,7 +31,14 @@ import type { SqlExecutor } from './executor.js'; import { isForeignKeyViolation, isUniqueConstraintViolation } from './executor.js'; import { buildWhereClause, buildOrderClause, getSortField } from './query.js'; import { fts5Strategy } from './fts5.js'; -import { rowToRecord, rowToAssociation, rowToType, rowToVersion, toMs } from './mappers.js'; +import { + rowToRecord, + rowToAssociation, + rowToType, + rowToVersion, + toMs, + associationKeyColumns, +} from './mappers.js'; import { makeCursor } from './cursor.js'; export type SharedSqlRecordLogicDeps = { @@ -690,15 +697,12 @@ export class SharedSqlRecordLogic { WHERE record_id = ? AND kind = ? AND label = ? - AND file_id = ? - AND related_id = ?`, - [ - recordId, - association.kind, - association.label, - association.kind === 'attachment' ? association.fileId : '', - association.kind === 'relationship' ? association.recordId : '', - ], + AND file_id = ? + AND related_scope = ? + AND related_id = ? + AND related_ns = ? + AND related_stack = ?`, + [recordId, association.kind, association.label, ...associationKeyColumns(association)], ); this.exec.exec('COMMIT'); } catch (err) { @@ -731,15 +735,9 @@ export class SharedSqlRecordLogic { try { this.exec.run( `INSERT OR IGNORE INTO associations - (record_id, kind, label, file_id, related_id) - VALUES (?, ?, ?, ?, ?)`, - [ - recordId, - assoc.kind, - assoc.label, - assoc.kind === 'attachment' ? assoc.fileId : '', - assoc.kind === 'relationship' ? assoc.recordId : '', - ], + (record_id, kind, label, file_id, related_scope, related_id, related_ns, related_stack) + VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, + [recordId, assoc.kind, assoc.label, ...associationKeyColumns(assoc)], ); } catch (err) { if (isForeignKeyViolation(err)) { diff --git a/packages/sqlite-shared/src/schema.ts b/packages/sqlite-shared/src/schema.ts index 95dad43..7637c50 100644 --- a/packages/sqlite-shared/src/schema.ts +++ b/packages/sqlite-shared/src/schema.ts @@ -24,13 +24,20 @@ export const RECORD_SCHEMA_SQL = ` permissions TEXT CHECK (permissions IS NULL OR json_valid(permissions)) ) STRICT; + -- A relationship's target is (related_scope, related_id) plus one + -- qualifier: related_stack for a record in another stack, related_ns for + -- a foreign namespace. All four are in the primary key, so two targets + -- differing only by namespace are two associations. CREATE TABLE IF NOT EXISTS associations ( - record_id TEXT NOT NULL REFERENCES records(id), - kind TEXT NOT NULL CHECK (kind IN ('tag', 'attachment', 'relationship')), - label TEXT NOT NULL, - file_id TEXT NOT NULL DEFAULT '', - related_id TEXT NOT NULL DEFAULT '', - PRIMARY KEY (record_id, kind, label, file_id, related_id) + record_id TEXT NOT NULL REFERENCES records(id), + kind TEXT NOT NULL CHECK (kind IN ('tag', 'attachment', 'relationship')), + label TEXT NOT NULL, + file_id TEXT NOT NULL DEFAULT '', + related_scope TEXT NOT NULL DEFAULT '' CHECK (related_scope IN ('', 'record', 'entity', 'external')), + related_id TEXT NOT NULL DEFAULT '', + related_ns TEXT NOT NULL DEFAULT '', + related_stack TEXT NOT NULL DEFAULT '', + PRIMARY KEY (record_id, kind, label, file_id, related_scope, related_id, related_ns, related_stack) ) STRICT; CREATE TABLE IF NOT EXISTS versions ( @@ -81,6 +88,7 @@ export const RECORD_SCHEMA_SQL = ` CREATE INDEX IF NOT EXISTS idx_assoc_record_id ON associations(record_id); CREATE INDEX IF NOT EXISTS idx_assoc_kind_label ON associations(kind, label); CREATE INDEX IF NOT EXISTS idx_assoc_kind_file_id ON associations(kind, file_id); + CREATE INDEX IF NOT EXISTS idx_assoc_related ON associations(related_scope, related_ns, related_id); CREATE INDEX IF NOT EXISTS idx_types_base_id ON types(base_id); CREATE INDEX IF NOT EXISTS idx_file_refs_file_id ON file_refs(file_id); `;