feat(core)!: discriminated target union on relationship associations

[cuibonobo]

Summary

Implements #16, and settles #15 by dissolving it.

A relationship's target now names which identifier space its value belongs to, rather than assuming every reference is a Record id in this stack:

type RelationshipTarget =
  | { scope: 'record'; recordId: RecordId; stackUrl?: string }
  | { scope: 'entity'; entityId: EntityId }
  | { scope: 'external'; ns: string; id: string };

Three arms rather than the two #16 proposes, because of a finding that changes the case for this change:

Group rosters already break the current type. Membership on a _group record stores a member's DID in a field typed RecordId — docs/spec/identity.md said so outright — and groupRoleFromAssociations() compared the two as plain strings. Both are string, so nothing caught it, in code that decides group-based reads. An internal/external union would have left that in place and tagged it scope: 'internal', asserting the DID is a record in this stack. The entity arm fixes it, and a roster entry carrying a record target now confers nothing even when its value equals a member's DID.

That also takes this off the ATProto dependency chain: it is a correctness fix in the identity model, with the cross-protocol reference case arriving for free.

RecordFilter.relatedTo moves with it, or the union is half-writable — you could store a target the query engine cannot address. It names a label, a target, or both, and each is a pattern:

Filter	Matches
{ label: 'series' }	every target under that label
{ target: { scope: 'external', ns: 'atproto' } }	a whole namespace
{ target: { scope: 'entity', entityId } }	every record naming a person
{ label, target }	both, intersected

The first two were not expressible before — recordId was required. A record target with no stackUrl matches only local targets; absence names this stack rather than acting as a wildcard.

Reference-creation gating now applies only to a relationship naming a Record in this stack. The other arms are ungated, and the rationale is written into the spec rather than left implicit: the gate exists so a reference cannot convey access to, or confirm the existence of, a Record the requester may not read. None of the other arms names a Record here, core never resolves them, so no access flows through one and an accepted write reports back only what the requester supplied. Gating a target in another stack would require dereferencing that stack at write time — and a stack cannot even recognize its own URL, since _config holds no stackUrl.

What is deliberately not here

Everything else #15 proposes describes a copy that left the stack, not the record that stayed. cid, previousCid and tombstone are properties of a syndicated copy; lexiconId is one row of one bridge's translation table, and a single field can hold exactly one mapping. All of it belongs to adapter-atproto — see the comment on #15.

externalIds on EntityContent is not here either, and does not need to be: as an alias relationship with an external target it is queryable through this same filter, where a content array would be opaque to the query engine and force a scan of every _entity record. alias and syndicated-to are documented as cross-type conventions in docs/commons/README.md.

No externalRelationshipQuery capability flag (proposed in #16's follow-up comment). Every adapter stores relationships in one table and the external case is the same lookup — a boolean that is true everywhere is dead weight apps branch on forever. AGENTS.md already says optional capabilities are optional methods, never booleans. Pinned by conformance fixtures instead; there were none for relatedTo at all.

No match-any relationship filter, either (62cc52d). An earlier draft let relatedTo be {}, meaning "carries any relationship at all" — which was the only reason the wire needed a boolean ?related=true switch, since a filter with every part optional can encode to no query parameters and arrive as an unfiltered query. Nothing else in RecordFilter has a match-any form (tags requires values, hasAttachment a label), so it was the odd exception rather than a missing convenience. Requiring a label, a target, or both removes the case and the switch together, and the type now guarantees the encoding can never widen a query. A @ts-expect-error test pins the refusal.

Unrelated work, folded in at the maintainer's request

Two things ride along that are not about the target union. Both were asked for explicitly; flagging them so a reviewer knows they were decisions rather than scope creep.

Prose that narrates a change (92c2f93). AGENTS.md and CONTRIBUTING.md both state the no-references-to-previous-implementations rule for code comments and stop there — so prose that dates itself passes review unremarked. This PR's own first draft did exactly that ("the reference fabric … is in core now", fixed in 79dc55d), which is how it surfaced. The rule now extends to spec and commons prose: AGENTS.md gets the checkable line, CONTRIBUTING.md the reasoning and a worked example, plus the note that issue numbers in prose are usually the same mistake wearing a different hat — a document that sequences itself against open issues stops being true the moment one closes. No changeset: contributor docs, not shipped package content.

Association filters read through their indexes (76a50af, d69e53b). Covered below, and carrying its own changeset so it lands in the changelog on its own terms rather than inside the target-union entry.

Spec

Observable behavior changes; all four sections updated in this PR:

• docs/spec/data-model.md § Associations — the union, plus a new § Relationship targets on what each arm means and how association identity works
• docs/spec/data-model.md § Filter — relatedTo as a label-or-target pattern, the stackUrl-is-not-a-wildcard rule, and why match-any is refused
• docs/spec/identity.md § Group — the roster example, which showed a DID in recordId
• docs/spec/access-control.md § Reference-creation gating — which arm is gated and why the others are not
• docs/spec/wire-format.md § Query parameters, § Associations — scope-tagged params and the association response shape

Also README.md § Associations, and the commons docs (README, contact, article, message, place, text-types) — every #15/#16 reference across docs/ is gone, rewritten to describe the mechanism rather than the issue that proposed it.

Verification

All five, clean:

pnpm run format:check   ✓
pnpm run lint           ✓
pnpm test               ✓  1,348 passed
pnpm run build          ✓
pnpm run typecheck      ✓

28 new tests. The invariants they pin, rather than the count:

• every target arm round-trips through SQLite storage
• targets differing only by namespace are two associations, not a silent no-op — dissociate removes only the one it names
• a record target does not match an entity target holding the same string (both in storage and in groupRoleFromAssociations)
• a record-scoped roster entry confers no membership
• a target in another stack, an entity target, and an external target are each ungated
• label-only, namespace-wide and entity-targeted filters match; a local record target does not match the same id in another stack
• a filter naming neither a label nor a target does not compile

The filter tests run twice over — against MemoryAdapter and against the real SQL predicate in record-adapter-sqlite — since the two are meant to agree and previously nothing checked that for relatedTo.

Notes for reviewers

Association filters are semi-joins, not correlated EXISTS. This began as a question about whether relatedTo was too broad a filter to allow, and measurement moved the answer somewhere else: breadth is not what costs, the SQL shape is. A correlated EXISTS makes the planner scan every record and probe the association primary key for each; r.id IN (SELECT …) drives from the association side and looks up only the matches. Measured on 20k records with 4k associations, the full-scan count drops from one to zero for all four filters:

Filter	EXISTS	semi-join
attachmentFileId	8.8ms	0.14ms	resolves as a multi-index OR
tags × 2, intersecting to none	4.4ms	1.6ms
tag, 1k matches	5.0ms	3.9ms
hasAttachment, 2k matches	6.5ms	4.5ms

The gain tracks selectivity, because what is removed is the part proportional to stack size: a filter matching most of the stack gains little, a selective one gains a great deal. attachmentFileId benefits most since its condition spans two tables — SQLite unions two index lookups where the EXISTS form scanned once and probed both sides per row.

This is also what makes idx_assoc_related load-bearing. Under the EXISTS phrasing the planner never chose it: the correlated subquery always enters through record_id, so it used the primary key and stopped at kind, which left a filter naming a specific target reading fewer index columns than one naming only a label. f.search in the same builder already used the semi-join shape, so this brings the association filters in line rather than introducing a pattern.

Results are identical throughout — tags still means ALL-of via one intersected subquery per tag, and attachmentFileId still matches an attachment association or a file-ref content field.

The association table gains three columns, all in the primary key — so two copies of one record on two networks are two associations rather than a silent no-op. Stack files created before this won't have the columns.

Naming. scope is #16's word, kept, though the values now name identifier spaces rather than inside/outside. space or ref would be more literal. Cheap to change now, the same migration again later.

Follow-up worth filing: an adapter-atproto issue to inherit cid, tombstones and the type mapping, once #15 is rewritten down to bridge work.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XasADdjuVnw8cW5Eoqym81

[changeset-bot]

🦋 Changeset detected

Latest commit: 535cc83

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@haverstack/record-adapter-sqlite	Minor
@haverstack/core	Minor
@haverstack/conformance-fixtures	Minor
@haverstack/adapter-api	Minor
@haverstack/adapter-local	Patch
@haverstack/blob-adapter-disk	Patch
@haverstack/commons	Patch
@haverstack/sqlite-shared	Patch
@haverstack/wire-types	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR