Skip to content

subscribe() cannot resume: since reaches the adapter, but nothing can set it #194

Description

@cuibonobo

An app that subscribes, stops, and starts again has no way to say where it left off. It gets every change from the present onward, and nothing about the gap.

The plumbing for the other half already exists:

  • SubscribeChangesOptions.since is on the adapter hook, and APIAdapter.subscribeChanges() honors it — it becomes Last-Event-ID on the connection.
  • RecordChange.seq reaches subscribers on relayed frames, so a consumer can retain the last cursor it saw.
  • Stack.openRelay() passes filter, includeRecords, onError and onReset through, and nothing else. SubscribeOptions has no since, so there is nothing to pass.

So a consumer can hold a cursor and an adapter can consume one, and the two cannot be introduced. Change events § Known limitations states the consequence — "a restart with no feed is a full resync" — as though it followed from having no feed. For a remote stack it doesn't: the feed resumes, the app just can't ask it to.

Why this matters more than a missing convenience

D4's rejection of blocking hooks rests on this. The argument for handlers never blocking a write is that an indexer which must not miss a write gets that from a resumable cursor and idempotent application, not from making every subscriber a latency and failure dependency of every write. That argument is right, and it names something an app cannot currently reach. Indexing is the first of #3's four use cases, and it is the one that needs this.

Delivery is already at-least-once and deduped on (recordId, version, kind), so the consumer side of resumption is a solved problem — persist the cursor after applying, and let dedupe absorb whatever the overlap replays.

Questions to settle

Does a refused cursor fire onReset? Today onReset is documented as never firing on first connect, which was true when the first connect had nothing to resume from. A caller-supplied since the server will not honor is exactly a gap on first connect, and it is the case an app most needs to hear about — it is the difference between "you are current" and "you are missing an unknown amount". The rule needs revisiting alongside the option.

What does since mean on a local stack? Local events carry no seq, so there is no cursor a local stack could ever have minted. Silently ignoring one lets an app believe it resumed when it did not; refusing loudly costs an app portability between a local and a hosted stack, which the same document sells as the point of designing the event model once. Neither is obviously right.

What does a resume: false server do with it? It answers every connection with a reset already, so presumably nothing new — but that is the path where an app's carefully persisted cursor is worth nothing, and it should be stated rather than inferred.

Not in scope

Minting seq locally. Ordering and durability are storage concerns, and core runs no storage it shares with anyone. A local stack has one writing process and therefore no gap to resume across; this issue is about a cursor a server minted reaching the adapter that can present it.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions