Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/agent-guides/inbox-notifications-and-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
- **No record missed or twice.** Each tab drops a record whose seq it already passed (its subscribe cursor, its list read's, counts as passed). A new leader opens its stream after the highest seq IT has seen. A tab that learns of a new leader (a handover, or its own join) first fetches what it may lack, `GET /api/inbox/events?cursor=<its last seq>&once=1` (the hello and the replay, then the response ends; not counted as a stream), holding the leader's frames until then, so a tab that slept or whose list read was older than the new leader's start loses nothing. A catch-up that fails (the Inbox restarting) keeps holding: the frames that arrive meanwhile wait (past 5,000 the oldest records go, which the catch-up fetches again), and it is tried again on a timer (1 s, doubling to 30 s) and at the next `hello`, from the same seq, so the gap is never skipped. A replay holds each record's latest line only (`changesSince`), exactly as an EventSource reconnect always did. `prepareInboxEvents()` opens the channel before the first list read, so a frame passed on between that read and the subscription is kept.
- **Handover.** Close, reload, navigation and a crash free the leader's lock; the next tab in the lock's queue leads (measured 1 to 8 ms after the old leader went, 217 ms in the simulated freeze). `freeze` and a `pagehide` into the back/forward cache step down and leave the queue; `resume` / `pageshow` (persisted) join again, as a follower, with a catch-up. A frozen tab is never a candidate.
- **A leader that stops without freezing** (a debugger, a long script in an attached HTML page's frame, which shares the tab's main thread, a phone's browser suspending a background tab) keeps its lock and passes nothing on. So every follower asks: `ping` on the channel every 3 s while the person can see it (`LEADER_PING_EVERY_MS`; at once when it becomes visible or gets focus), every 15 s while hidden (`LEADER_HIDDEN_PING_EVERY_MS`: a notification may still be owed); the leader answers `pong` from its message handler, so the answer proves its main thread runs, and any frame or announcement from a leader counts as an answer. No answer in 2 s (`LEADER_PONG_TIMEOUT_MS`): the follower takes the lock (`navigator.locks.request(..., { steal: true })`) and leads, opening its stream from its own last seq. The old leader learns it when it runs again (its lock request rejects): it closes its stream, steps down, joins, queues again and catches up, as a follower. Until then the server holds two streams (the stopped one and the new); records the two pass on are told apart by seq. A follower takes a leader only from its announcement (or, knowing none, from a frame), so the old leader's last frames never take the tabs back. Measured: Chromium with the leader held by the debugger for 8 s, the tab in front had the stream 3.4 s in, its longest gap without a record 3.2 s, two streams at most, every tab every record once; WebKit with a looping script in an attached page, 3.8 s and 3.7 s. A leader busy for more than 2 s for an ordinary reason is taken over the same way; nothing is lost.
- **Presence (`notify_reach`).** A follower holds no stream, so its `POST /api/inbox/window` reports `served_by` (the leader's window id, `inboxEventsServedBy()`), again whenever its leader changes and on every relayed `hello` (an Inbox restart); the server counts it while that leader holds its own stream, or held one less than the 15 s grace ago (a handover). Only a leader's own stream counts, never one link further: two tabs naming each other are never counted. Each tab holds the Web Lock `plannotator-inbox-window:<its id>` for its life; the leader queues for each follower's (on its `join`), and when the browser grants it the tab is gone (closed or crashed): the leader posts `{ window, gone: true }`. A tab also sends that itself at `pagehide` (a beacon). A gone tab stops counting at once; the followers it served keep counting through the grace. Worst case (the whole browser killed): followers stop counting 15 s after the leader's stream drops (one grace, and the pages hear it then).
- **Presence (`notify_reach`).** A follower holds no stream, so its `POST /api/inbox/window` reports `served_by` (the leader's window id, `inboxEventsServedBy()`), again whenever its leader changes and on every relayed `hello` (an Inbox restart); the server counts it while that leader holds its own stream, or held one less than the 15 s grace ago (a handover). Only a leader's own stream counts, never one link further: two tabs naming each other are never counted. Each tab holds the Web Lock `plannotator-inbox-window:<its id>` for its life; the leader queues for each follower's (on its `join`), and when the browser grants it the tab is gone (closed or crashed): the leader posts `{ window, gone: true }`. A tab also sends that itself at `pagehide` (a beacon), and from then until a `pageshow` brings it back it reports nothing: a closing leader still hears the next leader for a moment, and a report then could land after every `gone` and count a closed tab. A gone tab stops counting at once; the followers it served keep counting through the grace. Worst case (the whole browser killed): followers stop counting 15 s after the leader's stream drops (one grace, and the pages hear it then).
- **Health** carries `event_streams` (additive): the event streams open now, windows' and the phone door's.
- **Another site** is refused on `GET /api/inbox/events` (`Sec-Fetch-Site` cross-site or same-site, or an Origin naming another server: `403 cross_origin`), as on the other read routes; the window, the tailnet page (same-origin), the phone's door and agents (no fetch metadata) are answered.
- **With every tab hidden**, a stalled leader is taken over in about 16 s (the next hidden check, up to 15 s, plus the 2 s wait), longer where the browser throttles a hidden tab's timers (Chrome: at most once a minute in a tab hidden for more than 5 minutes; Safari suspends background tabs).
Expand Down
30 changes: 30 additions & 0 deletions packages/inbox/sharedEvents.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,27 @@ test('the leader tells the server when a follower is gone: a crash (its lock fre
expect(w.gone.some((g) => g.by === a.id && g.window === a.win.id)).toBe(false);
});

test('a leader still running after its pagehide reports nothing more: the next leader\'s announcement and hello do not count it again', async () => {
// The page is not gone the moment pagehide fires: its scripts can still hear the next leader. A report
// then (served_by the next leader) could reach the server after every `gone` and count a closed tab.
const w = world();
const a = w.open();
await settle();
const b = w.open();
await settle(10);
expect(w.leader()).toBe(a.id);
const connects = a.connects;
const hide = new Event('pagehide');
Object.defineProperty(hide, 'persisted', { value: false });
a.win2.dispatchEvent(hide);
w.server.write(2);
await settle(16);
expect(w.leader()).toBe(b.id);
expect(w.gone).toContainEqual({ by: a.id, window: a.win.id, unloading: true });
expect(a.connects).toBe(connects);
w.kill(a);
});

test('a frozen leader steps down so another leads; it queues again when it resumes, as a follower', async () => {
const w = world();
const a = w.open();
Expand Down Expand Up @@ -578,12 +599,15 @@ test('the back/forward cache: pagehide (persisted) steps down and says gone; pag
await settle();
const b = w.open();
await settle(8);
const hidden = a.connects;
const hide = new Event('pagehide');
Object.defineProperty(hide, 'persisted', { value: true });
a.win2.dispatchEvent(hide);
await settle(10);
expect(w.leader()).toBe(b.id);
expect(w.gone).toContainEqual({ by: a.id, window: a.win.id, unloading: true });
// Going into the cache, it heard B lead but did not report: it counts again only at pageshow.
expect(a.connects).toBe(hidden);
const connects = a.connects;
const show = new Event('pageshow');
Object.defineProperty(show, 'persisted', { value: true });
Expand All @@ -593,6 +617,12 @@ test('the back/forward cache: pagehide (persisted) steps down and says gone; pag
expect(a.connects).toBeGreaterThan(connects);
expectAll(a, 0, 2);
expect(w.server.streams.size).toBe(1);
// Back from the cache it is live: its later reports (a leader change, a hello) go out again.
const restored = a.connects;
w.kill(b);
await settle(16);
expect(w.leader()).toBe(a.id);
expect(a.connects).toBeGreaterThan(restored);
});

test('a sleeping follower behind a new leader that started past it fetches the lines between, once, in order', async () => {
Expand Down
22 changes: 19 additions & 3 deletions packages/inbox/sharedEvents.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@
* holds a lock of its own (`WINDOW_LOCK_PREFIX` + its id) for its life; the
* leader queues for each follower's lock, which the browser grants when that
* tab is gone (closed or crashed), and tells the server (`gone`). A tab also
* says so itself at pagehide (a beacon).
* says so itself at pagehide (a beacon), and reports nothing more until a
* pageshow brings it back from the back/forward cache.
*
* Where Web Locks or BroadcastChannel is missing (an older browser, an
* insecure context) `subscribeInboxEvents` opens this tab's own stream, as
Expand Down Expand Up @@ -291,6 +292,19 @@ export function subscribeSharedInboxEvents(
let stopped = false;
/** Frozen, or in the back/forward cache: not a candidate, not the leader. */
let suspended = false;
/**
* Said `gone` at pagehide: until a pageshow brings it back, this tab reports
* nothing. Its scripts can still hear the next leader before the page goes,
* and a report then could reach the server after every `gone` and count a
* closed tab for good.
*/
let departed = false;
const tabHandlers: InboxEventHandlers = {
...handlers,
onConnect: () => {
if (!departed) handlers.onConnect?.();
},
};
let closeStream: (() => void) | null = null;
let releaseLeader: (() => void) | null = null;
let queued: AbortController | null = null;
Expand Down Expand Up @@ -356,7 +370,7 @@ export function subscribeSharedInboxEvents(
lastSeq = seq;
trace({ type: 'record', seq, via });
}
env.dispatch(frame, handlers);
env.dispatch(frame, tabHandlers);
};

const clearRetry = () => {
Expand Down Expand Up @@ -421,7 +435,7 @@ export function subscribeSharedInboxEvents(
leader = id;
if (id !== null && id !== win.id && !stopped) {
void catchUp();
handlers.onConnect?.();
tabHandlers.onConnect?.();
}
return true;
};
Expand Down Expand Up @@ -631,12 +645,14 @@ export function subscribeSharedInboxEvents(
const onResume = () => wake();
const onPageHide = (event: Event) => {
// Leaving for good, or for the back/forward cache: either way this tab stops counting now.
departed = true;
env.sendGone(win.id, true);
if ((event as PageTransitionEvent).persisted) suspend();
else stepDown();
};
const onPageShow = (event: Event) => {
if (!(event as PageTransitionEvent).persisted) return;
departed = false;
wake();
// The server heard `gone` at pagehide: count this tab again.
handlers.onConnect?.();
Expand Down
Loading