Skip to content

feat(relay)!: expose outbound peer transport and connect history - #7776

Merged
kixelated merged 8 commits into
mainfrom
feat/mesh-peer-state
Nov 4, 2026
Merged

kixelated merged 8 commits into
mainfrom
feat/mesh-peer-state

Conversation

@kixelated

@kixelated kixelated commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Operators cannot distinguish a never-connected outbound peer, a failed redial, and a WebSocket fallback from /nodes. The endpoint forgets peers as soon as they disconnect.

Approach

  • List every configured outbound peer from the moment its dial task is built, keyed by its dial identity, across failed attempts and a dial restarted for a new cost or credential. The mesh retires a peer's history only when it leaves the dial set.
  • Report each live connection's transport and the age of the last successful handshake, including short sessions between observations.
  • Extend the existing weak connection monitor with coherent transport and last-connect fields. Reuse its lifecycle revisions rather than polling or parsing logs.
  • Adapt mesh readiness and the redial drill mutation to the structured view.

Impact

  • /nodes includes disconnected configured peers with an empty connections array. Consumers counting live peers must filter for nonempty connections. Live connections add transport; peers add nullable last_connected_seconds_ago, which remains null until their first successful handshake and is forgotten when the peer is removed. Recorded in rs/moq-relay/MIGRATE.md.
  • A LAN peer's node is its mDNS identity instead of the first dialed address, whose /.cluster/<credential> path exposed its credential on this endpoint.
  • Rust connection::Observation adds transport and last_connected. Struct-literal consumers must supply them; recorded in rs/moq-tokio/MIGRATE.md. No media wire or CLI changes.

Alternatives

Journal parsing depends on retention and log formatting. A second peer registry endpoint would duplicate ownership and identity already present in /nodes.

Validation

324e34fe2dd2d083094a74e44c6046842017ffff: just rs check-test -p moq-relay -p moq-tokio passed (1024 tests, 30 skipped). 192d3a166fce12419f6c0430e304b2a9d2defd0c: just rs check-test -p moq-relay -p moq-tokio passed (1013 tests, 30 skipped, clippy enforced), just fix origin/main left no changes, and test/drill/sensitivity.sh --apply-only applies 7 of 7 mutations. Earlier, 24924821df passed just ci on nixos2.

Review

Claude Code / claude-opus-5-5 / medium (quest review --harness claude; the code is GPT-6.1 Sol's), session b28194e4-bc74-41b1-9b24-0844e4c33081.

  • Round 1, head 0c372490c2: 3 findings.
    • medium, history erased when the API replaces a dial for a cost or credential change: fixed. History is keyed by dial identity, registered in dialer, and retired by Mesh::apply only when the peer leaves the DialMap. Dials no longer consume conn ids. Covered by mesh_keeps_peer_history_until_the_peer_is_removed.
    • low, no migration note for the /nodes semantics change: fixed in both MIGRATE.md files.
    • low, a LAN peer without a node URL is labeled by its mDNS instance: documented as the peer's identity. Reverting to the old label would re-expose the LAN credential in the dialed path (DialTarget::from_lan_peer).
  • Round 2, head 192d3a166fce12419f6c0430e304b2a9d2defd0c: no findings.
  • Head 324e34fe2dd2d083094a74e44c6046842017ffff merges origin/main with no new review round. The only conflict was rs/moq-relay/MIGRATE.md, where refactor(relay)!: poll admissions and configure stats in cluster Options #7673's Unreleased entry sits beside this PR's, and both are kept. git diff origin/main...HEAD has the same added and removed lines as at 192d3a1. just rs check-test -p moq-relay -p moq-tokio passed: 1024 tests, 30 skipped.

Follow-ups

None.

(Written by Claude Opus 5.5)

kixelated and others added 5 commits November 3, 2026 14:20
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
kixelated and others added 2 commits November 3, 2026 17:12
Key outbound peer history by dial identity and retire it only when the
peer leaves the dial set, so a cost or credential change that restarts
the dial no longer resets `last_connected_seconds_ago` to null. The dial
registers when its task is built, so a peer is listed before its first
attempt, and it no longer takes a `conn` id. Document `node` as the
peer's identity and record the `/nodes` and `Observation` changes in
MIGRATE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 4, 2026 01:28
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c7fa0515-8efc-4e5d-90e5-0864548e55a1
Commits

Reviewing files that changed from the base of the PR and between 8f8545f and 324e34f.

Files selected for processing: 10
  • doc/bin/relay/cluster.md
  • doc/bin/relay/http.md
  • rs/moq-relay/MIGRATE.md
  • rs/moq-relay/src/cluster.rs
  • rs/moq-relay/src/internal.rs
  • rs/moq-relay/src/nodes.rs
  • rs/moq-relay/tests/integration/cluster_unknown.rs
  • rs/moq-tokio/MIGRATE.md
  • rs/moq-tokio/src/connection.rs
  • test/drill/mutations/peer-link-never-redialed.patch
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head: 192d3a1.

No actionable finding. Peer history is registered before dialing, survives a cost or credential replacement, and is removed with the dial-set entry. The monitor publishes transport and last-connect time coherently with lifecycle revisions, including a short session missed between observations; the readiness consumer counts live connections rather than entries. LAN labels avoid the credential-bearing dial path.

Verification: static GitHub inspection of the registry, dial lifecycle, monitor, endpoint and tests. I did not run the tests, cluster drill or real transports.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • /nodes registry rewrite — Nodes now keys every configured outbound peer by its DialTarget identity, tracks live (conn id, Transport) sessions per dial, and keeps last_connected, so disconnected and retrying peers are listed and history survives failed attempts and a dial replaced for a new cost or credential.
  • Dial ownership and retirement — the dialer registers a peer before its first attempt; Mesh::apply retires history only for peers no longer in the DialMap, and Drop for Mesh clears the shared registry on shutdown.
  • Coherent lifecycle observation — run_remote_session now drives monitor.state()/state_changed instead of reconnect.status(), recording transport and last connect from the same revision, including a connect+disconnect that lands between polls.
  • moq-tokio API — State retains last_connected; public Observation adds last_connected and transport, with migration notes in both MIGRATE.md files.
  • Snapshot labels — canonicalization strips userinfo/query/slashes and merges identities sharing a label, removing the LAN peer's credential-bearing /.cluster/<credential> path from the endpoint.
  • Tests/docs/drill — new unit and integration assertions, docs for /nodes, and an updated peer-link-never-redialed mutation; git apply --check on that patch succeeds.

Invariants I verified: Nodes keys are exactly DialTarget.key, the same identity DialMap stores, so the retain predicate cannot drop a live dial; a nonempty connections always pairs with Some(last_connected) because Shared::connected writes both in one revision; and the monitor loop wakes on a superset of the old status changes while still terminating through terminal.

(Written by deepseek/deepseek-v4.1-flash-1017)

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 701e456 Nov 4, 2026
8 of 9 checks passed
@kixelated
kixelated deleted the feat/mesh-peer-state branch November 4, 2026 03:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant