Skip to content

fix(netwatch): mark the socket broken when a rebind fails, so sends retry it - #201

Open
jgeluk wants to merge 1 commit into
n0-computer:mainfrom
dataroadinc:upstream-fix-rebind-retry
Open

fix(netwatch): mark the socket broken when a rebind fails, so sends retry it#201
jgeluk wants to merge 1 commit into
n0-computer:mainfrom
dataroadinc:upstream-fix-rebind-retry

Conversation

@jgeluk

@jgeluk jgeluk commented Jul 30, 2026

Copy link
Copy Markdown

Fixes #200.

Problem

SocketState::rebind() deliberately stays in Closed on a bind failure ("will retry on next attempt"), but nothing arms that retry: UdpSocket::rebind() propagates the error without setting is_broken, and maybe_rebind() — the only recovery path on the send/recv side — is gated on is_broken. A single failed rebind (typically during sleep→wake on macOS, when the network-change handler fires before interfaces are back up) strands the socket in Closed permanently: every subsequent send logs socket closed at WARN and fails with BrokenPipe, at the caller's full send rate, until the process restarts.

Fix

Mark the socket broken when the rebind fails, so the next send/recv retries the bind and the socket heals as soon as the address is bindable again. A deliberate close() still stays closed: it never sets is_broken, pinned by a new test.

Tests

  • test_failed_rebind_marks_broken_so_sends_retry — forces a rebind failure, asserts the socket self-heals on the next send once the address is bindable again.
  • test_deliberate_close_stays_closed — pins that close() remains terminal.

cargo test -p netwatch --lib: 18 passed.

Field-tested for several days across daemons that previously exhibited the storm: sleep/wake now recovers within a second, zero socket closed events.

🤖 Generated with Claude Code

…etry it

SocketState::rebind deliberately stays in Closed on a bind failure
('will retry on next attempt'), but nothing armed that retry:
UdpSocket::rebind propagated the error without setting is_broken, and
maybe_rebind — the only recovery path on the send/recv side — is gated
on is_broken. A single failed rebind (typically during sleep->wake,
when the network-change handler fires before interfaces are back up)
therefore stranded the socket in Closed permanently: every subsequent
send logged 'socket closed' at WARN and failed with BrokenPipe, at the
caller's full send rate, until the process restarted.

Mark the socket broken when the rebind fails, so the next send/recv
retries the bind and the socket heals as soon as the address is
bindable again. A deliberate close() still stays closed: it never sets
is_broken, pinned by a new test.
@n0bot n0bot Bot added this to iroh Jul 30, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🚑 Needs Triage

Development

Successfully merging this pull request may close these issues.

netwatch: a single failed rebind strands UdpSocket in Closed permanently (WARN storm at full send rate after macOS sleep/wake)

1 participant