Repository navigation
Conversation
`_ZMQPipelineConnectorProxy.connect_recv` returned as soon as the SUB socket was connected, but a subscription takes a round trip through the proxy before the XPUB side starts forwarding. A sender that connected and published immediately could therefore have its first messages dropped, which surfaced as a flake in the Ray + ZMQ proxy integration test on GitHub Actions. - Extend the settle delay from 0.1s to 0.5s, which covered the observed propagation window in local reproduction. - Mark the affected integration test as flaky with 3 reruns, so a residual race is retried rather than failing the run. Split out of #284, where it was originally found: it is unrelated to the message data classes and deserves its own review. The delay is a mitigation, not a fix; a proper solution would confirm subscription propagation instead of waiting.
|
Benchmark comparison for |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_ZMQPipelineConnectorProxy.connect_recvreturned as soon as the SUB socket was connected, but a subscription needs a round trip through the proxy before the XPUB side starts forwarding. A sender that connected and published immediately could therefore have its first messages dropped (the classic ZMQ slow-joiner problem). This surfaced as a flake in the Ray + ZMQ proxy integration test on GitHub Actions.Change
connect_recvfrom0.1sto0.5s, which covered the observed propagation window in local reproduction.@pytest.mark.flaky(reruns=3)so a residual race is retried rather than failing the run.Notes
This is split out of #284, where it was originally found and diagnosed - it is unrelated to the message data classes and belongs in its own review.
The delay is a mitigation rather than a fix. A proper solution would confirm subscription propagation (e.g. a probe message with acknowledgement, or a subscribe/forward handshake) instead of guessing a wait; happy to follow up that way if preferred.
No issue is filed for this yet.
Fixes #None