Skip to content

Support strictly unidirectional UDP air-gap transfer - #19063

Merged
jt2594838 merged 5 commits into
masterfrom
feat/air-gap-strict-unidirectional
Nov 2, 2026
Merged

jt2594838 merged 5 commits into
masterfrom
feat/air-gap-strict-unidirectional

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Strict one-way links discard all reverse traffic, so an air-gap UDP sink waiting for acknowledgements stalls pipe creation. Add sink.air-gap.strict-unidirectional=true (also available under the connector prefix), restricted to UDP and disabled by default. The sender marks no-response datagrams, and the receiver suppresses both ACK and NACK.

Repeated identical strict handshakes reauthenticate without resetting receiver files or pending slices. Preserve strict-mode context through compressed requests and reconstructed slices, avoid session refreshes for empty heartbeats, and drain the UDP socket into a bounded queue with ordered request processing. Interrupted receiver retries exit without recursively reentering request handling.

The per-Pipe sink parameter sink.air-gap.udp.send-interval-ms (with a connector alias) spaces every strict UDP datagram, including slices. It defaults to 5 ms; 0 disables pacing. The receiver requests a 32 MiB socket buffer through the internal global property pipe_air_gap_receiver_udp_socket_buffer_size_in_bytes, without adding it to user configuration templates.

Validation:

  • Spotless apply and whitespace checks passed.
  • 49 selected receiver, sink, file-transfer, and configuration tests passed for the handshake and pacing changes. After the final cancellation fix, all 13 receiver tests passed, including the interrupted-retry regression. Coverage includes repeated handshakes, compressed requests, pending slices, bounded socket draining, shutdown, parameter aliases, per-slice pacing, suppressed replies, and default acknowledged behavior.
  • English and Chinese compilation passed for the changed modules and their transitive IoTDB dependencies.
  • Two-node Linux verification with all reverse traffic blocked checked every value in realtime and gzip-compressed historical transfers: 8,000 rows and 512,000 values per scenario. Both scenarios completed with zero receiver-directory recycling, slice/order/checksum failures, or UDP socket-drop increments.

Strict mode confirms only a local send. Delivery and receiver errors cannot be confirmed through the one-way link. Both endpoints must support this mode; acknowledged transmission remains the default.

@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.48148% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.16%. Comparing base (3a51384) to head (358020e).
⚠️ Report is 7 commits behind head on master.

Files with missing lines Patch % Lines
...iver/protocol/airgap/IoTDBAirGapReceiverAgent.java 57.62% 25 Missing ⚠️
...db/commons/pipe/sink/protocol/IoTDBAirGapSink.java 84.21% 9 Missing ⚠️
...g/apache/iotdb/commons/pipe/config/PipeConfig.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19063      +/-   ##
============================================
+ Coverage     48.05%   48.16%   +0.11%     
- Complexity      604      621      +17     
============================================
  Files          5572     5572              
  Lines        410194   410442     +248     
  Branches      54112    54165      +53     
============================================
+ Hits         197110   197688     +578     
+ Misses       213084   212754     -330     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Caideyipi
Caideyipi marked this pull request as draft October 30, 2026 08:54
@Caideyipi
Caideyipi marked this pull request as ready for review October 30, 2026 10:32
@Caideyipi
Caideyipi marked this pull request as draft October 30, 2026 11:28
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Initial self-review round: reviewed the complete 19-file diff and relevant context at 343708dba8b5fc52cf79fa4d8cbc0511f01e5467.

Found and fixed one P2 cancellation issue in IoTDBAirGapReceiver.handleReq: an interrupted temporary-unavailable retry restored the interrupt flag and recursively retried. Shutting down the ordered UDP consumer could therefore reenter request handling repeatedly. The fix preserves interruption and throws InterruptedIOException to terminate the retry. The regression requires exactly one receiver invocation, preserves the interrupt flag, and checks that no ACK or NACK is emitted.

Final local commit and confirmed remote branch/PR head: 358020e0a4a476660d144ce6398ff3ecc989e90d. The fix has been committed and pushed.

Verification:

  • The earlier handshake, buffering, and pacing snapshot passed 49 selected tests. After the final cancellation fix, all 13 receiver tests passed, including the new regression, with no failures, errors, or skips.
  • English and Chinese compilation passed for the changed modules and their transitive dependencies. Spotless apply and whitespace checks passed.
  • The final package passed two-node Linux realtime and gzip-compressed historical transfers with reverse traffic blocked. Each scenario verified all 512,000 values in 8,000 rows. Both windows had six initial handshakes, zero receiver-directory deletions, zero slice/order/checksum failures, and zero UDP socket-drop increments. The historical case used the default 5 ms per-Pipe sink interval.
  • Local executable tests were not run. The checks above ran on the two verification hosts. GitHub CI is pending for the final commit.

The PR remains draft because this round found a behavioral issue. A fresh independent review of the complete final diff is required before marking it ready.

@sonarqubecloud

sonarqubecloud Bot commented Nov 2, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@Caideyipi
Caideyipi marked this pull request as ready for review November 2, 2026 06:57
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Independent re-review round: reviewed the complete 19-file PR diff and relevant context at 358020e0a4a476660d144ce6398ff3ecc989e90d, including the previous cancellation fix. No actionable issues affecting source behavior or test logic were confirmed. No code changes, new commits, or pushes were needed.

The review covered repeated-handshake authentication and state reuse, partial files and pending slices, compressed and sliced request context, bounded UDP buffering and ordered consumption, shutdown interruption, ACK/NACK suppression, configuration scope, and compatibility with the acknowledged transports. The comparison merge base is 3a51384b2dcc8c29118b7d082c829450c4408b4c; the current target branch tip is 999721fcafd5e95ff5f3de931ad34175ea315946. Relevant intervening target changes were inspected without modifying the PR branch.

Verification: this was a read-only review. Existing recorded receiver tests, English/Chinese compilation, and realtime/compressed-history transfer evidence were inspected. Builds and executable tests were not rerun, as requested. Local HEAD and the confirmed source-branch/PR head are all 358020e0a4a476660d144ce6398ff3ecc989e90d.

GitHub CI is separate from this review approval and is not fully green. DataNode unit tests on Linux and Windows, dependency checks, and Chinese compilation completed successfully. SonarCloud Code Analysis reports a failed quality gate; its annotations cover equality, complexity, style, and other static-analysis suggestions. The Windows table-standalone shard failed at IoTDBCurrentQueriesIT.testMoreConfigurations:248 (expected 1 finished query, observed 6), outside the changed code and tests. The cause was not established or changed in this round; follow-up CI investigation remains necessary. Other jobs were still running when inspected.

This complete round meets the independent-review gate. Immediately before marking ready, the local commit, source branch, and PR head were rechecked and still matched the reviewed SHA. The ready transition succeeded and returned isDraft=false at that same commit. Remote CI remains a separate, incomplete gate.

private static final int LISTENER_THREAD_COUNT = 2;
private static final int LISTENER_THREAD_COUNT = 3;
private static final int UDP_REQUEST_QUEUE_CAPACITY = 4_096;
private static final int UDP_REQUEST_QUEUE_BYTES = 32 * 1024 * 1024;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new property only controls the kernel socket receive buffer. The copied application queue remains fixed at 4,096 packets and 32 MiB, and the sender UDP packet-size and request-slice settings do not configure these limits. Could we expose the queue bounds through receiver configuration, or explicitly confirm that they are intentionally fixed? As written, receive buffering cannot be tuned end to end.

@jt2594838
jt2594838 merged commit cc8e947 into master Nov 2, 2026
40 of 42 checks passed
@jt2594838
jt2594838 deleted the feat/air-gap-strict-unidirectional branch November 2, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants