Repository navigation
[To dev/1.3] [IT] Fix port occupation check that always reported ports as free (#15794, #19070) - #19120
Merged
JackieTien97 merged 1 commit intoNov 4, 2026
Conversation
…s as free
EnvUtils.checkPortsAvailable() passed a shell pipeline to
Runtime.exec(String), which does not use a shell:
"lsof -iTCP -sTCP:LISTEN,TIME_WAIT -P -n | awk '{print $9}' | grep -E ...\""
on Unix and "netstat -aon -p tcp | findStr ..." on Windows. lsof received
"|", "awk", ... as file names and exited with 1, which the method treats
as "available". Every port range was therefore accepted even when one of
its ports was in use, and a node could fail with BindException.
Port the master implementation, which runs lsof / netstat without a shell
and parses the output in Java:
- the EnvUtils part of apache#15794 (88abd3e): checkPortsAvailable() now
delegates to listPortOccupation(...);
- apache#19070 (2fd7d83): read the lsof address from column 8 instead of the
state column 9.
The port-check code now matches master. Adaptations:
- explicit java.io imports instead of java.io.* (checkstyle AvoidStarImport);
- the lsof state filter stays LISTEN,TIME_WAIT, as already on dev/1.3
(apache#17618) and on master (apache#17551);
- getSearchAvailablePortCmd, getWindowsSearchPortCmd and getUnixSearchPortCmd
become unused and are removed, as master did in apache#17551.
The rest of apache#15794 (cluster restart handling and server-side changes) is
not part of this change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On
dev/1.3, the IT framework never detects an occupied port.EnvUtils.checkPortsAvailable()passed a shell pipeline (lsof -iTCP -sTCP:LISTEN,TIME_WAIT -P -n | awk '{print $9}' | grep -E ...on Unix,netstat -aon -p tcp | findStr ...on Windows) toRuntime.exec(String), which does not use a shell. lsof received|,awk, ... as file names and exited with 1, which the method treats as "available".searchAvailablePorts()therefore accepted every port range even when one of its ports was in use, and a node could then fail withBindException.This PR ports the master implementation, which runs lsof / netstat without a shell and parses the output in Java:
EnvUtilspart of Enhance it for cluster-restart tests #15794 (source commit88abd3e3097f9f1c16c51945c0ab0cf74307263b):checkPortsAvailable()delegates tolistPortOccupation(...);2fd7d838cbab3bd8645681b8020c5597e340378d): read the lsof address from column 8 instead of the state column 9.Target base:
dev/1.3atc0811e54d01c1f4480d10506930c44d06cfb12e6. After this change, the port-check block ofEnvUtils(checkPortsAvailablethroughlistPortOccupationUnix) is identical to master at2fd7d838cb.Compatibility adaptations
java.ioimports instead ofjava.io.*, because the CheckstyleAvoidStarImportrule is error level.LISTEN,TIME_WAIT, as already ondev/1.3([To dev/1.3] Clean up dead RPC thread config and use node-specific selectorNum #17618) and on master (Clean up dead RPC thread config and use node-specific selectorNum #17551).getSearchAvailablePortCmd,getWindowsSearchPortCmdandgetUnixSearchPortCmdbecome unused and are removed, as master did in Clean up dead RPC thread config and use node-specific selectorNum #17551.Only the
EnvUtilspart of #15794 is included. Its cluster-restart handling (AbstractEnv,AbstractNodeWrapper) and server-side changes are not part of this backport; ondev/1.3onlycheckPortsAvailable()uses the(port, pid)map returned bylistPortOccupation(...).Validation
Local validation with JDK 17.0.5 and Maven 3.9.9:
mvn clean package -DskipTests -pl integration-test -am -P with-integration-tests: build success, Spotless clean, 0 Checkstyle violations.python3 -m http.server, or ports held by another local application, were all reported as available; lsof printedstatus error on |: No such file or directoryand exited with 1.EnvUtils): occupied ranges are reported with the owning pid andcheckPortsAvailable()returns false (IPv4 loopback, wildcard, a listener in the same JVM, the last port of the range). Free ranges and a port just past the range are reported as free, andsearchAvailablePorts()returns a free range.ubuntu:24.04container, lsof 4.95.0 as onubuntu-latest): LISTEN lines have 10 columns with the address at index 8 and the pid at index 1, also for a multi-threaded process. IPv4 loopback, IPv4 wildcard and IPv6 loopback listeners are detected; the old index 9 matches nothing.IoTDBSimpleQueryIT(34 tests, a fresh cluster per test) andIoTDBFlushQueryIT(7 tests) withwith-integration-tests(SimpleIT): 0 failures/errors; the 3 skipped tests are@Ignored in source.netstat -aon -p tcp) could not be run locally. It is the same code master runs, and the Windows job ofCluster IT - 1C1Dcovers it on this PR.🤖 Generated with Claude Code