Skip to content

[To dev/1.3] [IT] Fix port occupation check that always reported ports as free (#15794, #19070) - #19120

Merged
JackieTien97 merged 1 commit into
apache:dev/1.3from
JackieTien97:dev13-fix-it-port-occupation-check
Nov 4, 2026
Merged

JackieTien97 merged 1 commit into
apache:dev/1.3from
JackieTien97:dev13-fix-it-port-occupation-check

Conversation

@JackieTien97

Copy link
Copy Markdown
Contributor

Summary

On dev/1.3, the IT framework never detects an occupied port. EnvUtils.checkPortsAvailable() passed a shell pipeline (lsof -iTCP -sTCP:LISTEN,TIME_WAIT -P -n | awk '{print $9}' | grep -E ... on Unix, netstat -aon -p tcp | findStr ... on Windows) to Runtime.exec(String), which does not use a shell. lsof received |, awk, ... as file names and exited with 1, which the method treats as "available". searchAvailablePorts() therefore accepted every port range even when one of its ports was in use, and a node could then fail with BindException.

This PR ports the master implementation, which runs lsof / netstat without a shell and parses the output in Java:

Target base: dev/1.3 at c0811e54d01c1f4480d10506930c44d06cfb12e6. After this change, the port-check block of EnvUtils (checkPortsAvailable through listPortOccupationUnix) is identical to master at 2fd7d838cb.

Compatibility adaptations

Only the EnvUtils part of #15794 is included. Its cluster-restart handling (AbstractEnv, AbstractNodeWrapper) and server-side changes are not part of this backport; on dev/1.3 only checkPortsAvailable() uses the (port, pid) map returned by listPortOccupation(...).

Validation

Local validation with JDK 17.0.5 and Maven 3.9.9:

  • mvn clean package -DskipTests -pl integration-test -am -P with-integration-tests: build success, Spotless clean, 0 Checkstyle violations.
  • Before the change (macOS): ranges containing a port held by python3 -m http.server, or ports held by another local application, were all reported as available; lsof printed status error on |: No such file or directory and exited with 1.
  • After the change (macOS, compiled EnvUtils): occupied ranges are reported with the owning pid and checkPortsAvailable() returns false (IPv4 loopback, wildcard, a listener in the same JVM, the last port of the range). Free ranges and a port just past the range are reported as free, and searchAvailablePorts() returns a free range.
  • Linux (ubuntu:24.04 container, lsof 4.95.0 as on ubuntu-latest): LISTEN lines have 10 columns with the address at index 8 and the pid at index 1, also for a multi-threaded process. IPv4 loopback, IPv4 wildcard and IPv6 loopback listeners are detected; the old index 9 matches nothing.
  • IoTDBSimpleQueryIT (34 tests, a fresh cluster per test) and IoTDBFlushQueryIT (7 tests) with with-integration-tests (SimpleIT): 0 failures/errors; the 3 skipped tests are @Ignored in source.
  • The Windows path (netstat -aon -p tcp) could not be run locally. It is the same code master runs, and the Windows job of Cluster IT - 1C1D covers it on this PR.

🤖 Generated with Claude Code

…s as free

EnvUtils.checkPortsAvailable() passed a shell pipeline to
Runtime.exec(String), which does not use a shell:
"lsof -iTCP -sTCP:LISTEN,TIME_WAIT -P -n | awk '{print $9}' | grep -E ...\""
on Unix and "netstat -aon -p tcp | findStr ..." on Windows. lsof received
"|", "awk", ... as file names and exited with 1, which the method treats
as "available". Every port range was therefore accepted even when one of
its ports was in use, and a node could fail with BindException.

Port the master implementation, which runs lsof / netstat without a shell
and parses the output in Java:
- the EnvUtils part of apache#15794 (88abd3e): checkPortsAvailable() now
  delegates to listPortOccupation(...);
- apache#19070 (2fd7d83): read the lsof address from column 8 instead of the
  state column 9.

The port-check code now matches master. Adaptations:
- explicit java.io imports instead of java.io.* (checkstyle AvoidStarImport);
- the lsof state filter stays LISTEN,TIME_WAIT, as already on dev/1.3
  (apache#17618) and on master (apache#17551);
- getSearchAvailablePortCmd, getWindowsSearchPortCmd and getUnixSearchPortCmd
  become unused and are removed, as master did in apache#17551.
The rest of apache#15794 (cluster restart handling and server-side changes) is
not part of this change.
@JackieTien97
JackieTien97 merged commit c8254a7 into apache:dev/1.3 Nov 4, 2026
16 checks passed
@JackieTien97
JackieTien97 deleted the dev13-fix-it-port-occupation-check branch November 4, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant