Skip to content

Fix hybrid pipe queue memory admission across data regions - #19080

Merged
jt2594838 merged 1 commit into
masterfrom
fix/pipe-queue-memory-admission
Nov 2, 2026
Merged

jt2594838 merged 1 commit into
masterfrom
fix/pipe-queue-memory-admission

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Pipe creation can be rejected as the DataRegion count grows because the admission check charges a full InsertNode queue budget for every Region. Default, hybrid, stream, and log sources use a pipe-wide floating-memory limit and can downgrade to TsFile events, so estimate this queue budget once per Pipe.

Keep the per-Region estimate for forced-log sources. Skip the Tablet queue budget for sources that do not retain InsertNode Tablets: realtime disabled, snapshot/query, non-streaming, and file/batch modes. Add regression coverage for the mode aliases and memory-budget boundaries.

Validation

  • Spotless passed on this branch with mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode.
  • The unchanged admission-method patch and the same regression test class passed a prior clean dependency-reactor build on JDK 17: 4 tests, 0 failures, 0 errors, 0 skipped.
  • A clean build of this final master-based commit was attempted, but dependency repository connectivity prevented parent POM resolution before compilation. Compilation and test execution on the final base remain unverified.

This PR has:

  • been self-reviewed.
  • added comments explaining the memory-estimation decision.
  • added unit tests covering the affected modes and budget checks.

@sonarqubecloud

sonarqubecloud Bot commented Nov 2, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@codecov

codecov Bot commented Nov 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.11%. Comparing base (1b2158f) to head (f1a9373).

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19080      +/-   ##
============================================
- Coverage     48.12%   48.11%   -0.01%     
  Complexity      625      625              
============================================
  Files          5572     5572              
  Lines        411488   411495       +7     
  Branches      54166    54169       +3     
============================================
- Hits         198010   198001       -9     
- Misses       213478   213494      +16     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jt2594838
jt2594838 merged commit cebe663 into master Nov 2, 2026
41 of 42 checks passed
@jt2594838
jt2594838 deleted the fix/pipe-queue-memory-admission branch November 2, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants