Skip to content

feat: add Microservices Bulkhead pattern (#3228) - #3597

Merged
iluwatar merged 3 commits into
iluwatar:masterfrom
ylcn91:feat/microservices-bulkhead
Sep 13, 2026
Merged

feat: add Microservices Bulkhead pattern (#3228)#3597
iluwatar merged 3 commits into
iluwatar:masterfrom
ylcn91:feat/microservices-bulkhead

Conversation

@ylcn91

@ylcn91 ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds the Microservices Bulkhead pattern as a new microservices-bulkhead module.

  • Problem: when every downstream call shares one thread pool, a single slow or hanging dependency (here a payment provider) consumes all threads and healthy dependencies (inventory) start failing although nothing is wrong with them.
  • Solution: each downstream dependency gets its own Bulkhead, a dedicated fixed-size thread pool with a bounded queue. When the compartment is full the call fails fast with BulkheadFullException instead of blocking or borrowing threads from other compartments.
  • Key components:
    • Bulkhead: named, bounded ThreadPoolExecutor (threads + queue), fail-fast rejection, metrics (active, queued, rejected), AutoCloseable.
    • BulkheadFullException: unchecked exception carrying the compartment name.
    • RemoteService, PaymentService (slow), InventoryService (healthy): simulated downstream dependencies.
    • App: runs the same load first through one shared pool (inventory call gets rejected) and then through dedicated bulkheads (payment overflow rejected fast, inventory keeps answering), with log output tracing every step.
    • README.md: intent, real-world example, sequence diagram, code walkthrough, applicability, trade-offs, related patterns. PlantUML class diagram under etc/.
  • Tests: 12 JUnit 5 tests (latch-based, no sleeps) covering execution, rejection when threads and queue are full, capacity release, failure propagation, thread naming, shutdown, configuration validation, plus AppTest.
  • Module registered in the parent pom.xml. ./mvnw clean verify -pl microservices-bulkhead passes locally on JDK 21 and inside an eclipse-temurin:21 container.

Fixes #3228

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

PR Summary

Introduced the Microservices Bulkhead pattern as a new microservices-bulkhead module. The change isolates each downstream dependency in its own bounded thread pool so slow or failing services cannot starve healthy ones. It adds Bulkhead with fail-fast rejection and metrics, a BulkheadFullException, and simulate downstreams via RemoteService, PaymentService (slow) and InventoryService (healthy). Includes App demo, unit tests, a README, and diagrams. Also fixed shutdown to cancel queued tasks to avoid hangs. Module registered in the parent POM and builds with JDK 21.

Changes

File Summary
microservices-bulkhead/README.md Documentation detailing the Bulkhead pattern with real-world analogy, code walkthrough, and usage guidance, including diagrams and a Java example with Bulkhead, BulkheadFullException, and RemoteService.
microservices-bulkhead/etc/microservices-bulkhead.urm.png New URM diagram image for the Bulkhead module.
microservices-bulkhead/etc/microservices-bulkhead.urm.puml PlantUML diagram source file describing the Bulkhead classes and relationships.
microservices-bulkhead/pom.xml Module pom defining the microservices-bulkhead module with dependencies and build config.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java Demo app showcasing scenario with a shared pool and dedicated bulkheads per downstream dependency, logs results.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java Core Bulkhead implementation: bounded ThreadPoolExecutor per compartment, fail-fast on rejection, metrics, and shutdown behavior.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java Unchecked exception BulkheadFullException carrying the bulkhead name for fast rejection.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java Healthy downstream mock service implementing RemoteService.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java Slow downstream mock service implementing RemoteService.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java Functional interface representing a downstream call routed through a Bulkhead.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java JUnit 5 tests verifying App behavior and bulkhead interactions.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java JUnit 5 tests covering execution, rejection, release, and shutdown behavior of Bulkhead.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java Test ensuring InventoryService reserves inventory immediately.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java Test ensuring PaymentService approves after latency and handles interrupts.
modified:pom.xml Root pom updated to include the microservices-bulkhead module.

autogenerated by presubmit.ai

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (2)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-69]

    readability: "Logger naming consistency with Lombok"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-110]

    readability: "Logger naming inconsistency in Bulkhead"

Skipped Comments (0)

Comment thread microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Note on the automated review comments: LOGGER is the Lombok logger field name configured for this repository in lombok.config (lombok.log.fieldName = LOGGER), the same name every other module uses, so the code compiles as is. Local ./mvnw clean verify -pl microservices-bulkhead passes on JDK 21, also inside an eclipse-temurin:21 container.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.39130% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.18%. Comparing base (dd24a29) to head (5e4e26c).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
.../src/main/java/com/iluwatar/bulkhead/Bulkhead.java 92.50% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3597      +/-   ##
============================================
+ Coverage     83.87%   84.18%   +0.31%     
- Complexity     4315     4355      +40     
============================================
  Files          1128     1133       +5     
  Lines         15285    15400     +115     
  Branches        732      739       +7     
============================================
+ Hits          12820    12965     +145     
+ Misses         2168     2137      -31     
- Partials        297      298       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 94d0736 to 3ada5e2 Compare September 3, 2026 09:52
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on the Codecov note: added five AppTest cases that exercise the failure, rejection and interruption branches of the demo helpers (callInventory, awaitAll), which are now package-private so they can be called from the test. 17 tests; the only uncovered line left in the module is the implicit App constructor. ./mvnw clean verify -pl microservices-bulkhead passes locally on JDK 21 and in an eclipse-temurin:21 container.

@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 3ada5e2 to 9532148 Compare September 3, 2026 10:11
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

The last CI run failed in BulkheadTest.shouldAcceptCallsAgainAfterCapacityIsReleased: with a queue-less bulkhead (SynchronousQueue) the completed Future does not guarantee that the single worker thread is already back polling the queue, so a submit issued in that window is rejected. The test now uses a bulkhead with a queue of one, drains it, and only then submits again, which is deterministic. Production code unchanged. Verified with 12 consecutive local runs, ./mvnw clean verify -pl microservices-bulkhead on JDK 21, and the same build inside an eclipse-temurin:21 container.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (4)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-64]

    bug: "Logger name mismatch with Lombok"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java [38-38]

    bug: "Logger name mismatch in InventoryService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java [51-51]

    bug: "Logger name mismatch in PaymentService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-111]

    bug: "Logger name consistency in Bulkhead"

Skipped Comments (0)

Comment thread microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java
@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 9532148 to 4cd5d56 Compare September 3, 2026 11:32
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Coverage follow-up: added AppTest.shouldBeInstantiable for the implicit App constructor, the last uncovered line. The module now reports 100% instruction, branch and line coverage in JaCoCo. Verified with ./mvnw clean verify -pl microservices-bulkhead on JDK 21 locally and in an eclipse-temurin:21 container, plus five consecutive test runs.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (1)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [60-69]

    readability: "Logger field name mismtach with Lombok SLF4J"

Skipped Comments (2)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java [36-39]

    readability: "Logger field naming mismatch in InventoryService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java [50-59]

    readability: "Logger field naming mismatch in PaymentService"

Comment thread microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java
Shutdown now cancels the queued futures so callers do not hang, a shutdown racing a submit is no longer counted as a rejection, and interrupted inventory checks are logged. The class diagram is a rendered PNG.
@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 565f429 to 0165376 Compare September 7, 2026 08:04

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (2)
  • 0165376: fix: cancel queued calls on bulkhead shutdown

Shutdown now cancels the queued futures so callers do not hang, a shutdown racing a submit is no longer counted as a rejection, and interrupted inventory checks are logged. The class diagram is a rendered PNG.

Files Processed (15)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.png (0 hunks)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (6)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-64]

    best_practice: "Logger naming mismatch with Lombok @slf4j"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [71-71]

    best_practice: "Logger naming mismatch in multiple logging sites"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-110]

    best_practice: "Logger naming mismatch in Bulkhead - debug path"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [116-121]

    best_practice: "Logger naming mismatch in Bulkhead - warn path"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java [38-38]

    best_practice: "Logger naming mismatch in InventoryService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java [51-51]

    best_practice: "Logger naming mismatch in PaymentService"

Skipped Comments (0)

Comment thread microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java
Comment thread microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (1)
  • 5e4e26c: Merge branch 'master' into feat/microservices-bulkhead
Files Processed (7)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.png (0 hunks)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (0)
Skipped Comments (3)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-64]

    readability: "Logger naming mismatch in App.java"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-110]

    readability: "Logger naming mismatch in Bulkhead.java"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [117-121]

    readability: "Logger usage for full bulkhead rejection in Bulkhead.java"

@iluwatar
iluwatar merged commit 4cabb20 into iluwatar:master Sep 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Microservices Bulkhead pattern

2 participants