Async Bulkhead

Isolation by semaphore rather than thread-pool — the option that makes Bulkhead viable on a hub with 500 partners.

Problem

The classic Bulkhead allocates a thread pool per partner. On a hub with 500 active partners, 20 threads per pool = 10,000 threads. At 1 MB stack per thread, that is 10 GB RAM just for stacks. Most threads sleep waiting on I/O. Kernel context-switching becomes the bottleneck. No longer viable.

Forces

• Per-partner isolation stays necessary. Walmart saturation must not affect Carrefour.
• OS threads are expensive. Memory stack, kernel scheduling, context-switch.
• I/O is historically almost always blocking. But Project Reactor, Vert.x, Netty, async Python changed the game.
• A coroutine or Future costs nothing. A few bytes of stack, no kernel scheduling.

Solution

Replace the thread pool with a semaphore capping the number of concurrent calls. Application code calls semaphore.acquire() before the partner call (typically non-blocking, returns a CompletableFuture/Mono), executes the call on a shared event-loop, and releases the permit at the end. If all permits are taken, the request fails fast (BulkheadFullException) or queues into a bounded queue. Isolation is logical, not physical. Resilience4j offers both modes; in Reactive Streams the pattern is built in via backpressure.

Thread-pool Bulkhead variant:        Semaphore Bulkhead variant:

  partner Walmart                       partner Walmart
   ─────────────────                     ─────────────────
   ThreadPool(20)                        Semaphore(20)
   ↓ 20 dedicated threads                ↓ 20 permits
   ↓ ~1 MB stack per thread              ↓ a few bytes of counting
   ↓ blocking I/O OK                     ↓ non-blocking I/O required
   ↓ expensive context-switch            ↓ event-loop scheduling
   ↓ overhead 20 MB / partner            ↓ negligible overhead

  For 500 partners:
  - thread-pool  → 10 GB RAM in stacks alone
  - semaphore   → a few MB total

EDI implementation

Concrete case: Spring WebFlux EDI hub with Resilience4j semaphore mode. Config resilience4j.bulkhead.instances.walmart.maxConcurrentCalls=20 and similar per partner. The AS2 client uses WebClient + Reactor Netty (shared event-loop). When Walmart is slow, its 20 permits get used, new requests to Walmart are rejected with BulkheadFullException, routed to a backoff retry queue. For Carrefour, its own 20 permits remain available. Memory cost: 500 partners × 20 permits = 10,000 atomic integers ~80 KB. Vs 10 GB with thread-pool mode. On Kafka Streams, equivalent: partitions by partner key, parallelism bounded by max.poll.records. On Python asyncio: asyncio.Semaphore(20) per partner.

Anti-patterns

• Semaphore bulkhead on blocking code. The semaphore does not protect the blocking thread: a thread-pool is needed. Pick the right variant for the code.
• No bounded queue. If requests waiting on the semaphore pile up without limit, memory saturation returns. Set maxWaitDuration and maxQueueSize.
• Mixing semaphore and thread-pool incorrectly. A semaphore-bulkhead in front of a thread-pool-bulkhead with uncoordinated sizes: two overlapping limits with unpredictable behaviour.
• One event-loop for all partners. A CPU-bound task in an event-loop thread brings down every partner. Reserve the event-loop for strict I/O.

Related patterns

• Bulkhead — the parent pattern (thread-pool variant).
• Back-pressure — often combined in reactive mode: backpressure regulates upstream.
• Rate Limiter — the semaphore caps concurrency, the rate limiter caps throughput.
• Circuit Breaker — often combined: if a partner fails, the circuit opens before the bulkhead saturates.

Sources

• Resilience4j — Bulkhead. Documentation details both variants: SemaphoreBulkhead and ThreadPoolBulkhead. resilience4j.readme.io/docs/bulkhead
• Nygard M. — Release It!, Pragmatic Bookshelf, 2nd ed. 2018. The parent pattern.
• Reactive Streams Specification. Native backpressure makes the concurrency bound explicit. reactive-streams.org
• Project Reactor — Reactor Netty bulkhead. Idiomatic implementation in Reactor. projectreactor.io
• Microsoft Architecture Center — Bulkhead pattern (mentions both variants). learn.microsoft.com — Bulkhead