Rate Limiter, Bulkhead and Time Limiter

A rate limiter on the caller side protects a service you call (for example a bank API that allows 20 calls per second). Resilience4j splits time into cycles of limit-refresh-period and allows limit-for-period calls per cycle; extra calls wait up to timeout-duration, then fail with RequestNotPermitted. (The gateway's token bucket from the API Gateway module limits incoming traffic; this limits outgoing calls.)

resilience4j:
  ratelimiter:
    instances:
      bankApi:
        limit-for-period: 20
        limit-refresh-period: 1s
        timeout-duration: 0      # do not wait, fail at once

Ships have watertight compartments (bulkheads) so one leak does not sink the ship. A bulkhead limits how many calls to one dependency run at the same time, so a slow payment provider cannot use all 200 Tomcat threads; product pages keep working. A simulated timeline with max 2 concurrent payment calls:

import java.util.*;

public class Main {
    record Call(String name, int startMs, int durationMs) {}

    public static void main(String[] args) {
        int maxConcurrent = 2;
        List<Integer> inFlightEndTimes = new ArrayList<>();
        List<Call> calls = List.of(
            new Call("charge order-1001", 0, 300),
            new Call("charge order-1002", 50, 300),
            new Call("charge order-1003", 100, 300),
            new Call("charge order-1004", 320, 100),
            new Call("charge order-1005", 360, 100));

        for (Call c : calls) {
            inFlightEndTimes.removeIf(end -> end <= c.startMs());   // finished calls free a slot
            if (inFlightEndTimes.size() < maxConcurrent) {
                inFlightEndTimes.add(c.startMs() + c.durationMs());
                System.out.printf("t=%3dms %s -> ACCEPTED (in flight: %d)%n", c.startMs(), c.name(), inFlightEndTimes.size());
            } else {
                System.out.printf("t=%3dms %s -> REJECTED BulkheadFullException (in flight: %d) -> \"payment pending\"%n",
                        c.startMs(), c.name(), inFlightEndTimes.size());
            }
        }
    }
}

t=  0ms charge order-1001 -> ACCEPTED (in flight: 1)
t= 50ms charge order-1002 -> ACCEPTED (in flight: 2)
t=100ms charge order-1003 -> REJECTED BulkheadFullException (in flight: 2) -> "payment pending"
t=320ms charge order-1004 -> ACCEPTED (in flight: 2)
t=360ms charge order-1005 -> ACCEPTED (in flight: 2)

resilience4j:
  bulkhead:
    instances:
      paymentService:
        max-concurrent-calls: 10
        max-wait-duration: 0       # reject at once when full

@Bulkhead(name = "paymentService", fallbackMethod = "paymentPending")
@CircuitBreaker(name = "paymentService")
public PaymentResponse charge(ChargeRequest req) {
    return paymentClient.charge("order-" + req.orderId(), req);
}

PaymentResponse paymentPending(ChargeRequest req, Throwable ex) {
    return PaymentResponse.pending(req.orderId());   // confirm later by event / webhook
}

@TimeLimiter cancels a call that takes too long. It works only on async return types (CompletableFuture, Mono, Flux). For normal blocking Feign calls, the Feign read-timeout does this job.

@TimeLimiter(name = "productService")
@CircuitBreaker(name = "productService", fallbackMethod = "cachedProductAsync")
public CompletableFuture<ProductDto> getProductAsync(Long id) {
    return CompletableFuture.supplyAsync(() -> productClient.getProduct(id), executor);
}

resilience4j:
  timelimiter:
    instances:
      productService:
        timeout-duration: 2s
        cancel-running-future: true