Careeroza
Consuming Events with KafkaListener
A Listener in payment-service
@Component
@RequiredArgsConstructor
@Slf4j
class OrderPlacedListener {
private final PaymentService paymentService;
@KafkaListener(topics = "order-placed", groupId = "payment-service", concurrency = "3")
void onOrderPlaced(OrderPlacedEvent event,
@Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
@Header(KafkaHeaders.OFFSET) long offset) {
log.info("OrderPlaced {} (p{}@{}) amount {}", event.orderId(), partition, offset, event.totalAmount());
paymentService.startPayment(event);
}
}spring:
kafka:
bootstrap-servers: ${KAFKA_SERVERS:localhost:9092}
consumer:
group-id: payment-service
auto-offset-reset: earliest # new group: start from the oldest record
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.springframework.kafka.support.serializer.ErrorHandlingDeserializer
properties:
spring.deserializer.value.delegate.class: org.springframework.kafka.support.serializer.JsonDeserializer
spring.json.value.default.type: com.shopease.payment.events.OrderPlacedEvent
spring.json.trusted.packages: "com.shopease.*"concurrency = "3"starts 3 consumer threads, one per partition. More threads than partitions would sit idle.ErrorHandlingDeserializerwraps the JSON deserializer so one broken message ("poison pill") does not crash the listener in a loop.- Spring commits the offset after the method returns normally (ack mode
BATCHby default). - Each service has its own copy of the event class in its own package; they share the JSON contract, not a jar.
Consumer Groups in Action
Example
Two instances of payment-service share the 3 partitions; notification-service reads the same events independently:
docker exec kafka /opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --all-groupsExample output
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID notification-service order-placed 0 12 12 0 consumer-notification-service-1-4f1c... notification-service order-placed 1 9 9 0 consumer-notification-service-1-4f1c... notification-service order-placed 2 18 18 0 consumer-notification-service-1-4f1c... payment-service order-placed 0 12 12 0 consumer-payment-service-1-a9d2... payment-service order-placed 1 9 9 0 consumer-payment-service-1-a9d2... payment-service order-placed 2 16 18 2 consumer-payment-service-2-c07e...
Tip: LAG = records written but not yet processed by that group. Rising lag means consumers are too slow: add instances (up to the partition count) or make processing faster. Alert on it.