Idempotency
Why Duplicates Happen
An operation is idempotent when doing it twice has the same effect as doing it once. In distributed systems duplicates are normal, not rare:
- A client times out and retries a
POST /paymentsthat actually succeeded. - Resilience4j Retry or the gateway retries a call after a slow response.
- Kafka redelivers a message when a consumer crashes before committing its offset (at-least-once).
- The outbox relay re-sends a row after a crash.
- A user double-taps the "Pay ₹2,998" button on a slow 3G connection.
| HTTP method | Idempotent by definition? | Note |
|---|---|---|
| GET, HEAD | Yes | Read only |
| PUT | Yes | Sets a full value; repeating gives the same state |
| DELETE | Yes | Second call finds nothing to delete |
| POST | No | Needs an idempotency key to be safe to retry |
| PATCH | Depends | "set stock=5" yes, "add 1 to stock" no |
Example: Idempotent Payment with an Idempotency Key
Example
The client generates a unique key per payment attempt and sends it in an Idempotency-Key header. The server stores the first response for that key and replays it for retries. Reusing a key with a different body is rejected. The card is charged only twice for two real payments, even though four requests arrive:
import java.util.*;
public class Main {
public static void main(String[] args) {
CardGateway gateway = new CardGateway();
PaymentService service = new PaymentService(gateway);
String key = "7f3c9a2e-order-1001";
System.out.println(service.pay(key, new PayRequest(1001, 2998)));
System.out.println("(client timed out before seeing the response, so it retries)");
System.out.println(service.pay(key, new PayRequest(1001, 2998)));
System.out.println(service.pay(key, new PayRequest(1001, 5000))); // same key, different body
System.out.println(service.pay("a91d44b0-order-1002", new PayRequest(1002, 1499)));
System.out.println("card charged " + gateway.charges + " times, total Rs." + gateway.total);
}
}
record PayRequest(long orderId, int amount) {}
record Response(int status, String body) {
public String toString() { return status + " " + body; }
}
record StoredResult(PayRequest request, Response response) {}
class CardGateway {
int charges = 0, total = 0;
String charge(int amount) { charges++; total += amount; return "pay_" + (1000 + charges); }
}
class PaymentService {
private final Map<String, StoredResult> results = new LinkedHashMap<>(); // a DB table in real life
private final CardGateway gateway;
PaymentService(CardGateway gateway) { this.gateway = gateway; }
Response pay(String idempotencyKey, PayRequest req) {
StoredResult previous = results.get(idempotencyKey);
if (previous != null) {
if (!previous.request().equals(req))
return new Response(422, "{error: idempotency key reused with a different request}");
return new Response(previous.response().status(), previous.response().body() + " [replayed]");
}
String paymentId = gateway.charge(req.amount());
Response response = new Response(201, "{paymentId: " + paymentId + ", orderId: " + req.orderId()
+ ", amount: " + req.amount() + "}");
results.put(idempotencyKey, new StoredResult(req, response));
return response;
}
}Output
201 {paymentId: pay_1001, orderId: 1001, amount: 2998}
(client timed out before seeing the response, so it retries)
201 {paymentId: pay_1001, orderId: 1001, amount: 2998} [replayed]
422 {error: idempotency key reused with a different request}
201 {paymentId: pay_1002, orderId: 1002, amount: 1499}
card charged 2 times, total Rs.4497Idempotency Keys in Spring Boot
@Entity
@Table(name = "idempotency_record",
uniqueConstraints = @UniqueConstraint(columnNames = "idempotency_key"))
public class IdempotencyRecord {
@Id @GeneratedValue private Long id;
@Column(name = "idempotency_key", nullable = false) private String key;
private String requestHash; // SHA-256 of the request body
private int responseStatus;
@Lob private String responseBody;
private Instant createdAt; // purge after 24 hours
// constructor, getters
}
@RestController
@RequestMapping("/api/payments")
@RequiredArgsConstructor
public class PaymentController {
private final IdempotencyRecordRepository records;
private final PaymentService payments;
private final ObjectMapper json;
@PostMapping
@Transactional
public ResponseEntity<String> pay(@RequestHeader("Idempotency-Key") String key,
@Valid @RequestBody PaymentRequest req) throws Exception {
String hash = Hashing.sha256(json.writeValueAsString(req));
Optional<IdempotencyRecord> existing = records.findByKey(key);
if (existing.isPresent()) {
IdempotencyRecord r = existing.get();
if (!r.getRequestHash().equals(hash)) {
return ResponseEntity.unprocessableEntity().body("{\"error\":\"key reused with a different body\"}");
}
return ResponseEntity.status(r.getResponseStatus()).header("Idempotent-Replayed", "true").body(r.getResponseBody());
}
PaymentResponse result = payments.charge(req); // calls Razorpay with the same key
String body = json.writeValueAsString(result);
records.save(new IdempotencyRecord(key, hash, 201, body)); // unique constraint stops a parallel duplicate
return ResponseEntity.status(201).body(body);
}
}Example
The second call with the same key does not charge again:
curl -X POST localhost:8084/api/payments \
-H "Idempotency-Key: 7f3c9a2e-order-1001" -H "Content-Type: application/json" \
-d '{"orderId": 1001, "amount": 2998.00, "method": "UPI"}'
# same command again (client retry)Example output
HTTP/1.1 201 Created
{"paymentId":"pay_NqX81kLm2","orderId":1001,"amount":2998.00,"status":"CAPTURED"}
HTTP/1.1 201 Created
Idempotent-Replayed: true
{"paymentId":"pay_NqX81kLm2","orderId":1001,"amount":2998.00,"status":"CAPTURED"}Note: Two retries can arrive at the same millisecond. The unique constraint on the key makes the second insert fail, so only one charge wins. Some teams first insert an IN_PROGRESS row and return 409 Conflict to a parallel duplicate.
Idempotent Kafka Consumers
For events, use the event ID as the key: record every processed event ID in the same transaction as the business change, and skip IDs you have already seen.
@KafkaListener(topics = "order-events", groupId = "inventory-service")
@Transactional
public void onOrderPlaced(OrderPlaced event) {
if (processedEvents.existsById(event.eventId())) {
log.info("Duplicate event {} ignored", event.eventId());
return;
}
inventoryService.reserve(event.orderId(), event.items());
processedEvents.save(new ProcessedEvent(event.eventId(), Instant.now())); // PK = eventId
}Tip: Natural idempotency is even better when possible: "set order 1001 status to PAID" is safe to repeat; "add ₹2,998 to the wallet" is not. Prefer absolute updates over increments, or guard increments with a processed-events table.
Exercise
Exercise: In the idempotent payment example, keys stay in memory forever. Change StoredResult to also store a createdAtMinute (an int), and make pay treat a key older than 1440 minutes (24 h) as new. Pass the current minute as a parameter so the program stays deterministic.
Show answer
record StoredResult(PayRequest request, Response response, int createdAtMinute) {}
Response pay(String key, PayRequest req, int nowMinute) {
StoredResult previous = results.get(key);
if (previous != null && nowMinute - previous.createdAtMinute() > 1440) {
results.remove(key); // expired: forget it
previous = null;
}
if (previous != null) { /* same replay / 422 logic as before */ }
String paymentId = gateway.charge(req.amount());
Response response = new Response(201, "{paymentId: " + paymentId + "}");
results.put(key, new StoredResult(req, response, nowMinute));
return response;
}