Heartbeats and Self-Preservation

Leases and Heartbeats

Registration in Eureka is a lease. Each client renews it with a heartbeat every 30 seconds. If the server gets no renewal for 90 seconds, the lease expires and the eviction task removes the instance. A clean shutdown deregisters immediately.

eureka:
  instance:
    lease-renewal-interval-in-seconds: 30     # heartbeat interval (default 30)
    lease-expiration-duration-in-seconds: 90  # expire after (default 90)

Example

A simulated clock shows when a crashed order-service instance disappears:

import java.util.*;

public class Main {
    public static void main(String[] args) {
        Registry registry = new Registry(90);
        registry.renew("order-service#1", 0);
        registry.renew("order-service#2", 0);
        registry.renew("product-service#1", 0);

        for (long t = 30; t <= 150; t += 30) {
            registry.renew("order-service#1", t);
            registry.renew("product-service#1", t);
            if (t <= 30) registry.renew("order-service#2", t); // #2 crashes after t=30s
            registry.evictExpired(t);
            System.out.println("t=" + t + "s registry=" + registry.names());
        }
    }
}

class Registry {
    private final long leaseSeconds;
    private final Map<String, Long> lastRenewal = new TreeMap<>();

    Registry(long leaseSeconds) { this.leaseSeconds = leaseSeconds; }

    void renew(String instance, long now) { lastRenewal.put(instance, now); }

    void evictExpired(long now) {
        lastRenewal.entrySet().removeIf(e -> {
            long silent = now - e.getValue();
            boolean expired = silent > leaseSeconds;
            if (expired) System.out.println("  EVICT " + e.getKey() + " (no heartbeat for " + silent + "s)");
            return expired;
        });
    }

    Set<String> names() { return lastRenewal.keySet(); }
}

Output

t=30s registry=[order-service#1, order-service#2, product-service#1]
t=60s registry=[order-service#1, order-service#2, product-service#1]
t=90s registry=[order-service#1, order-service#2, product-service#1]
t=120s registry=[order-service#1, order-service#2, product-service#1]
  EVICT order-service#2 (no heartbeat for 120s)
t=150s registry=[order-service#1, product-service#1]

Self-Preservation Mode

If a network problem stops many heartbeats at once, Eureka cannot tell "instances died" from "my network is broken". When renewals fall below 85% of the expected number, Eureka enters self-preservation: it stops evicting anyone. Better to keep a few dead entries than to wipe the whole registry.

Example

The dashboard then shows a red warning:

open http://localhost:8761

Example output

EMERGENCY! EUREKA MAY BE INCORRECTLY CLAIMING INSTANCES ARE UP WHEN THEY'RE NOT.
RENEWALS ARE LESSER THAN THRESHOLD AND HENCE THE INSTANCES ARE NOT BEING EXPIRED JUST TO BE SAFE.

Tip: On a laptop you often see this warning because you start and stop services all the time. It is fine. You may set eureka.server.enable-self-preservation: false in dev only; keep it on in production.