Service Discovery in System Designing

In a microservices architecture, services scale horizontally by spinning up and shutting down dynamically. This means instance IP addresses and port numbers change constantly. Hardcoding these endpoints inside your application code is impossible. Service Discovery is the mechanism that automates this tracking, allowing services to find and communicate with each other dynamically. 1. The Core Components Every service discovery architecture relies on three distinct elements: The Service Registry: A centralized database containing the network locations (IP addresses and ports) of all active service instances. This registry must be highly available and strictly consistent (often using consensus algorithms like Raft). Service Registration: When a microservice instance spins up, it automatically calls the Service Registry to announce its presence: "I am the payment-service , and I am listening at 10.0.1.55:8080 ." Service Discovery: When order-service needs to make an API call to payment-service , it queries the registry to obtain a valid, working IP address. 2. Discovery Patterns: Client-Side vs. Server-Side The major architectural choice is deciding where the routing intelligence lives. A. Client-Side Discovery The client microservice is directly responsible for looking up the registry and deciding which instance to call. order-service queries the Service Registry for a list of all healthy payment-service instances. The registry returns a list of IPs (e.g., Instance 1, Instance 2, Instance 3). order-service uses a built-in client-side load-balancing library (like Netflix Ribbon) to select one of the instances and calls it directly. Example Tool: Netflix Eureka . Pros: Fewer network hops; no single point of failure in the direct request path. Cons: Couplage. You must implement discovery libraries in every single language/framework used across your tech stack. B. Server-Side Discovery The client microservice makes a standard call to a proxy router, completely unaware of how the service pool is structured. order-service simply sends its request to a centralized proxy/load balancer: http://payment-service/api/charge . The Load Balancer queries the Service Registry behind the scenes to see where the healthy payment instances are. The Load Balancer forwards the request to a valid instance. Example Tools: Consul, AWS ALB, Envoy (used in Service Meshes like Istio), Kubernetes DNS . Pros: Clean abstraction. Microservice code stays lightweight and language-agnostic. Cons: Introduces an extra network hop and a critical infrastructure component to manage. 3. Continuous Reliability: Health Checks Services don't just spin down cleanly; they crash, drop connections, freeze up due to out-of-memory errors, or suffer from database timeouts. If the Service Registry continues to hand out the IP of a dead server, your system will experience cascading failures. To prevent this, the discovery mechanism enforces Health Checks : Heartbeats: The service instance must continuously ping the registry every few seconds (e.g., "I am still alive" ). If the registry misses a predefined number of heartbeats, it assumes the node is dead. Active Polling: The Service Registry actively hits a designated endpoint on the microservice (like /healthz ) at regular intervals. Self-Healing: If a check fails (e.g., returning a 500 status code instead of a 200 OK ), the registry instantly evicts (removes) that instance from its routing table, shielding it from future traffic until it recovers. 4. System Design Takeaway If you are deploying your system inside an orchestrated container ecosystem like Kubernetes , service discovery is built right into the platform. Kubernetes manages service registries natively via its internal CoreDNS system, assigning permanent internal domain names to logical server pools and handling container health checks automatically via readiness and liveness probes.