Microservices I — Concepts, Gateway, IPC & Discovery

A monolith is simple until it isn't. This chapter builds the vocabulary for why large teams eventually break an application into independently deployable services — the Scale Cube, database-per-service, the API Gateway — and then tackles the two hardest new problems microservices create: how services talk to each other across a network instead of a function call (inter-process communication), and how they find each other when instances are constantly starting, stopping and moving (service discovery). One running example carries the whole chapter: an online shopping application with Product, Order, Customer, Shipping and Notification services.

From monolith to microservices

The monolith

A monolithic application is one that, however modular its internal code structure, is built, packaged and deployed as a single unit. Our shopping application — product catalog, shopping cart, checkout, order history, customer accounts, shipping tracking, email notifications — all lives in one codebase, compiles to one deployable artifact (a WAR file on Tomcat, a single ASP.NET Core project publishing one binary, whatever your stack is), and runs as one process. Scaling it means running several identical copies of that one process behind a load balancer.

Monoliths aren't a mistake — they're the right starting point for most projects. They're simple to build (your IDE, debugger and build tooling are all designed around "one project"), simple to test end-to-end (spin up the one app, drive the UI with Selenium), simple to deploy (copy one artifact to a server), and you can even scale them horizontally by running clones behind a load balancer. Most successful microservices systems — including the companies that popularised the pattern — started as monoliths.

The trouble starts as the shopping application grows. That's when the following drawbacks, which are invisible at small scale, start to dominate:

What is a microservice?

The Microservices Architecture pattern structures an application as a set of small, independently deployable services, each built around a business capability. Instead of one shopping-app process, you get a Product Catalog Service, an Order Service, a Customer Service, a Shipping Service and a Notification Service — each independently buildable and deployable, often owned by a separate team. Most services expose a REST (or similar) API consumed by other services or by clients; some communicate asynchronously via messages instead. For example, the Order Service asks the Customer Service to reserve credit; the Shipping Service tells the Notification Service to email a tracking link.

ShoppingApp.sln
 ├─ ProductCatalog/
 ├─ Orders/
 ├─ Customers/
 ├─ Shipping/
 ├─ Notifications/
 └─ SharedDb.mdf        ← one schema, everyone reads/writes it

Deploy: publish ShoppingApp.dll → copy to server → restart

product-catalog-service/   → ProductDb
order-service/              → OrderDb
customer-service/           → CustomerDb
shipping-service/           → ShippingDb
notification-service/       → (no DB — stateless)

Deploy: each service ships independently, on its own schedule

The Scale Cube

The Scale Cube (from Abbott & Fisher's The Art of Scalability) is a mental model with three independent axes along which you can scale any application. It's a favourite exam question because one diagram cleanly answers "what actually is microservices architecture, scaling-wise?" — and the answer is precise: microservices architecture is Y-axis scaling.

Application vs. database relationship

Microservices change the relationship between application and database more than almost anything else about them. Rather than sharing one database schema, each service owns its own schema — and no other service may reach into it directly. All access to a service's data goes through that service's API.

This is, on the surface, odd: it's the opposite of an "enterprise-wide data model," and it causes real duplication — Order Service might cache a copy of the customer's name and shipping address rather than joining across to Customer DB every time. But database-per-service is essential if you actually want the benefits of microservices, because it's what makes services loosely coupled: Customer Service can change its schema, switch database engines, or add a column, without ever coordinating with the Order team. If they shared one schema, you'd be back to a monolith wearing a microservices costume.

Microservices vs. SOA

On the surface, Microservices Architecture looks a lot like SOA (Service-Oriented Architecture, covered in Chapter 8) — both structure an application as a set of services. The honest way to describe microservices is: SOA without the enterprise commercialisation baggage — no WS-* web service specifications, no Enterprise Service Bus (ESB).

Benefits and drawbacks

Be honest with yourself about this trade — nothing on the benefits side is free, and nothing on the drawbacks side is a dealbreaker for the right kind of application.

The API Gateway

The problem: direct client-to-microservice

Consider a mobile client rendering a product details page. With the monolith, this is one call: GET api.company.com/productdetails/{productId} — a load balancer routes it to any instance, which queries several tables and returns one response.

With microservices, that data is now spread across Product Catalog, Reviews, Recommendations, Pricing and Shipping services. In theory, the client could call each one directly — each service exposed at its own public endpoint (https://reviews.api.company.com, etc.):

This breaks down for two concrete reasons:

• Mismatch between client needs and fine-grained APIs. The client wants one page; it now has to make several requests and stitch the results together itself — slow and chatty, especially on mobile networks.
• Web-unfriendly protocols leak out. One internal service might use a binary RPC protocol like Thrift, another an AMQP message queue — neither is browser- or firewall-friendly. Outside the firewall you want HTTP/WebSocket; internal services are free to use whatever's efficient.
• Hard to refactor. If the client hard-codes calls to five specific services, splitting or merging services later means changing every client — mobile app releases are slow and painful to ship.

The solution: API Gateway

An API Gateway is a single entry point that sits between clients and the microservices. It plays the same role as the Facade pattern plays for objects — one simplified interface hiding a complex set of subsystems behind it — just at the scale of a whole system instead of a class.

The Gateway's core responsibilities: request routing (send this path to that service), composition (call several services and merge results into the one response the client actually wants), and protocol translation (HTTP/WebSocket outside, Thrift/AMQP/whatever inside). It often also picks up cross-cutting concerns: authentication, monitoring, load balancing, caching, and request shaping. Many real systems use a Backend-for-Frontend (BFF) variant — a separate, thinner gateway per client type (mobile app, web app, partner API), each tailored to what that client needs.

Chapter 11 shows the gateway as code — both the routing half and the aggregating half — and explains why an aggregating gateway is the component most likely to need a circuit breaker.

Inter-process communication (IPC)

In a monolith, components call each other via language-level method calls — fast, reliable, and (mostly) all-or-nothing. In microservices, every one of those calls potentially crosses a network to a different process, on a different machine, that might be slow, unreachable, or simply down. This single fact — replacing function calls with network calls — is the root of almost every new problem microservices introduces.

Client-service interaction styles

Two independent dimensions describe any interaction: how many participants handle a request (one-to-one vs one-to-many), and whether the caller blocks (synchronous vs asynchronous).

In the shopping app: the Order Service calling Customer Service to "reserve credit and reply now" is request/response. The Order Service telling Notification Service "order shipped, no reply needed" is a notification. The Order Service publishing an OrderCreated event that Customer Service, Analytics Service and Fraud-Check Service all independently consume is publish/subscribe — the pattern Chapter 10 builds an entire architecture around.

Synchronous request/response vs. asynchronous messaging

Messages travel over channels. A point-to-point channel delivers a message to exactly one consumer (for one-to-one interactions). A publish-subscribe channel delivers a copy of the message to every attached consumer (for one-to-many interactions). Popular messaging middleware includes RabbitMQ, Apache Kafka, ActiveMQ and NSQ.

Handling partial failures

The defining new problem of distributed IPC: any call can fail for reasons a local method call never could — the network drops the packet, the other service is overloaded, or it's mid-deploy and simply not there. A monolith's method call never has to think about this. A microservice's remote call always does.

// No explicit timeout: this falls back to HttpClient's default
// of 100 seconds — an eternity to wait on a hot request path.
var response = await _http.GetAsync($"/products/{productId}");
response.EnsureSuccessStatusCode();

public class ProductServiceClient
{
    private readonly HttpClient _http;

    public ProductServiceClient(HttpClient http)
    {
        _http = http;
        _http.Timeout = TimeSpan.FromSeconds(2); // never wait forever
    }

    public async Task<ProductDetails?> GetProductAsync(string productId, CancellationToken ct)
    {
        const int maxAttempts = 3;
        for (int attempt = 1; attempt <= maxAttempts; attempt++)
        {
            try
            {
                using var response = await _http.GetAsync($"/products/{productId}", ct);

                if (response.IsSuccessStatusCode)
                    return await response.Content.ReadFromJsonAsync<ProductDetails>(ct);

                bool transient = (int)response.StatusCode >= 500
                    || response.StatusCode == HttpStatusCode.RequestTimeout      // 408
                    || response.StatusCode == HttpStatusCode.TooManyRequests;    // 429

                if (!transient)
                    break; // 401, 404, ... — retrying won't help; go to fallback
            }
            catch (HttpRequestException)
            {
                // network-level failure: transient, worth retrying
            }
            catch (TaskCanceledException) when (!ct.IsCancellationRequested)
            {
                // HttpClient.Timeout fired (not our caller cancelling): transient
            }

            if (attempt < maxAttempts)
                // linear backoff; production adds jitter and honours Retry-After
                await Task.Delay(TimeSpan.FromMilliseconds(200 * attempt), ct);
        }
        return null; // caller shows "temporarily unavailable" instead of hanging
    }
}

The circuit breaker pattern, properly

A circuit breaker wraps every outgoing call and moves through three states:

• Closed — the normal state. Requests go through; the breaker just counts successes and failures.
• Open — tripped. Every call fails immediately, without even attempting the network call — "fail fast." This protects both the caller (no wasted waiting) and the struggling downstream service (no pile-up of retries making things worse).
• Half-open — after a cooldown period, the breaker cautiously lets exactly one trial request through. Success ⇒ back to Closed. Failure ⇒ back to Open, cooldown restarts.

public enum CircuitState { Closed, Open, HalfOpen }

public class CircuitBreaker
{
    private CircuitState _state = CircuitState.Closed;
    private int _consecutiveFailures;
    private DateTime _openedAt;
    private readonly int _failureThreshold;
    private readonly TimeSpan _openDuration;

    public CircuitBreaker(int failureThreshold = 3, TimeSpan? openDuration = null)
    {
        _failureThreshold = failureThreshold;
        _openDuration = openDuration ?? TimeSpan.FromSeconds(30);
    }

    public async Task<T> CallAsync<T>(Func<Task<T>> action, Func<T> fallback)
    {
        if (_state == CircuitState.Open)
        {
            if (DateTime.UtcNow - _openedAt < _openDuration)
                return fallback();          // fail fast, don't even try
            _state = CircuitState.HalfOpen; // cooldown elapsed, allow a trial
        }

        try
        {
            var result = await action();
            _consecutiveFailures = 0;
            _state = CircuitState.Closed;
            return result;
        }
        catch
        {
            _consecutiveFailures++;
            if (_state == CircuitState.HalfOpen || _consecutiveFailures >= _failureThreshold)
            {
                _state = CircuitState.Open;
                _openedAt = DateTime.UtcNow;
            }
            return fallback();
        }
    }
}

IPC technologies

Message formats are a separate choice from the transport: human-readable text (JSON, XML) is easy to debug; binary formats (Avro, Protocol Buffers) are smaller and faster to (de)serialise, at the cost of needing a schema and tooling to read them.

The ancestors: RMI vs CORBA (and their modern heir, gRPC)

Before REST and gRPC, distributed objects talked via RMI (Java Remote Method Invocation) and CORBA (Common Object Request Broker Architecture). The module — and the exam — asks you to compare them, and the comparison is genuinely instructive: every axis below is a question you should ask of any IPC technology.

Evolving APIs

A service's API changes over time, but you can't force every client to upgrade the moment it does. Design for the robustness principle: be liberal in what you accept, conservative in what you send. Minor, backward-compatible changes (adding an optional field) should never break old clients — the service supplies sensible defaults for anything missing, and clients ignore any extra fields they don't recognise.

For breaking changes, embed a version in the URL (/v1/products/{id} vs /v2/products/{id}) and keep serving both for a deprecation window — either one service instance handling multiple versions, or separate instances per version behind the gateway.

Service registration & discovery

In a traditional application on physical hardware, a service's network location (IP + port) barely changes. In a cloud-based microservices application it changes constantly — autoscaling adds and removes instances, deployments replace them, failures kill them, and container schedulers move them between hosts. Your client code can no longer hard-code an address; it needs a proper service discovery mechanism.

Underneath both discovery patterns below is a service registry: a highly-available database of currently-live service instances and their locations. An instance registers when it starts, is removed when it terminates, and typically refreshes its registration with a periodic heartbeat.

Client-side discovery

Simple — no extra moving part besides the registry itself — and since the client sees all instances, it can make smart, application-specific load-balancing decisions (e.g. consistent hashing). The drawback: it couples every client to the registry, so you must implement discovery logic in every language/framework your clients use.

Server-side discovery

Discovery logic is centralised in the load balancer, so clients need zero discovery code at all — and many cloud platforms (AWS ELB, Kubernetes Services) give you this for free. The drawback: unless the deployment platform already provides it, the load balancer is yet another highly-available component you must build and run.

The service registry

Netflix Eureka is the textbook example: a REST API for registering and querying instances. An instance POSTs to register, sends a PUT heartbeat every 30 seconds to stay registered, and is removed via an explicit DELETE or by simply timing out. Clients GET the current list. Other well-known registries: etcd (used by Kubernetes and Cloud Foundry), Consul (adds health checking), and Apache ZooKeeper (general-purpose distributed coordination, born out of the Hadoop ecosystem).

Self-registration vs. third-party registration

Chapter 9 quiz