R1.3 — Gateways: reverse proxies, load balancers and API gateways at the front door

Part of R1. In R1's story, the request met a gateway before any service. Each kind of gateway accepts the client's connection and forwards the request on, so R1.1's picture of connections sits underneath everything here. "Gateway" covers several different jobs, and interviewers notice when they are blurred.

The kinds of gateway

The part people miss about the message gateway: it is not a queue. It transforms and forwards; a queue stores. In payments it is the piece that turns your JSON into the XML a rail expects — R25 covers the message formats.

Reverse proxy, load balancer, API gateway: one shape, three jobs

Three names get used for "the thing in front of the service", and they are easy to blur because physically they are the same kind of box: a server that accepts the client's connection and forwards the request on. What separates them is the job, and how much of the request each one reads to do it.

• A reverse proxy is the shape itself: any server that receives requests on behalf of your servers and forwards them, so clients never talk to a backend directly. (A forward proxy is the mirror image: it acts for clients, like a corporate egress proxy.)
• A load balancer is a reverse proxy, or a network device, whose job is to spread traffic across identical copies of one service and stop sending to the ones that are sick.
• An API gateway is a reverse proxy whose job is to manage the API at the edge: who the caller is, what they may call, how often, and which of many services handles it.

Three questions tell them apart:

"Something in front of multiple instances of the same service" is the definition of a load balancer. The other two do not need multiple instances at all.

Read them as a ladder of how much each one understands:

A single NGINX in front of a single Spring Boot instance is a perfectly normal reverse proxy. It terminates TLS, compresses responses, serves static files and holds slow clients so they do not tie up application threads, with no load balancing involved. Add a second instance and one more config block turns it into a load balancer. R14 covers the load balancer's algorithms, health checks and the L4 versus L7 choice in depth.

The trap to name: products overlap, the jobs do not. NGINX and Envoy can play all three roles; ALB does path routing, which is a little of a gateway's job; AWS API Gateway does some balancing. In an interview, name the job first — "I need per-client rate limiting and OAuth validation at the edge" — and the product second.

What an API gateway does

The API gateway takes every concern that would otherwise be copy-pasted into each service and does it once:

• Authentication — verify the token or API key.
• Authorization — does this caller have the scope for this route and this tenant?
• Rate limiting and quotas — per consumer, after authentication, so you limit the customer rather than an IP.
• Routing — which service owns /v1/payments.
• Protocol translation — REST outside, gRPC inside.
• Response aggregation — one client call fanned out to several services and merged.
• Logging, metrics and tracing — a request ID stamped at the door.

These run as an ordered filter chain, and the order is the design: cheap rejections before expensive work.

flowchart LR
  T["TLS"] --> I["Request ID"]
  I --> W["WAF / size"]
  W --> R["Route match"]
  R --> A["Authn"]
  A --> Z["Authz"]
  Z --> L["Rate limit"]
  L --> X["Transform"]
  X --> U["Upstream call"]
  U --> G["Async log"]

Bad traffic is dropped before the body is parsed; unknown routes get 404 before anyone verifies a token; rate limiting comes after authentication so it knows who to limit. The upstream call is wrapped in timeouts, retries on idempotent calls only, and a circuit breaker (R14). Logging is asynchronous, because the gateway must never block on its own telemetry.

Two design points worth having ready, both covered in full in R17:

• Verify tokens locally. The gateway checks a JWT's signature against the identity provider's cached public keys — pure CPU, no network call. The price is revocation, which is why access tokens are short-lived and backed by a server-side refresh token in a shared store, as described earlier in this section.
• Separate the data plane from the control plane.

The control plane can fail; the data plane must not notice. Nodes keep serving on their last good config — the principle called static stability.

What it costs

An API gateway is not a load balancer.

The gateway decides which service a request belongs to and whether it is allowed in; the load balancer decides which instance of that service takes it. A gateway usually has a load balancer in front of it and balances across upstream instances itself, but neither replaces the other — the stacking below shows how they fit together.

Run a few requests through the pipeline below. Send a normal request and watch it pass authentication, rate limiting, routing and aggregation. Then send one without a token and one in a burst, and notice where each is rejected — a 401 never costs a rate-limit check, and a 429 never reaches a service.

Which one, when

How production systems stack them

The gateway is a service like any other: it runs as several instances across zones, so something has to balance traffic across the gateway itself. That is why real systems stack these layers, each one balancing the layer behind it.

Why an L4 balancer in front of an L7 tier? The L7 tier is many instances, and the L4 layer gives it one stable address, absorbs huge connection counts cheaply, can pass TLS through untouched, and can expose fixed IPs — which a corporate client's firewall team will ask for.

Follow one partner payment through the full stack below. Watch the card underneath: it shows which layers of the request each hop actually reads. The L4 balancer sees only addresses, the gateway is the first to know who is calling, and the internal balancer chooses a pod while skipping one that is draining for a deploy. The last step shows the subsets real systems use when they do not need every layer.

Three mistakes interviewers listen for:

• Business logic in the gateway. "Is the token valid and is this client under quota" is edge policy. "Does this account have the funds, is this payment within its limit" belongs to the payment service, which is the only place that can answer it correctly.
• Internal traffic through the edge gateway. Hairpinning east-west calls through the front door turns it into a bottleneck and a single point of failure for the whole platform. Inside, use client-side balancing or a mesh.
• Sticky sessions to paper over state. Affinity at the load balancer is a symptom of a stateful service. Keep the service stateless, as R1.2 argues, and any healthy instance will do.

The defensible position: the gateway centralises cross-cutting concerns in a cheap, ordered chain, stays stateless and fast, and never holds business logic. Reverse proxy is the shape; load balancing spreads one service across its copies; the gateway applies API policy across many services. In production they stack — an L4 balancer for a stable entry point, a replicated gateway behind it, and per-service balancing or a mesh behind that.

What to say in the interview

"An L4 load balancer gives the edge a stable address; behind it a replicated gateway does TLS, authentication, rate limiting and routing in a cheap-first filter chain, with the data plane independent of the control plane, and an internal load balancer or mesh picks the instance — the gateway decides which service, the balancer decides which copy."