R1.2 — Where state lives: stateful services, stateless services and hybrid systems

Part of R1. The story in R1 relied on one property: any copy of the payment service could take the request. This part explains where state lives so that this is true, and what to do when it cannot be.

Every service has state. The design question is never "does it have state" — it is where the state lives between requests, and what happens to it when the machine holding it dies.

Stateful: the server remembers

A stateful server keeps the client's state in its own memory between requests: the session, a game position, the open order book.

You pick this when latency is non-negotiable. A multiplayer game updates its match state 50 to 60 times a second; a chat server holds millions of open WebSocket connections; an exchange matching engine matches in microseconds. A network hop to a shared store on every one of those would be the bottleneck.

The price is that the client must keep coming back to the same server. That is sticky sessions (session affinity) at the load balancer, done one of three ways:

And the drawbacks follow from the pinning:

• Memory and CPU per session, on every server.
• Hard to scale out. A new server takes no load from existing sessions.
• A crash loses every session on that server. The state existed in one place.
• Deployments are complex. You drain sessions before you can restart.
• Replicating sessions to survive crashes brings back every consistency problem in R0.

Stateless: the server keeps nothing between requests

A stateless server keeps nothing in its own memory between one request and the next. Whatever the next request needs about this client — who they are, which step of a flow they are on — comes from somewhere other than the server that happens to receive it.

Any server can serve any request. That one property buys almost everything:

• Scale-out is instant: add a copy, it takes traffic.
• The load balancer can use the simplest algorithms — round robin, least connections.
• It fits containers, autoscaling and serverless, where instances appear and vanish.
• A crashed instance loses nothing but the requests it was in the middle of.

What "stateless" does not mean is "has no data". A stateless payment service writes payments to Postgres on every call, and it is still stateless: the term is about the conversational state between one client's requests — the session — not about whether the system persists anything. The Twelve-Factor App puts it the standard way: processes are stateless and share nothing, and any data that must persist lives in a stateful backing service such as a database.

So for a stateless service the only design question left is where the session lives, and there are two answers:

Both are stateless services. The second does not make the server stateful; it moves the session off the server into a store every instance can reach. Real systems usually combine them: a short-lived JWT so most requests need no lookup, and a refresh token or session record in the store so access can be revoked.

Keeping the session in a shared store

The shared store is where most of the operational cost of a stateless design ends up, so it is worth naming:

The practices that make it safe: run the store redundantly, back it up, pool connections to it, and put a TTL on every session so abandoned ones expire on their own.

Store choice is the usual trade: Redis gives microsecond latency, native TTL and replication; a cloud-managed Redis removes the operating burden; SQL is durable but slower; MongoDB or DynamoDB give a flexible shape.

Hybrid systems: stateful and stateless services side by side

Stateful and stateless describe a service. "Hybrid" describes a system that runs both kinds deliberately: a large stateless tier that absorbs scale, and a few stateful services where holding state in memory is the whole point.

Start stateless; keep the session in the request or a shared store; add a stateful service only for a clear real-time case, and keep it small, isolated and rebuildable.

The rules that make a hybrid system work:

• Keep the stateful part narrow. It does the one job that needs memory; everything else stays stateless around it.
• Route to it by ownership, not by stickiness. A stateful service owns entities — a match, a chat room, an order book, a document — and requests find the owner through consistent hashing or a registry (R10), not through a sticky cookie at the load balancer.
• Give it a rebuild path. A journal, a checkpoint or a changelog, so a crashed instance comes back with its memory.

Where real systems put their state

The categories are easiest to defend with examples. Learn a few from each table.

Stateless, session in the request or none at all — every instance is interchangeable.

Stateless, session or shared counters in a fast store — the servers still hold nothing.

Stateful — the state in memory is the product.

Read the third column of the stateful table again. Every stateful system that survives in production has a way to rebuild its memory — a journal, a checkpoint, a changelog. Stateful does not mean "the state exists only in RAM"; it means the hot copy is in RAM and the durable copy is somewhere you can replay from.

Hybrid systems — a stateless tier around a few stateful services.

The trap to name: "our login is stateless because we use JWTs." A pure JWT cannot be revoked before it expires, because nobody is asked. Banking logins pair a short-lived JWT with a refresh token held server-side precisely so that a stolen phone can be cut off. The service is still stateless — the revocable part of the session lives in a shared store — and saying so is the senior answer.

Before reading on, step through the walkthrough below. It starts with a stateful server behind sticky sessions, crashes it, and shows the sessions vanishing. Then watch the same traffic served by stateless JWTs, and finally by stateless servers over an external session store — and note which step survives the crash and why.

The defensible position: name the state, then say where it lives. "The payment API is stateless; sessions and refresh tokens are in Redis with a TTL; the matching engine is stateful, journaled, and rebuilt by replay." That sentence describes a hybrid system: stateless services everywhere except the one place where memory is the product.

What to say in the interview

"The payment API is stateless — any instance serves any request, and the state lives in the payments database, the idempotency table and a Redis session store with a TTL. I keep stateful components, like a matching engine, for genuine real-time cases, and journal them so they can be rebuilt after a crash."