01What operations do clients actually get?
GET, SET with a per-key TTL (time-to-live — the key erases itself after the deadline), and DELETE — plus atomic increment for counters. A tiny surface is the point: every operation has one predictable cost.
Premium preview
Distributed Cache
Design a distributed in-memory cache service at Redis/Memcached scale that serves roughly 1M QPS at p99 under 5ms.
00
Practice checkpoints
The requirements are open as a taste. From the numbers onward, the full guide opens in the app.
- 01Clarify scope
- 02Requirements + scale
- 03API + data modelUnlocks in the app
- 04Draw architectureUnlocks in the app
- 05Deep diveUnlocks in the app
- 06Trade-off decisionUnlocks in the app
01
Requirements that shape the design
Do not only state requirements. Ask for them. Each card pairs the design constraint with a clarification question you can say out loud before drawing the architecture.
Functional requirements
02Who decides which node owns a key — the client or a router?
A smart client library holds the hash ring and talks straight to the owning shard — one network hop, no proxy in the hot path.
03Memory fills up — do writes start failing?
No: the node evicts per policy (sampled LRU by default) to stay under a hard memory ceiling — writes always land, older keys pay for it.
04A node dies — do its keys just disappear?
Its replica is promoted within seconds and clients receive a new ring version; the freshest un-replicated writes are lost, and the SLA says so out loud.
05One key goes viral — is that the caller’s problem or ours?
Ours: shards count per-key traffic, and a detected hot key is served from every replica plus a short client-local cache, so one shard never melts alone.
06A popular key expires and thousands of clients miss at the same instant?
The service grants a per-key lease on the first miss: one caller rebuilds from the origin while the rest wait briefly or take the stale value.
Out of scopeDurability as a system of record (the origin database owns truth; the cache may forget) · Cross-region active-active replication and conflict resolution · Rich queries — secondary indexes, scans, or transactions across keys
Non-functional requirements
01What latency are we promising, and measured where?
p99 stays under 5 ms client-observed: the in-node budget is sub-millisecond per operation — all RAM, one hop, no disk or cross-AZ (between datacenter availability zones) call on the hot path.
02How much traffic, and how does capacity grow?
Sustains 1M QPS and grows linearly — capacity is shard count times per-shard ops, with nothing centralized on the request path.
03What does scaling the cluster do to the hit rate?
Resizing the cluster moves only about 1/N of the keys, so adding a node costs a ~2% hit-rate dip, never a cache flush.
04When things break, do we choose consistency or availability?
Availability over consistency: async replication, stale reads allowed, failover in seconds — and the last ~1 s of writes on a dead primary may vanish, stated in the SLA.
05Can a node ever run out of memory?
A node never OOMs or swaps: a hard maxmemory ceiling plus eviction keeps memory bounded, because one page of swap ends the sub-millisecond story.
Keep asking — the interview is a conversation
Real interviews probe far more than a tidy list. These are the scope questions that separate candidates who interrogate the problem from those who recite it.
- Is every key reconstructable from the origin, or are some keys the only copy of their data?
- What read/write mix and value-size distribution am I designing for — a few 10 MB values change every budget?
- How much extra QPS can the origin absorb when a shard dies and its whole key range misses at once — and do we jitter TTLs so mass expiry cannot cause the same storm?
- Do any callers need read-your-writes, or is a few seconds of staleness always acceptable?
- Are clients homogeneous enough for a smart client library, or do polyglot callers force a proxy tier?
01
Unlock the full playbook for Distributed Cache
Numbers, architecture diagram, API and data model, deep dives, expected topics, self-check, whiteboard starter, and common mistakes unlock inside the app.
Numbers that force architecture decisions
Locked in the app
Architecture path
Locked in the app
API and data model
Locked in the app
Deep dive directions
Locked in the app
+Series