Caching System Design Questions That Separate Senior From Staff
Related service: Mock interviews
In a caching system design interview, the cache itself scores nothing. Every candidate adds one.
The score comes from 6 questions. Wrong value. Cold cache. A hot key expiring everywhere at once.
What you evict, where the cache lives, how you know it is working.
On this page
Senior candidates answer 2 or 3 of the 6. Staff candidates answer all 6 unasked.
The cache itself goes in during the first 5 minutes. The 6 caching system design questions follow.
1. What happens when the cached value is wrong?
A cache is a second copy of the truth, and 2 copies can disagree.
Four caching strategies for the disagreement window:
Time to live only. Wrong for up to 60 seconds. Fine for a catalogue, not for an account balance.
Invalidate on write. Delete the key when the row changes. Now describe the race.
A read misses and fetches the old value. It writes that into the cache after the invalidation arrived. Stale data, no expiry.
Write through. Update cache and database in the same path. Consistent, slower, and it still fails when the second write fails.
Versioned keys. A new value is a new key; the old one ages out. Costs memory, removes the race.
“We invalidate on write” is correct and incomplete. The follow-up is always “what if the invalidation is lost”.
Naming the race unprompted is a staff signal.
2. What happens when it is cold?
Every cache is empty at some point: a deploy, a restart, a failover.
If your database cannot survive the full read load, nothing survives a cache restart.
A cold cache sends every read to the database at once. Plan for that moment:
- Warm the cache before taking traffic. Join the load balancer once the hit rate is above a threshold.
- Restart nodes in a rolling order. The miss traffic is a fraction, never all of it.
- Rate limit the miss path. A cold cache then degrades latency instead of taking down the database.
3. What happens when a hot key expires everywhere at once?
A thousand requests miss on the same key in the same millisecond. All hit the database. The stampede.
A lock on the miss path. The first request fetches, the rest wait. Now you have a lock to reason about.
Serve stale while refreshing. Return the expired value and refresh in the background. The right default for most read paths.
Jitter the expiry. 300 seconds becomes 300 plus a random 30. One line that removes a whole class of incident.
Mention jitter. It costs 5 seconds, and panels remember cheap, real fixes.
4. What are you evicting?
LRU is the default and usually right. It fails on a scan. One batch job over a million cold keys evicts your hot set.
Give bulk readers a separate cache.
Eviction is memory pressure; expiry is correctness. Name the eviction rate.
Watch the trend: it tells a 1-off spike from a problem.
5. Where does the cache live? The caching system design layers
“Add Redis” is 1 of 4 options, and the other 3 are often better.
In-process. Nanoseconds, a different copy per server. Fine for configuration, dangerous for anything users see change.
A shared cache. One copy, consistent across servers, and a dependency that can fail.
The CDN. The cheapest correct answer for anything public. Candidates often skip it.
The browser. Free and outside your control. Cache-Control: immutable is safe on a hashed asset, a mistake on HTML.
Naming all 4 and choosing with reasons separates levels.
6. How do you know it is working?
Hit rate, and p99 latency.
A 95% hit rate with a slow miss path leaves an unlucky 5%. For them the cache is worse than nothing, and the average hides it.
Three alerts: hit rate falling, eviction rate rising, miss latency rising. Say “p99”, not “average”.
The pattern behind all 6 caching system design questions
Every question asks one thing. What does the system do when the cache is missing or wrong?
Treat a cache as a speed feature and you answer 2. Treat it as a second source of truth and you answer all 6.
Common questions
What is the single strongest thing to say about caching in an interview?
Name the invalidation race unprompted. A read misses, fetches the old value, and writes it in after the invalidation arrives. You have cached stale data with no expiry. It is real, it is common, and candidates rarely raise it before being pushed.
Is jitter really worth mentioning?
Yes, and it costs 5 seconds of interview time. Replace a 300-second expiry with 300 plus a random 30. That 1 line removes a whole class of incident. It is the kind of specific, cheap fix that panels remember. They write the scorecard an hour later and it is still there.
When should you not add a cache at all?
When the read to write ratio is close to 1 to 1. When the data must never be stale, or the database is not the bottleneck. Adding a cache to a write-heavy system buys staleness and a new failure mode. It buys almost nothing back. Say that out loud and you score better.
What separates a senior answer from a staff answer here?
Senior treats the cache as a speed feature. That is 2 or 3 of the 6. Staff treats it as a second source of truth, with its own failure modes. Staff answers all 6 unprompted. The difference is rarely more technology. It is which questions you were already asking yourself.
What is caching in system design?
Caching keeps a copy of a value closer to the reader than its source. It trades correctness for speed, because the copy can be wrong. In an interview the cache itself is never the question. What happens when it is wrong, cold, or expires all at once is the question.
The short version
Say what happens when the value is wrong, the cache is cold, a key expires.
Name your eviction policy and the scan that defeats it.
Choose the layer instead of defaulting to Redis. Measure p99, not the average.
Add jitter to your expiry. It is 1 line and it prevents a real outage.
The rest of the round is in the system design interview playbook. A worked example is the URL shortener.
To hear these asked at you for real, book a system design mock interview. Written scorecard within 24 hours.