The 30 second answer
I cache code-to-URL lookups in Redis with cache-aside: check Redis, fall back to Postgres on a miss, then store the result. On edit or delete I simply delete the key instead of updating it, because a Postgres re-fetch is cheap. That left the database about 70% idle and took throughput to roughly half a million per second, after a short warm-up while the cache filled.
What changed. With Redis cluster mode on, throughput didn't move, because the database CPU was also saturated by the code lookups. Turning on link caching moved those reads to Redis.
The pattern. On a redirect, the resolver checks the link cache first. On a hit it returns; on a miss it queries Postgres and stores the result. When a link is edited or deleted, the key is deleted rather than updated, so there is no window where the cache holds a value the database no longer has. The speaker's reasoning: a fetch from Postgres is cheap, so deleting is simpler than keeping two copies in sync.
The numbers. With about 140,000 codes in the test set, the database went to about 70% idle and throughput reached close to half a million requests per second. The first seconds of each run were slower while the cache filled; on a re-run every code was already in Redis. At this point the database wrote only about 200 MB/s against 1 GB/s.
The new bottleneck. The app server had no idle CPU left. Redis cluster and the database still had headroom, and the tester was half idle.
The general cache-aside tradeoffs (TTL, stampedes, hot keys, eviction) are covered in the canonical question When do you add a cache, and what are the risks?.
Alternate interview scenarios
"Why did the first seconds of every test run slower?"
Cold cache. Until each code has been requested once, misses go to Postgres. For a launch or a viral link you can pre-warm the hottest keys.
Sources (2)
- Same video, Moving to Our Production App (1:07:47)youtube.com
- Microsoft Azure Architecture Center: Cache-Aside patternlearn.microsoft.com