The 30 second answer
I'd say the hardware is the easy part; at this scale egress dominates the bill. The video estimates about $100,000 a month on a cloud provider, mostly for 4.8 petabytes sent to clients, and argues on-premises servers are cheaper at that volume. A real setup also needs at least four servers, a load balancer, a separate Redis machine and backups. Most products never get close, so I'd design for measured traffic.
The cost picture from the video.
- Estimate: around $100,000 a month to run this in production on AWS pricing, which the speaker calls conservative.
- Egress dominates: 4.8 PB sent to clients, close to 5,000 TB. Even with an aggressively negotiated 2 cents per GB, he says it adds up to something big.
- On premises: at this volume he argues it's cheaper to run your own machines and use the cloud mainly for backups, as Netflix and other large companies do.
- A production layout: at least four servers for redundancy and room for spikes, because real traffic adds encryption and compression that burn more CPU. DNS-based load balancing is much cheaper than an AWS application or network load balancer. Redis moves to its own machine as the shared source of truth for all servers. Database backups, IPv4 addresses and monitoring such as CloudWatch add more. Even with aggressive cost cutting, he puts it at thousands of dollars a month.
When it's overkill. Bitly's published figure was 10 billion clicks a month in 2022. Simple arithmetic on that figure gives an average of roughly 3,900 requests per second over a 30-day month, far below a million. The speaker himself says only a handful of companies ever reach this scale. So I'd size for measured peak traffic plus headroom, and scale with the cheap steps first: an index, batching through a queue, a cache, then more servers. A rewrite or kernel-bypass networking comes last.
What to say in the room. Name the cost driver (egress, not CPU), the redundancy you'd add, and the threshold where owning hardware beats renting it. Then say you'd start much smaller and let the load tests tell you when to take the next step.
Alternate interview scenarios
"Your traffic is 5,000 requests per second. Which parts of this design do you keep?"
The index, the cache-aside lookup, and moving analytics writes off the request path into batched inserts. Skip the native rewrite and the custom stream sharding; a couple of ordinary app servers behind a load balancer will do.
Sources (2)
- Same video, Introduction (0:00)youtube.com
- Same video, Final Gigantic Test (2:18:07)youtube.com