Hard system-design scaling

Design and scale a real URL shortener's redirect to 1M HTTP requests/s and about 2B Postgres rows every 30 minutes. Where do you start?

The 30 second answer

I'd start by pinning down what one redirect must do, because that's what scales. Here every click reads the link, appends a log line to disk and inserts a view row for paid analytics, and nothing may be lost. Then I fix the budget: one big app server, one load generator and a 16-core Postgres box. From there I load test, find the hardware bottleneck, fix it, and repeat.

The product and the hot path. The app is a real, paid URL shortener with users and revenue. Only the redirect (GET /:code) is scaled; shortening links is not benchmarked. Each redirect does three units of I/O work:

  1. A database read to resolve the short code to the real URL.
  2. A disk write to the request log (time, IP, method, URL, status, response time), one line per request.
  3. A database write: an insert into a views table (IP, user agent, headers, a visitor hash for unique views). Paying customers buy click analytics by date, country and referrer, so these rows can't be dropped.

The response is a 302 redirect.

The rules the speaker sets. No dropping logs, no dropping IP addresses, no data inconsistency, and no extra hardware: the budget is fixed, so every gain must come from design and code.

The hardware, as stated in the video.

  • App server and load tester: two identical AWS c8gn.12xlarge instances, each with 48 CPU cores, 96 GB of RAM and 150 Gbit/s networking.
  • Database: a separate Postgres machine with 16 cores and 32 GB of RAM.
  • Disks: half a terabyte of gp3 SSD on the server and the database. The database volume gets 8,000 IOPS and 1 GB/s of throughput; the other gets 3,000 IOPS. Half a terabyte is sized for 30 minutes of logs and rows.
  • Placement: all machines in one Availability Zone and one cluster placement group, so there are no cross-zone data transfer charges and the network path is short.
  • Pricing for the test: spot instances at roughly $2.8 per hour at most, against about $7 per hour on demand. Everything is defined in Terraform, so a spot reclaim just means re-running terraform apply.

The scale target, in context. The speaker compares it with Bitly's published figure of 10 billion clicks a month in 2022 and says this run is about 200 times that rate. He estimates that running this setup in production would cost around $100,000 a month, which he calls conservative.

How I'd open in an interview. State the per-request work, the durability requirement for view rows, and the read/write mix (every read also causes a write). Then say the plan out loud: baseline, find the bottleneck on every machine, change one thing, re-measure.

Alternate interview scenarios

"Why is a URL shortener's redirect a write-heavy problem?"

Because analytics turns every click into an insert. At 1 million redirects per second, the speaker's database receives about 1 million view rows per second, so the read is cheap to cache but the write can't be skipped.

"Why spot instances, and would you use them in production?"

They are much cheaper but AWS can take them back; AWS gives a two-minute interruption notice before it stops or terminates a Spot Instance. That's fine for a reproducible benchmark defined in Terraform. It is not fine for a stateful production database without replication and backups.

Sources (5)
  1. Scaling to 1 Million HTTP Requests/s, with Real Business Logic! (video, Introduction, 0:00)youtube.com
  2. Same video, Introduction to Our Infrastructure (21:10)youtube.com
  3. Same video, Moving to Our Production App (1:07:47)youtube.com
  4. AWS docs: Placement strategies (cluster placement groups)docs.aws.amazon.com
  5. AWS docs: Spot Instance interruption noticesdocs.aws.amazon.com