Design a URL Shortener

Reference Answer

Review the Caching, Databases, and Load Balancers building blocks for background on read-heavy scaling, key-value storage, and request distribution.

Problem Statement

Design a URL shortening service similar to TinyURL or Bitly that converts long URLs into compact, shareable links and redirects visitors back to the original address. Users paste a long URL, receive a short code (for example, https://sho.rt/Ab3Cd), and anyone clicking that code is instantly redirected.

Although the product sounds simple, it exercises core distributed-systems skills: globally unique short-code generation without collisions, extreme read-heavy traffic on the redirect path, low-latency edge serving, asynchronous analytics capture, abuse prevention, and thoughtful data modeling. Interviewers use this question to see whether you can define crisp requirements, estimate scale, choose the right storage and caching strategy, and make pragmatic trade-offs around availability, consistency, and cost.

Key Requirements

Functional

• Short link creation -- Users submit a long URL and receive a unique short code that maps to it
• Redirection -- Visiting the short link returns an HTTP redirect to the original URL with minimal latency
• Link management -- Authenticated users can view, disable, or delete their short links through a dashboard
• Basic analytics -- Track total clicks per link with optional breakdowns by time, referrer, and geography

Non-Functional

• Scalability -- Handle billions of redirects per month with tens of thousands of new links created per second during spikes
• Reliability -- 99.99 percent availability on the redirect path; no data loss for link mappings
• Latency -- P99 redirect latency under 50ms at the edge; link creation acknowledged within 200ms
• Consistency -- A newly created short link must be resolvable within seconds; eventual consistency is acceptable for analytics

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Short-Code Generation and Uniqueness

Generating globally unique short codes without collisions or coordination bottlenecks is the central design challenge. Naive approaches like auto-incrementing a single database counter create hot keys and a single point of failure.

Hints to consider:

• Pre-allocate ranges of integer IDs to each application server so they can mint codes locally without cross-server coordination
• Encode IDs using base-62 (a-z, A-Z, 0-9) to keep URLs short and avoid characters that cause encoding issues in URLs
• Use hash-based approaches (MD5 or SHA-256 truncated to N characters) with collision detection and retry
• Consider the trade-off between sequential codes (predictable, easy to enumerate) and random codes (harder to guess, better for privacy)

2. Read-Heavy Scaling and Caching Strategy

Redirect traffic is overwhelmingly read-heavy -- often thousands of reads for every write. Serving redirects directly from the primary database will miss latency targets and inflate costs.

Hints to consider:

• Place a CDN or edge cache in front of the redirect endpoint so popular links never reach origin servers
• Use an in-memory cache such as Redis to store the hottest code-to-URL mappings with a TTL
• Design cache invalidation for link disabling or deletion so stale redirects do not persist indefinitely
• Consider geographic replication of the cache layer so users worldwide experience low latency

3. Analytics Capture Without Blocking Redirects

Every redirect generates an analytics event (timestamp, referrer, country, device). Writing these synchronously on the redirect path would increase tail latency and couple redirect availability to the analytics pipeline.

Hints to consider:

• Fire-and-forget the event to a message queue or streaming platform like Kafka immediately after serving the redirect
• Aggregate raw events in a stream processor before writing to an analytics store to reduce write amplification
• Use approximate counters for real-time dashboards and reconcile with exact batch counts periodically
• Separate the analytics data store (append-optimized, columnar) from the link metadata store (key-value, low latency)

4. Abuse Prevention and Rate Limiting

A public URL shortener is a target for spam, phishing, and redirect loops. Interviewers expect a plan for detecting and mitigating misuse.

Hints to consider:

• Rate-limit link creation per IP and per authenticated user using a token-bucket algorithm in a shared cache
• Scan destination URLs against blocklists and safe-browsing APIs before accepting them
• Implement a report-and-review workflow that allows users to flag malicious links
• Automatically disable links that exceed a redirect velocity threshold, which may indicate bot amplification

Suggested Approach

Step 1: Clarify Requirements

Confirm the expected scale: how many new links per day, how many redirects per second, and what the average and peak traffic patterns look like. Ask whether custom aliases (vanity URLs) are needed, whether links expire after a retention period, and whether the system should support private links that require authentication to redirect. Establish latency and availability targets for the redirect path versus the management API.

Step 2: High-Level Architecture

Sketch the core components: a lightweight redirect service sitting behind a CDN or edge layer, a link-creation API behind an API gateway, a key-value store (DynamoDB or a sharded relational database) holding the code-to-URL mapping, a Redis cluster for hot-path caching, a Kafka topic for redirect events, and a downstream analytics pipeline that aggregates events into a columnar store for dashboards. Show write flow (create link, persist mapping, warm cache) and read flow (edge cache hit or miss, Redis lookup, database fallback, redirect response).

Step 3: Deep Dive on Code Generation and Storage

Walk through how a request to create a short link flows through the system. The API server draws the next ID from its pre-allocated range, encodes it as a base-62 string, writes the mapping to the primary database with a uniqueness constraint, and warms the Redis cache. Discuss the trade-off between hash-based and counter-based code generation, noting that counters are simpler but require range allocation, while hashes avoid coordination but need collision handling. Explain how DynamoDB conditional writes or Postgres unique indexes prevent duplicate codes. Show the redirect path: CDN checks its cache, falls through to the application, which checks Redis, then the database, and returns a 301 or 302 redirect.

Step 4: Address Secondary Concerns

Cover reliability by deploying the redirect service across multiple availability zones with health checks and automatic failover. Discuss monitoring redirect latency, cache-hit ratios, and error rates. Explain how analytics events flow from Kafka consumers through a stream processor into a time-series or columnar database for dashboard queries. Address link expiration by running a background sweeper that deletes or archives links past their TTL. Mention rate limiting and abuse detection as described above. If time permits, discuss multi-region deployment with global DynamoDB tables or database replication for low-latency redirects worldwide.

Real Interview Insights

Candidates report that interviewers at financial and enterprise companies often push hard on the code generation strategy, asking you to walk through collision math (for example, birthday paradox calculations for hash-based approaches). Another common probe is the choice between 301 (permanent) and 302 (temporary) HTTP redirects and how that decision impacts caching, analytics accuracy, and the ability to disable links. Be prepared to estimate storage requirements and cost at scale.