Design a URL Shortener

Reference Answer

Review the Caching, Databases, and Load Balancers building blocks for background on read-heavy scaling, key-value storage, and request distribution.

Problem Statement

Design a URL shortening service similar to TinyURL or Bitly that converts long URLs into compact, shareable links and redirects visitors back to the original address. Users submit a long URL, receive a short code (for example, https://sho.rt/Ab3Cd), and anyone clicking that code is instantly sent to the destination.

The product appears simple but tests deep distributed-systems knowledge: globally unique ID generation without collisions, extreme read scaling for the redirect path, low-latency edge serving, asynchronous analytics capture, abuse prevention, and pragmatic data modeling. Interviewers use it to see whether you can define crisp requirements, estimate scale, choose the right storage and caching strategy, and reason about trade-offs between availability, consistency, and cost.

Key Requirements

Functional

• Short link creation -- Users submit a long URL and receive a unique short code that maps to it
• Redirection -- Visiting the short link returns an HTTP redirect to the original URL with minimal overhead
• Link management -- Authenticated users can view, disable, or delete their short links through a dashboard
• Analytics -- Track total clicks per link with optional breakdowns by time period, referrer, and geographic region

Non-Functional

• Scalability -- Handle billions of redirects per month with tens of thousands of new link creations per second at peak
• Reliability -- 99.99 percent availability on the redirect path; no data loss for link mappings
• Latency -- P99 redirect response under 50ms at the edge; link creation acknowledged within 200ms
• Consistency -- A newly created link must be resolvable within seconds; eventual consistency is acceptable for analytics

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Short-Code Generation Strategy

Generating unique short codes without collisions or coordination bottlenecks is the core algorithmic challenge. A naive auto-incrementing counter creates a single point of failure and contention under load.

Hints to consider:

• Pre-allocate ID ranges to each application server so they can generate codes locally without cross-server coordination
• Encode numeric IDs using base-62 (a-z, A-Z, 0-9) to produce compact, URL-safe strings
• Alternatively, hash the long URL (MD5 or SHA-256) and take the first N characters, using collision detection and retry for conflicts
• Evaluate the trade-off between sequential codes (easy to enumerate, potentially a privacy concern) and random codes (harder to guess, more cache-friendly distribution)

2. Read-Heavy Caching Architecture

Redirect traffic vastly exceeds write traffic. Serving every redirect from the primary database will miss latency targets and inflate infrastructure cost.

Hints to consider:

• Place a CDN or edge cache at the outermost layer so popular links are served without reaching origin servers at all
• Use Redis as an application-level cache for the code-to-URL mapping, with a TTL to handle link disabling
• Design a multi-tier cache hierarchy: edge, regional Redis cluster, database fallback
• Plan cache invalidation for link deletion or disabling so stale redirects do not persist

3. Decoupled Analytics Pipeline

Every redirect generates a click event. Writing these synchronously on the redirect path would increase tail latency and couple the redirect endpoint's availability to the analytics backend.

Hints to consider:

• Emit click events asynchronously to a message queue (Kafka) immediately after serving the redirect
• Use stream processing to aggregate raw events before writing to an analytics store, reducing write amplification
• Separate the analytics data store (append-optimized, columnar) from the link metadata store (key-value, low latency)
• Provide real-time approximate counts for the dashboard and reconcile with exact batch aggregates periodically

4. Abuse Prevention and Rate Limiting

A public URL shortener is an attractive target for spam, phishing, and redirect loops. Interviewers expect both proactive and reactive defenses.

Hints to consider:

• Rate-limit link creation per IP and per authenticated user using a token-bucket algorithm in a shared cache
• Scan destination URLs against safe-browsing APIs and known blocklists before accepting them
• Implement a report-and-review workflow for users to flag malicious short links
• Monitor redirect velocity per link and auto-disable links that exceed suspicious thresholds

Suggested Approach

Step 1: Clarify Requirements

Confirm expected scale: how many new links per day, how many redirects per second, peak-to-average ratio. Ask whether custom aliases (vanity URLs) are needed, whether links expire after a retention period, and whether the service is single-tenant or multi-tenant. Establish latency and availability targets for the redirect path separately from the management API.

Step 2: High-Level Architecture

Sketch the core components: a redirect service sitting behind a CDN layer, a link-creation API behind an API gateway, a key-value store (DynamoDB or sharded Postgres) for the code-to-URL mapping, a Redis cluster for hot-path caching, a Kafka topic for click events, and an analytics pipeline that aggregates events into a columnar store. Show the write flow (create link, persist mapping, warm cache) and the read flow (CDN hit, Redis hit, database fallback, redirect response).

Step 3: Deep Dive on Code Generation and Redirect

Walk through the link creation flow. The API server draws the next ID from its pre-allocated range, encodes it as base-62, writes the mapping to the database with a uniqueness constraint, and warms the Redis cache. Discuss hash-based versus counter-based code generation trade-offs. Show the redirect flow: the CDN checks its cache; on a miss the request reaches the application, which checks Redis, then the database, constructs a 301 or 302 redirect response, and caches the result for future requests. Explain when to use 301 (permanent, browser caches it, reduces server load but loses analytics visibility) versus 302 (temporary, every request hits your server, better for analytics and link disabling).

Step 4: Address Secondary Concerns

Cover reliability by deploying the redirect service across multiple availability zones with health checks and failover. Discuss the analytics pipeline: Kafka consumers aggregate click events in a stream processor and write to a time-series or columnar database. Address link expiration with a background sweeper that removes or archives links past their TTL. Explain rate limiting and abuse detection. If time permits, discuss multi-region deployment with DynamoDB global tables or database replication for worldwide low-latency redirects, and estimate storage costs at scale.

Real Interview Insights

Candidates at Okta report being given a choice between designing a URL shortener, a news feed, or a domain-specific system. When the URL shortener is selected, interviewers push hard on the code generation strategy (asking for collision probability calculations), the caching hierarchy, and the analytics pipeline. One Okta Staff-level interview specifically explored how the service would be used within an organization rather than as a public tool, adding requirements around internal authentication and link audit trails.