Design Instagram

Problem Statement

Design a photo-sharing social media platform where users can upload photos, follow other users, and view a personalized feed of posts from people they follow. Users expect fast uploads, instantly visible posts, and a smooth, duplicate-free infinite scroll experience.

Instagram is a media-sharing social network where people post photos and videos and scroll a personalized feed from accounts they follow. The system must handle two fundamentally different challenges simultaneously: reliably uploading large media files and serving low-latency, high-scale feeds. Feed generation for celebrity accounts with millions of followers introduces fan-out complexity, while maintaining scroll consistency requires stable cursor-based pagination.

Interviewers at Uber ask this to test whether you can design for high-scale media uploads alongside low-latency feed delivery, balancing write amplification from fan-out with read performance. They expect clear API contracts, pragmatic caching strategies, a hybrid feed approach for high-follower accounts, robust pagination, and efficient media storage using direct-to-object-storage patterns.

Key Requirements

Functional

• Post creation -- users upload a photo or video (up to 3 GB) with an optional caption, and the post becomes visible to followers shortly after upload completes
• Personalized feed -- users view posts from accounts they follow in reverse chronological order with infinite scroll pagination
• Stable pagination -- users can continue scrolling where they left off without duplicates or gaps, even as new posts arrive
• Post detail view -- users can view a post's media, caption, and author information quickly after it is created

Non-Functional

• Scalability -- support hundreds of millions of daily active users with peak upload rates of thousands of posts per second and millions of feed reads per second
• Reliability -- ensure uploaded media is never lost and feed generation tolerates individual service failures
• Latency -- feed loading under 300 ms for the first page; media display under 2 seconds for typical photo sizes
• Consistency -- eventual consistency acceptable for feed propagation (seconds-level delay); strong consistency for post creation confirmation

What Interviewers Focus On

Based on real interview experiences at Uber, these are the areas interviewers probe most deeply:

1. Media Upload and Storage Strategy

Handling large file uploads reliably determines system cost and user experience. Interviewers want to see you avoid routing binary data through application servers and instead use direct-to-storage patterns.

Hints to consider:

• Generate pre-signed URLs so clients upload directly to object storage (S3/GCS) without proxying through your API layer
• Support multipart/resumable uploads for large files to handle network interruptions gracefully
• Store only metadata in your relational database while keeping media blobs in cheap, durable object storage
• Trigger asynchronous post-upload processing (thumbnail generation, transcoding) via events to a message queue

2. Feed Generation and Fan-Out Strategy

The feed is the highest-traffic read path and the most complex write path. Interviewers evaluate whether you understand the tradeoffs between fan-out-on-write and fan-out-on-read, especially for celebrity accounts.

Hints to consider:

• Use fan-out-on-write for regular users: when a user posts, push the post ID into each follower's precomputed feed in a fast store like Redis
• For celebrity accounts (millions of followers), use fan-out-on-read to avoid write storms that would overwhelm the system
• Implement a hybrid approach where a threshold (e.g., 10,000 followers) determines which strategy applies
• Use an asynchronous worker pool to handle fan-out, decoupled from the post creation API via a message queue

3. Pagination and Feed Consistency

Users expect a smooth scrolling experience without seeing duplicates or missing posts. Interviewers probe how you maintain consistent pagination under concurrent writes.

Hints to consider:

• Use cursor-based pagination with (timestamp, post_id) as the cursor rather than offset-based pagination
• Ensure the cursor is opaque to the client and encodes enough state to resume exactly where the user left off
• Handle the merge of precomputed feed entries with real-time celebrity posts at read time
• Cache the first few pages of each user's feed with short TTLs, invalidating on new posts from followed accounts

4. CDN and Media Delivery Optimization

Serving media at scale requires a content delivery strategy that minimizes latency and bandwidth costs. Interviewers look for CDN integration and multi-resolution support.

Hints to consider:

• Generate multiple thumbnail sizes during upload processing (small for feed, medium for detail, original for full view)
• Serve all media through a CDN with long cache TTLs since media is immutable after upload
• Use progressive image loading (low-resolution placeholder followed by full image) to improve perceived latency
• Consider regional CDN placement based on user geography to minimize round-trip time

Suggested Approach

Step 1: Clarify Requirements

Confirm the scope of media types (photos only or also videos), expected scale (DAU, posts per second, feed reads per second), and whether features like likes, comments, or stories are in scope. Clarify the consistency requirements for feed visibility and the maximum acceptable delay from posting to appearing in followers' feeds. Ask about the follow graph scale (average followers, maximum followers) since this drives the fan-out strategy.

Step 2: High-Level Architecture

Sketch the core components: API gateway, post service (handles creation and metadata), media storage (object storage + CDN), feed service (precomputed feeds in Redis), fan-out workers consuming from a message queue, user/follow graph service, and a relational database for metadata. Show two key flows: the upload flow (client to pre-signed URL to object storage to event to processing workers) and the feed flow (client to feed service to merge precomputed feed with celebrity posts to return paginated results).

Step 3: Deep Dive on Feed Generation

Walk through what happens when a user creates a post. The post service stores metadata, emits an event to Kafka partitioned by poster ID. Fan-out workers consume events, look up the poster's follower list, and for each follower push the (timestamp, post_id) into their Redis sorted set (ZADD). For celebrity accounts above the follower threshold, skip fan-out and instead flag the post for fan-out-on-read. When a user loads their feed, the feed service reads their precomputed sorted set, merges in recent posts from followed celebrities (fetched from a celebrity post index), deduplicates, and returns a paginated response with a cursor. Discuss how you handle new follows (backfill recent posts) and unfollows (lazy cleanup).

Step 4: Address Secondary Concerns

Cover media processing: asynchronous thumbnail generation and transcoding via worker pools with dead letter queues for failures. Discuss storage lifecycle policies (move originals to cold storage after 90 days). Address feed cache invalidation and TTL strategies. Explain monitoring for upload success rates, fan-out lag, and feed latency percentiles. Mention horizontal scaling: shard Redis by user ID, partition Kafka by poster ID, and use read replicas for the metadata database.

Related Learning Resources

• Building block: CDN -- content delivery networks are critical for serving media at low latency globally
• Building block: Blob Storage -- direct-to-storage upload patterns and object lifecycle management
• Building block: Message Queues -- event-driven fan-out and asynchronous processing pipelines