Design Instagram

Problem Statement

Design a photo-sharing social media platform where users can upload photos, follow other users, and view a chronological feed of posts from people they follow. Users expect fast uploads, instantly visible posts, and a smooth, duplicate-free infinite scroll experience.

The primary challenge is designing for two hard things at once: handling very large media uploads reliably and serving a low-latency, high-scale feed. Interviewers want to see clear API contracts, pragmatic caching strategies, a hybrid feed approach for celebrities, and robust pagination. You should abstract the follow graph (assume it exists) and focus on client-visible performance rather than infrastructure details like sharding or load balancing.

Key Requirements

Functional

• Post creation -- users can upload a photo or video (up to 3 GB) with an optional caption, which becomes visible to followers
• Personalized feed -- users view a reverse-chronological feed of posts from accounts they follow with smooth infinite scroll
• Reliable pagination -- users can continue scrolling where they left off without duplicates or gaps across refreshes
• Post details -- users can view a post's media, caption, and author information quickly after creation

Non-Functional

• Scalability -- support hundreds of millions of users with heavily skewed follower distributions (celebrities with millions of followers)
• Reliability -- ensure uploads complete reliably even on flaky mobile networks with resumable upload support
• Latency -- serve feed pages in under 300ms; media uploads should not block user interaction
• Consistency -- eventual consistency acceptable for feed propagation; strong consistency for post creation confirmation

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Feed Generation Strategy

Interviewers want to understand your approach to building personalized feeds at scale, particularly the tradeoffs between fan-out-on-write and fan-out-on-read.

Hints to consider:

• Precompute feeds for most users using fan-out-on-write to enable O(1) feed lookups
• Use a hybrid approach for celebrity accounts where fan-out-on-write would be too expensive
• Store precomputed feeds in Redis sorted sets keyed by user ID with post IDs and timestamps
• Consider how new followers see historical posts and how unfollows affect precomputed feeds

2. Media Upload and Delivery Pipeline

Large photo and video uploads require special handling to ensure reliability and performance. Interviewers expect you to keep media off the application server hot path.

Hints to consider:

• Use pre-signed URLs for direct-to-object-storage uploads, bypassing application servers entirely
• Support multipart and resumable uploads for large files on unreliable mobile networks
• Trigger asynchronous processing (thumbnails, transcoding, CDN warming) via events after upload completion
• Serve media through a CDN with edge caching to minimize latency globally

3. Pagination and Scroll Consistency

Interviewers probe how you handle the common pitfall of users seeing duplicates or missing posts as they scroll through a continuously updating feed.

Hints to consider:

• Use cursor-based pagination with stable sort keys (timestamp + post ID) rather than offset-based pagination
• Return opaque cursor tokens that encode the last-seen position for deterministic next-page queries
• Handle the case where new posts arrive while the user is scrolling without disrupting their position
• Consider how caching interacts with pagination to avoid stale or inconsistent pages

4. Write Amplification and Fan-Out Management

When a user with many followers creates a post, the system must update potentially millions of precomputed feeds. Interviewers assess whether you understand the cost and mitigation strategies.

Hints to consider:

• Use message queues to decouple post creation from feed updates, allowing asynchronous fan-out
• Implement priority-based fan-out: update active users first, lazy-load for inactive users on next login
• Set capacity limits on precomputed feeds (e.g., keep only the latest 500 posts per user feed)
• Monitor fan-out lag and provide mechanisms for users to see very recent posts even before fan-out completes

Suggested Approach

Step 1: Clarify Requirements

Confirm the scope with the interviewer. Ask about supported media types (photos only or videos too?), maximum file sizes, expected user scale, and follower distribution. Clarify whether the feed is purely chronological or includes ranking/recommendations. Establish latency targets for feed serving and acceptable delay for new post visibility. Ask about additional features like likes, comments, and stories to understand scope boundaries.

Step 2: High-Level Architecture

Sketch the major components: API Gateway for mobile and web clients, Post Service for creating and storing posts, Feed Service for generating and serving personalized feeds, Media Service for upload orchestration and processing, and a Notification Service for alerting followers. Show how uploads flow directly to object storage via pre-signed URLs, how post creation events fan out through a message queue to update follower feeds, and how feed reads hit a fast cache layer. Include a CDN for media delivery and Redis for precomputed feed storage.

Step 3: Deep Dive on Feed Generation and Serving

Walk through the lifecycle of a new post. When a user creates a post, the Post Service writes metadata to the database and publishes an event to Kafka. Fan-out workers consume the event, look up the poster's follower list, and append the post ID to each follower's precomputed feed in Redis (ZADD with timestamp score). For celebrity accounts (followers > threshold), skip fan-out and merge their posts at read time. When a user requests their feed, the Feed Service reads from the precomputed Redis feed, merges in any celebrity posts via fan-out-on-read, hydrates post metadata from a cache or database, and returns a paginated response with a cursor token.

Step 4: Address Media Pipeline and Scalability

Describe the upload flow: client requests a pre-signed URL from the Media Service, uploads directly to S3, and the Media Service receives a completion callback. This triggers async workers for thumbnail generation, video transcoding, and CDN pre-warming. For scalability, partition the feed fan-out by user ID ranges, use Redis Cluster for feed storage, and implement read replicas for post metadata. Discuss monitoring feed lag (time from post creation to feed visibility), cache hit rates, and upload success rates. Cover graceful degradation: if the feed cache is unavailable, fall back to on-demand feed computation from the follow graph.