Design Instagram

Problem Statement

Design a photo-sharing social media platform where users can upload photos, follow other users, and view a chronological feed of posts from people they follow. Think of Instagram, where hundreds of millions of users scroll through personalized feeds daily, double-tap to like, and share moments through images and short videos.

The core technical challenge is building a system that handles massive media uploads reliably while serving low-latency, personalized feeds at scale. Users expect their posts to appear almost instantly in followers' feeds, uploads of large media files to succeed even on flaky connections, and infinite scrolling to feel seamless without duplicates or gaps. You need to balance write amplification from fan-out strategies against read latency, design efficient pagination for feeds that change constantly, and ensure media delivery through a global CDN keeps load times minimal.

At Roblox-scale, interviewers want to see how you handle the tension between precomputed feeds for fast reads and on-demand assembly for users with millions of followers. They also look for clear API contracts, practical caching strategies, and robust handling of edge cases like celebrity accounts and bursty traffic.

Key Requirements

Functional

• Photo and video uploads -- users upload media up to several gigabytes with optional captions, and the system processes thumbnails and transcodes formats asynchronously
• Personalized feed -- users see a reverse-chronological feed of posts from accounts they follow, with reliable cursor-based pagination
• Follow graph -- users follow and unfollow other accounts, and the system maintains these relationships for feed generation
• Post interactions -- users can like, comment on, and share posts, with counts visible on each post

Non-Functional

• Scalability -- support 500 million monthly active users with 10 million concurrent feed viewers during peak hours
• Latency -- feed loads return within 300ms at p95; media uploads complete acknowledgment within 2 seconds
• Reliability -- uploads never lose data even on interrupted connections; feeds remain available during partial outages
• Consistency -- new posts appear in followers' feeds within 30 seconds; feed pagination is consistent across sessions

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Feed Generation Strategy

Interviewers want to see how you balance precomputed feeds against on-demand assembly, especially when some users have millions of followers and fan-out on write becomes prohibitively expensive.

Hints to consider:

• Use a hybrid approach: fan-out on write for normal users (push posts to followers' feed caches) and fan-out on read for celebrity accounts (merge at query time)
• Store precomputed feeds in Redis sorted sets keyed by user ID, with post timestamps as scores for efficient range queries
• Discuss the trade-off between write amplification (updating millions of feed entries per celebrity post) and read latency (merging multiple sources on every feed request)
• Consider how you detect and classify "celebrity" accounts dynamically based on follower count thresholds

2. Media Upload and Processing Pipeline

Large file uploads over unreliable networks require careful design. Interviewers probe how you handle partial failures, retries, and asynchronous processing without blocking the user experience.

Hints to consider:

• Use presigned URLs for direct-to-S3 uploads with multipart and resumable upload support to handle large files and network interruptions
• Decouple upload completion from post creation using a message queue (Kafka or SQS) that triggers thumbnail generation, transcoding, and CDN warming
• Implement idempotent upload handlers so retries do not create duplicate media objects
• Consider client-side image compression and progressive upload indicators to improve perceived performance

3. Pagination and Feed Consistency

Infinite scroll must work reliably even as the feed changes underneath the user. Interviewers look for cursor-based pagination that avoids duplicates and gaps.

Hints to consider:

• Use cursor-based pagination with the last seen post ID or timestamp rather than offset-based pagination, which breaks when new posts are inserted
• Handle the edge case where a user unfollows someone mid-scroll, ensuring already-fetched pages remain stable
• Design the cursor to encode enough state (timestamp plus tie-breaking post ID) to resume deterministically
• Consider caching paginated feed segments to reduce database load on repeated scroll requests

4. CDN and Media Delivery

Serving billions of images and videos globally with low latency requires a thoughtful content delivery strategy that goes beyond simply saying "use a CDN."

Hints to consider:

• Store original media in regional object storage (S3) and push optimized variants to CDN edge nodes based on predicted popularity
• Generate multiple resolutions and formats during processing so the client can request the appropriate size for their device and network
• Use content-addressable URLs (hash-based) for cache-friendly, immutable media references
• Implement adaptive bitrate streaming for video content using HLS or DASH protocols

Suggested Approach

Step 1: Clarify Requirements

Confirm the scope with the interviewer: What types of media are supported (photos only, or also videos and stories)? Is the feed strictly reverse-chronological or does it incorporate ranking signals? How many followers can a single user have, and what is the expected distribution? Are features like direct messaging, explore/discover, or stories in scope? What are the latency SLAs for feed loads and post visibility? Clarify whether you should design the follow graph or assume it exists as a service.

Step 2: High-Level Architecture

Sketch the main components: a client layer (mobile/web apps), an API gateway handling authentication and routing, a Post Service managing CRUD operations for posts and media metadata, a Feed Service responsible for generating and serving personalized feeds, a Media Service handling uploads and processing, and a Notification Service for push alerts. Use PostgreSQL or DynamoDB for post metadata and user profiles, Redis for precomputed feed caches, Kafka for event streaming (new posts, follow changes), S3 for media storage, and a multi-region CDN for delivery. Show the write path (upload media, create post, fan-out to feeds) and the read path (fetch cached feed, merge celebrity posts, paginate).

Step 3: Deep Dive -- Feed Generation and Fan-Out

Walk through the lifecycle of a new post. When a user creates a post, the Post Service persists metadata and publishes an event to Kafka. A Fan-Out Service consumes this event, looks up the poster's follower list, and for each follower pushes the post ID into their Redis sorted set (keyed by follower user ID, scored by timestamp). For celebrity accounts (those exceeding a follower threshold, say 500K), skip the fan-out entirely and mark the post for on-read merging. When a user requests their feed, the Feed Service reads their precomputed feed from Redis, then queries the Post Service for recent posts from any celebrity accounts they follow, merges and deduplicates the results by timestamp, and returns the top N with a cursor token. Discuss how you handle feed cache eviction (TTL plus lazy rebuild), follower count changes (reclassifying users), and edge cases like a user following 10,000 accounts.

Step 4: Address Secondary Concerns

Cover media processing: uploads go directly to S3 via presigned URLs, triggering a Lambda or worker that generates thumbnails and transcoded versions, then updates post metadata with CDN URLs. Discuss monitoring metrics like feed generation latency, fan-out lag, upload success rates, and CDN cache hit ratios. Address reliability: use dead-letter queues for failed fan-out events, implement circuit breakers around the Feed Service's Redis dependency, and fall back to database-backed feed generation if cache is unavailable. For cost optimization, archive old feed entries from Redis to cold storage after 30 days and use tiered storage for infrequently accessed media.

Related Learning

Deepen your understanding of the patterns used in this problem:

• Caching -- feed caches in Redis are central to serving low-latency personalized feeds at scale
• Message Queues -- Kafka powers the fan-out pipeline and decouples post creation from feed updates
• Blob Storage -- understanding object storage patterns is key for the media upload and delivery pipeline
• Content Delivery Networks (CDN) -- global media delivery depends on effective CDN caching and edge routing strategies