Design Recommender System

Problem Statement

Design a recommendation engine that analyzes user behavior and preferences to surface personalized lists of items such as products, articles, videos, or jobs. Users see ranked results tailored to their interests, and those results update as their activity changes. The system must combine offline model training with online serving to deliver fresh, relevant recommendations under strict latency budgets while supporting continuous experimentation through A/B tests.

Building a production recommender involves a multi-stage pipeline: ingesting behavioral signals at high volume, computing and storing features, training models offline, serving candidates in real time, and closing the feedback loop to measure and improve quality. Interviewers use this problem to assess your ability to balance freshness against cost, design layered retrieval and ranking architectures, handle cold-start scenarios for new users and items, and build systems that evolve through controlled experimentation without disrupting the user experience.

Key Requirements

Functional

• Personalized ranking -- Users see a ranked list of items relevant to their interests, context, and recent activity
• Filtering and refinement -- Users apply filters (category, price range, genre) and receive updated personalized results
• Explicit feedback -- Users can like, dislike, save, or hide items, and these signals influence future recommendations
• Live refresh -- Recommendations update as the user's recent activity changes without requiring a manual page reload

Non-Functional

• Scalability -- Serve 100 million daily active users with 10,000 recommendation requests per second at peak, over an item catalog of 100 million entries
• Reliability -- Maintain 99.9 percent uptime for the serving path with graceful fallback to cached or popularity-based recommendations during degradation
• Latency -- Return personalized results within 200 ms at p95, with precomputed fallback lists available in under 50 ms
• Consistency -- Eventual consistency for feature updates and model deployments; strong consistency for user preference writes

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Multi-Stage Retrieval and Ranking Architecture

Scoring every item in a 100-million-item catalog at request time is infeasible. Interviewers expect a funnel-shaped pipeline that narrows candidates efficiently before applying expensive ranking models.

Hints to consider:

• Use a candidate generation stage that retrieves a broad set (thousands) of items using lightweight signals like collaborative filtering, embedding similarity, or category affinity
• Follow with a ranking stage that applies a more expressive model (gradient-boosted trees, neural ranker) to score the narrowed set using rich user and item features
• Add a re-ranking stage that enforces business rules such as diversity, freshness, and deduplication before returning the final list
• Discuss how each stage trades off recall against computational cost

2. Feature Engineering and Online/Offline Separation

Recommendations depend on features that vary in freshness and cost to compute. Interviewers want to see a clear separation between batch, near-real-time, and request-time features.

Hints to consider:

• Compute long-term features (user preference vectors, item popularity scores, collaborative filtering embeddings) in batch jobs and store them in a feature store
• Update short-term features (session click count, rolling CTR, recency signals) via a streaming pipeline (Flink or Spark Streaming) and write them to a low-latency store
• At request time, assemble a feature vector by reading from both the batch feature store and the streaming feature store, then pass it to the ranker
• Discuss how stale features from the batch path are acceptable for slowly changing signals while freshness matters for session-level context

3. Cold Start for New Users and New Items

A recommender that only works for established users and popular items provides a poor experience for newcomers. Interviewers look for practical fallback strategies.

Hints to consider:

• For new users, start with popularity-based or trending recommendations and transition to personalized results as behavioral signals accumulate
• Use contextual signals (device, location, time of day, referral source) to bootstrap initial personalization even without history
• For new items, use content-based features (title, description, category, metadata embeddings) to place them into existing item clusters
• Implement exploration mechanisms (epsilon-greedy, Thompson sampling) that surface new items alongside established ones to gather early signal

4. Experimentation and A/B Testing Infrastructure

Improving a recommender requires continuous experimentation. Interviewers expect a robust framework for running controlled tests without degrading the user experience.

Hints to consider:

• Assign users to experiment buckets deterministically (hash of user_id and experiment_id) so assignment is stable across sessions
• Route each request through a configuration service that returns the active model version, feature set, and ranking parameters for that user's bucket
• Track engagement metrics (click-through rate, dwell time, conversion, long-term retention) per experiment arm with statistical significance testing
• Support gradual rollouts: start a new model at 1 percent of traffic, increase to 10 percent, then 100 percent, with automated rollback on metric regression

5. Serving-Layer Scalability and Caching

Recommendation serving is read-heavy and latency-sensitive. Interviewers want to see how you keep response times low under peak load without excessive infrastructure cost.

Hints to consider:

• Precompute top-N recommendation lists for users with predictable patterns and cache them in Redis with TTLs tied to expected freshness
• Use the precomputed list as an instant fallback and apply lightweight re-ranking at request time to incorporate very recent signals
• Partition the feature store by user_id to distribute lookups across nodes and avoid hot-key bottlenecks
• Auto-scale the ranking service based on request rate and latency percentiles, with a circuit breaker that falls back to cached lists under overload

Suggested Approach

Step 1: Clarify Requirements

Confirm the item catalog size, user base, and expected request volume. Ask whether the recommender targets a single surface (homepage feed) or multiple surfaces (search, email, notifications) with different latency budgets. Clarify which engagement metrics matter most (clicks, purchases, watch time, long-term retention). Determine whether real-time personalization based on the current session is required or if daily batch updates suffice. Ask about cold-start expectations and whether new users arrive in large bursts (marketing campaigns) or gradually.

Step 2: High-Level Architecture

Sketch the major layers: an event ingestion pipeline (Kafka) that captures impressions, clicks, and conversions from clients. A streaming feature processor (Flink) that maintains real-time aggregates and writes to a low-latency feature store (Redis). A batch training pipeline (Spark) that reads historical data, trains collaborative filtering and ranking models, and publishes artifacts to a model registry. An online serving layer that receives a recommendation request, retrieves candidate items from a candidate generation service, assembles feature vectors from the feature store, scores candidates using the deployed ranking model, applies re-ranking rules, and returns the final list. Include an experiment configuration service that controls traffic routing across model versions.

Step 3: Deep Dive on Serving Path

Walk through a single recommendation request in detail. The API receives a request with user_id and optional filters. It first checks a Redis cache for a precomputed list. On a cache miss, it calls the candidate generation service, which performs an approximate nearest-neighbor lookup on user embedding against item embeddings to retrieve the top 1,000 candidates. It then loads user features (batch preferences, streaming session signals) and item features (popularity, freshness, category) from the feature store. The ranking model scores all candidates and produces a sorted list. The re-ranker applies diversity constraints (no more than three items from the same category in the top ten), filters out previously seen items, and enforces any business boosting rules. The final top-50 list is returned and cached with a short TTL. The API also logs the served items to Kafka for downstream impression tracking and model training.

Step 4: Address Secondary Concerns

Cover the feedback loop: impression and click events flow through Kafka into the training pipeline, closing the loop between serving and model improvement. Discuss cold start: new users receive popularity-based recommendations enriched with contextual signals; as clicks accumulate, the system transitions to personalized results within a few sessions. Address monitoring: track recommendation latency, cache hit rate, CTR by experiment arm, model serving errors, and feature staleness. Mention cost optimization: precompute lists for the most active users to reduce real-time inference load, use tiered model complexity (lightweight model for low-value surfaces, full model for high-engagement surfaces), and auto-scale inference pods based on traffic patterns. Briefly touch on reliability: if the ranking service is unavailable, serve the cached or popularity-based fallback list so the user experience is never blank.