Practice/Meta/Design CamelCamelCamel

Design CamelCamelCamel

System DesignMust

Problem Statement

Design a service that monitors product prices across multiple e-commerce platforms, similar to how price comparison tools work. Users can track items by submitting product URLs, view historical pricing trends through interactive charts, and receive notifications when prices drop below their desired thresholds. The system should handle millions of products with thousands of price checks per minute, support multiple notification channels (email, SMS, push notifications), and provide sub-second response times for chart rendering. Your design must respect external API rate limits, handle flash sale traffic spikes, deduplicate redundant price updates, and ensure alerts are accurate and timely. Consider how you'll efficiently store time-series data, schedule periodic price checks without overwhelming external APIs, and fan out notifications to thousands of subscribers when popular products go on sale.

Key Requirements

Functional

• Product tracking -- users can add products via URL or browser extension and see current price with basic statistics
• Historical visualization -- display price trends over configurable time windows (1 week, 1 month, 3 months, 1 year, lifetime)
• Alert configuration -- users set target prices or percentage thresholds and choose delivery channels (email, SMS, push)
• Subscription management -- users can pause, modify, or delete alerts and view all tracked products in a dashboard

Non-Functional

• Scalability -- support 50M tracked products, 100K price updates per minute, 1M daily active users
• Reliability -- 99.9% uptime for alerts; no missed notifications during price drops
• Latency -- price charts load in under 500ms; alerts delivered within 5 minutes of price change
• Consistency -- eventual consistency acceptable for historical data; strong consistency required for alert triggers

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Price Ingestion and Scheduling Strategy

The core technical challenge is fetching prices from external sources that impose strict rate limits while ensuring fresh data for millions of products. Naive approaches that poll every product hourly will either violate API terms or leave data stale.

Hints to consider:

• Design a priority queue where popular products or those near alert thresholds get checked more frequently
• Implement distributed rate limiting with token buckets per API provider to respect quotas
• Consider adaptive scheduling that increases check frequency when price volatility is detected
• Use a job scheduler (cron-like system) that partitions products across workers and time slots

2. Time-Series Storage and Query Performance

Price history queries are read-heavy and latency-sensitive. Users expect smooth chart rendering even for products with years of daily price points. Poor schema design leads to full table scans and timeout errors.

Hints to consider:

• Partition data by product ID and time ranges to enable efficient range scans
• Pre-compute rollups at different granularities (hourly, daily, weekly) to accelerate long-range queries
• Use columnar storage or specialized time-series databases if relational databases become a bottleneck
• Cache rendered chart data in Redis with time-based expiration aligned to your update frequency

3. Alert Evaluation and Notification Fan-Out

When a popular product drops in price, thousands of subscribers need instant notifications. Scanning the subscriptions table repeatedly is inefficient, and sending duplicate alerts damages user trust.

Hints to consider:

• Build an inverted index mapping product IDs to subscriber lists for O(1) lookup on price changes
• Use a message queue to decouple alert detection from notification delivery for better throughput
• Implement idempotency keys to prevent duplicate notifications if workers retry failed sends
• Apply rate limiting per user to avoid notification spam if prices fluctuate rapidly

4. Data Quality and Verification

False alerts caused by scraped data errors, currency mismatches, or temporary glitches erode user confidence. You need mechanisms to validate incoming prices before triggering alerts.

Hints to consider:

• Normalize prices to a common currency and unit before comparison
• Flag suspicious changes (e.g., 90% drops) for manual review or require confirmation from multiple sources
• Store raw scraped HTML or API responses for audit trails and re-parsing after format changes
• Implement a grace period where new prices are staged before becoming "official" to filter transient errors

Suggested Approach

Step 1: Clarify Requirements

Start by confirming scope boundaries with your interviewer. Ask how many products need tracking, expected price check frequency (hourly vs. daily), whether the system scrapes websites directly or uses official APIs, and which e-commerce platforms are in scope (single marketplace vs. multiple). Clarify notification delivery guarantees (at-least-once vs. exactly-once), chart granularity requirements, and whether real-time price updates are needed or if 30-minute delays are acceptable. Confirm scale expectations: are we handling 1M products or 100M, and how many users subscribe to popular items during Black Friday sales?