Design Metrics Collection and Alerting

Design a metrics monitoring and alerting system that can collect, store, query, visualize, and generate alerts based on system metrics (e.g., CPU usage, memory, request counts). The system should support both real-time monitoring and historical analysis. It must ingest continuous streams of metric data from servers, applications, and services, then make time-series metrics available for low-latency queries, power dashboards, and trigger alerts via email/SMS/Slack when user-defined thresholds or rules are met. The interviewer expects you to walk through the end-to-end pipeline: collection (push vs pull), streaming ingestion, time-series storage with retention tiers, query layer, alerting engine, and notification routing. You should also address how to scale to millions of metrics per second, maintain high availability and durability, and provide multi-tenant isolation and security.