Design Harmful Content Detection System

Design a real-time harmful-content detection system for a social-media platform that supports text, image, and video posts. The system must automatically flag or remove content that violates policies (hate speech, harassment, nudity, violence, CSAM, spam) while minimizing false positives and supporting 500 M daily active users who generate 1 B posts/day with a P99 review latency of <200 ms. Your design should include: (1) a multi-stage ingestion pipeline that can apply fast heuristics (blocklists, perceptual hashes) first, then ML models (text toxicity CNN/BERT, image NSFW CNN, video frame + audio transcription classifiers), and finally queue uncertain items for human review; (2) a scalable serving layer that returns allow/block/pending decisions to the upload API; (3) a human-review workflow that prioritizes high-reach content, assigns tasks to reviewers, stores decisions, and handles appeals; (4) feedback loops so reviewer labels retrain models daily; (5) adversarial-resilience features (text normalization, OCR on embedded text, multi-modal fusion, adversarial training); (6) regional compliance (EU DSA, U.S. COPA) with audit logs; and (7) clear SLAs: 99.9 % uptime, <5 % false-negative rate for high-severity categories, <1 % false-positive rate for ordinary posts. Discuss the architecture, data stores, model deployment, queueing, edge caching, and how you would evaluate and iterate the system.