Design a Yield-to-Pedestrian Recognition System

Design a machine-learning system that decides whether the autonomous vehicle must YIELD (come to a complete stop and wait) or GO at a pedestrian cross-walk. The system must run on-board in real time (<100 ms end-to-end), ingest synchronized multi-modal sensor data (RGB camera as primary, LiDAR for depth & low-light fallback, optional radar), and output a binary yield/go decision plus a confidence score for every frame. The model should generalize across daylight, dusk, night, rain, and occlusions (umbrellas, groups, strollers), and must meet safety-critical recall targets (≥99.5 % on pedestrians who actually intend to cross). You are expected to describe the full data pipeline (collection, labeling, augmentation), model architecture trade-offs (CNN vs ViT vs hybrid), loss functions, evaluation metrics, on-line failure detection, and deployment optimizations (quantization, pruning, distillation) necessary to ship the feature in a production self-driving stack.