Idempotency in Message Processing

Design a Kafka consumer that processes payment events from a topic and updates account balances in a relational database. The consumer must guarantee that each payment is applied exactly once even though Kafka delivers messages at-least-once and the consumer may crash and restart at any point. You must implement the consumer logic, the deduplication mechanism, and the database schema. The system should handle 10 k messages/second, scale horizontally across many partitions, and tolerate consumer rebalances without double-processing or losing payments.