Design a simplified Kafka-like message queue system.
Components:
Producers: Send messages
Consumers: Receive messages
Brokers: Store messages
Topics/Partitions: Organize messages
Key Discussion Points:
How to provide message sending functionality to producers? (SDK with retry mechanism)
How to ensure Durability/Consistency/Availability?
Message ordering problem: If producer1 sends to broker1, and producer2 accidentally sends to broker1 (meant for broker2), how to maintain message order?
Answer: Use Raft consensus to elect a leader, leader determines message ordering
Architecture:
Producer SDK with batching and retry
Broker cluster with replication
Consumer groups for load balancing
ZooKeeper/Raft for coordination
// Functionality to add full integration TBD, in the meantime, please check out the problem description to see what you might expect in the interview!