Loading...
Clients should be able to increment and decrement a value globally.
Clients should be able to retrieve the global value.
Examples: video views in youtube.
All operations should be fast.
Counter value should be accurate.
Eventual consistency.
Highly available, scalable.
Peak Write QPS per counter = 20M/h = 5k
Average Write QPS 1B user * 10 video views / 24h = 120k
Average Read QPS = 10 read per write = 1M
Storage = 1B * 1 Video per day * 5 year * 20 byte = 36Tb
http://web/post/counter/id/{value}
http://web/get/counter/id
The data has no relation.
I would go with no sql, consist of
counter_id (key), counter_value
I would choose cassandra over redis as not every counter will be read frequently, hence using in-memory only storage would be a waste of resource.
Stateless API service that takes get/post requests from user.
Partitioned, replicated cassandra db system.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
For hot counter, one single leader might not be able to process all the updates efficiently, which could cause delay on user end.
To solve this problem, we may shard the counter into multiple leaders. Upon write, user requests get routed to one shard with proper algorithm.
Upon read, the api server collect value from all shards and sum them up.
The sum can be stored in cache to speed up request, and need to be frequently synced with db.
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?