Loading...
Clients should be able to increment and decrement a value globally.
Clients should be able to retrieve the global value.
Examples: video views in youtube.
All operations should be fast.
Counter value should be accurate.
Eventual consistency.
Highly available, scalable.
Peak Write QPS per counter = 20M/h = 5k
Average Write QPS 1B user * 10 video views / 24h = 120k
Average Read QPS = 10 read per write = 1M
Storage = 1B * 1 Video per day * 5 year * 20 byte = 36Tb
http://web/post/counter/id/{value}
http://web/get/counter/id
The data has no relation.
I would go with no sql, consist of
counter_id (key), counter_value
I would choose cassandra over redis as not every counter will be read frequently, hence using in-memory only storage would be a waste of resource.
Stateless API service that takes get/post requests from user.
Partitioned, replicated cassandra db system.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
For hot counter, one single leader might not be able to process all the updates efficiently, which could cause delay on user end.
To solve this problem, we may shard hot counter into multiple multiple shards. Each shard has a leader and multiple followers. Data can be replicated from leader to followers asynchronously. To ensure write is fast. This solution would cause data loss upon leader failure. Another solution should be to have multiple leaders per shard, and the synchronizations between leaders can be based on deterministic rule, like leader_id + timestamp. Alternatively, db system with conflict free replicated data type could work too.
Upon write, user requests get routed to one shard with hashing algorithm like consistent hash, or the system can randomly pick a shard, as long as the write request are evenly distributed among shards. .
Upon read, the api server collect value from all shards and sum them up.
The sum can be stored in cache to speed up request, and need to be frequently synced with db. Write through cache won't be very good because the writer wouldn't know the sum value, besides knowing its own effect; we could have the cache have short TTL like second, then have read requests refresh it every second. So I'd prefer look-aside cache in this case.
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?