System requirements
Functional:
- post tweet, potentially with images
- follow another user
- feed with posts from people we follow
- like tweet
Non-Functional:
- CAP theorem: we choose availability over consistency (eventual consistency is accepted)
- latency: <1s for adding a new post/following an user/liking a post
- throughput (computed below): 100K reads/second and 100K writes/second (1K have images which have in total ~5GB)
Capacity estimation
500M daily active users
2 posts/user/day
20 feed requests*/user/day
5 likes/user/day
*feed request: opening the feed or scrolling through it
1B posts/day
100M have images (most have just 1 image)
~200M images/day
10B feed requests/per day
2.5B likes/day
new followers:
- celebrities: 1k-100k/day. Average: 5k
- most users (99.9%): <100/day. Average: 15
Total new followers/day: 99.9% * 500M * 15 + 0.1% * 500M * 5k = 8B + 2.5B = 10B
Total number of reads:
10B feed requests/day
100K/second
Total writes per day:
100M image posts/day => 1K/second
1B text posts => 10K/second
12B new followers/likes => 100K/second
Storage: ~20PB/month
Images:
5MB/image
1MB/low resolution alternative
100KB/very low resolution alternative
6MB/image
600TB/day
18PB/month
Followers, likes: just a few bytes, so the total is low
Text posts:
500B/post
still much lower than the storage needed for the images
API design
POST /tweets
POST /follows
GET /feeds
POST /tweets/likes
Database design
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
We have several entities with several relationships between them, so we will go for a SQL DB.
(Note: images themselves will be stored in an S3 bucket; object storage is suitable because images are large, unstructured data)
Tables:
User: user_id, username, fullname
Tweet: tweet_id, poster_id (foreign key), text, timestamp
Image: image_id, s3_url, s3_url_low_res, s3_url_very_low_res, tweet_id (FK)
TweetToImage: tweet_id (FK), image_id (FK)
Follow: follower_id (FK), followee_id (FK)
Like: tweet_id (FK), liker_id (FK)
High-level design
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explanations:
- why likes and follows use the same service: although we have a lot of them (12B compared to 1B new tweets), their payload is very small (just two IDs compared to an entire text). hence this service should not become more under stress than the other services
- message queue when posting new image: the tweet service adds the image and some metadata to the MQ; the image uploader workers take from the queue and actually add the images to the S3 bucket
- DB: asynchronous replication because we value availability more than consistency
- DB: master-slave replication for increased throughput at read operations
- DB: might add sharding, if we find a way to split users into subgroups (e.g. by location, by common interests/followers)
Request flows
- post tweet: LB->tweet service->write DB; if the tweet has an image, then put it in the MQ, and an image uploader will take it and put it in the S3 bucket
- like, follow: LB->likes and follows service->write DB
- feed: LB->feed service->read DB (take most recent followees' tweets)
Detailed component design
- DB scales well, because we can add more read replicas or sharding
- services scale well, because we have a LB in front of them, and they are horizontally scalable, since they don't hold any state -- we could add, for example, add multiple feed services on several machines
Trade offs/Tech choices
- We have several entities with several relationships between them, so we will go for a SQL DB.
- DB: asynchronous replication because we value availability more than consistency
- DB: master-slave replication for increased throughput at read operations; master-master would help write operations, but is harder to configure -- however, it would be a valid choice if we want to focus on write throughput; the fact that it makes the system less consistent is not a problem, since we value availability much more
- using a MQ+dedicated uplaoder worker between tweet service and S3 bucket decouples the tweet service from the upload of the image, and increases scalabiltiy, as we could add more workers
Failure scenarios/bottlenecks
- LB might be a single point of failure and bottleneck
- master DB dies => a replica has to be promoted
- we keep some warm replicas for all services. service dies => prepare that replica. can be done easily, since the services are stateless
Future improvements
- load balancer issue: add a replica that can take over. the replica and the in-use LB could communicate through heartbeats to make sure that the in-use LB is working well
- add caching between the services and the DB
- use previously generated feeds to generate a new one