Requirements

Functional Requirements:

The main functions of Twitter will be -

1. Tweet - compose and share a tweet.
2. Like
3. share
4. View Updates page (Tweets of other followed users)

The users of this system will be logged in/out users

Non-Functional Requirements:

Performance -

• the system must be able to support -
  • read - tens of ms per request
  • write- 1/2 - 1 sec per request

Scalability -

the system should be able to support -

• view - tens millions of read updates per second
• tweet - millions of write per second.
• like/share - tens millions per second.

Reliability -

• the system must be highly available by having millions of horizontal instances of the services.
• On holidays/viral users the user must be able to not crash due to high traffic.
• The system must be available on p99

Capacity Estimation

API Design

POST /tweets/{tweet_id} body={tweet-body}

POST /likes/{tweet-id}&liked_by={user-id}

PUT /shares/{tweet-id}&shared_by={user-id}

GET /timeline/

High-Level Design

1. load balancer -
2. API Gateway
3. Logic Services -
   1. Tweets Service
   2. Likes Service
   3. Share Service
   4. Timeline Service
4. An LRU distributed cache for viral posts to be used by all the services
5. DB - for keeping metadata
6. Storage - for keeping data of tweets
7. Monitoring and Logging service

Database Design

No SQL such as MongoDB/DynamoDB for having support on big data and high horizontal scale. we must have many read replicas, write replicas will be sharded by a hash that has both the tweet id and number of replicas for avoiding rebalancing issues.

Entities -

Tweets table -

• tweet id - primary key
• user id - foreign key
• body reference (pointing to the storage)

Likes table -

• tweet_id - primary key and foreign key
• user_id - primary key and foreign key

Shares table -

• share_id - primary key
• tweet_id - foreign key
• user_id - foreign key

Users table -

• user_id - primary key
• email

Detailed Component Design

1. tweets service -
   1. Scalability/Performance - must be able to support millions of requests per second - on tweet request the system will update the db, will upload the body to the storage and will also update the LRU cache.
   2. for reliability and scalability - must have thousands of horizontal instances of the service that can be autoscaled based on CPU metrics.
2. timeline service -
   1. Performance - In order to view timeline - the request will be paginated let's say with 20 posts per request for avoiding multiple networks hops and db query - by paginating and batching 20 reads this will improve the stress on the read requests but a single read might take longer than a single tweet fetch.
   2. Reliability/Scalability - must have thousands of horizontal instances of the service that can be autoscaled based on CPU metrics.
   3. Performance - another point that will decrease the latency is the use of the LRU cache.