Requirements


Functional Requirements:


  • Allow users to tweet messages up to 140 characters.
  • Enable users to follow other users.
  • Allow users to like tweets from other users.
  • Display tweets from followed users in the home feed.
  • Show top K popular tweets in the home feed based on likes and followers.
  • Allow user to create, login or delete account




Non-Functional Requirements:


  • The system must support 1MM of user at same time
  • Messages must be tweeted with less then 1 second
  • System must be available at 99,98% of time
  • Low latency for search tweets of people that users follow (less than 1 second to find 15 messages)


API Design

POST /tweet -> tweet a post

POST /like -> like a tweet

POST /answer -> answer a tweet

GET /tweet-> retrieve a list of tweets to list on user timeline

GET /tweet/top -> retrieve tweet of top users according to likes and followers

POST /follow -> follow or unfollow a user





High-Level Design

  • There are 2 services for front-end
    • web-application for users who access the app from the browser
    • phone application for users who acces the app from mobile
  • engage-service: will manage all the likes and retweets actions
  • follow-service: This service will be responsible to manage all relationship users, like follow and unfollow relationshio
    • every time a user follow/unfollow someone the follow-service will control this relationship.
    • everytime a user have more than 5k of followers it will create a element on queue
    • everytime that has previous 5k of follower and the count of follower decrease to 4.999 follower, it will create a element on queue
    • this queue will be an sqs to be consumed on top-user-service
  • user-service: it is responsible to manage the login, create account, and delete account process
  • fan-out workers: will show all tweets on timeline users
    • it will show all sugested tweets of top users. The search will occur accessing the top-user-service
    • it will show all following users tweet, the search will be made on redis-cache
  • timeline-cache-service:
    • this will use fanout write strategy
    • will consume tweets from sqs-tweet posted by tweet-register
    • will not consume tweets if it from top-user-service
    • for every tweet, it will search all the follower users and create a register on redis-cache to be consumed to show on timeline-service
  • tweet-register:
    • Every time a user post a tweet, it will post this tweet on sqs-tweet to be consumed on time-cache-service
  • top-user-service:
    • It will store all users that have more than 5k followers.
    • will be responsible to retrieve those users recents posts

The flow of tweet

  • A user post or delete a tweet
  • the tweet is persisted or deleted on tweet-database by tweet-register
  • the tweet-register post the action o sqs-tweet
  • the fan-out workers consumes this action and put or delete the tweet on redis cache
  • the timeline-service consumes all the tweets from redis-cache from normal users and consume the tweet directly from top-user-service.

Timeline:

  • will get the first 50 posts on redis-cache
  • if more feeds needed, it will start to search tweets on tweet-register, and add on redis-cache



Detailed Component Design

  • The load balancer will route the
  • We are going to run all services in a AWS ECS
  • Every service is going to be a ECS service with some tasks definitions:
    • memory and cpu definition
    • specific docker image of the service to pull
  • Every ecs service will have defined:
    • minimun desired task: 2 for redundancy
    • autoscaling based on memory and cpu consumption
  • Every service will have a rate limiter based on redis, and the key will be user_id and the value will be the access or action count
  • Mysql database:
    • follow-service
    • user-service
    • will use readonly replica
    • backup routines avery 4 hours
  • dynamodb database:
    • tweet-register
    • top-user-service
    • engage-service
    • will use horizontal scaling
  • tradoffs:
    • the timeline and engagement will prioritize low latency and availability, so probably will have eventual consistency on number of likes and timeline