System requirements


Functional:


  • User can post messages
    • Assume no video and image
  • User can follow other users
  • User can track the posts from followees and him/herself in reverse chronological order



Non-Functional:

  • Service needs to scale in peak time to support 500M daily active users
  • Service needs to be available




Capacity estimation

Estimate the scale of the system you are going to design...

  • Assume 3B users in total
    • Assume it does not increase for simplicity
  • Assume they have 500M DAU
    • Assume it does not increase for simplicity
  • An average user posts 5 times a day
  • An average user refresh the page 20 times a day
  • Posts will be saved for 10 years
  • An average user follows 50 users
  • A post is 300 byte
  • 300 * 500M * 5 * 365 * 10 = 2.74 PB -> 3 PB to store posts
  • User metadata is 200 byte
  • 200 * 3 billion = 600 GB
  • One follow is 16 byte
  • 16 * 3B * 50 = 2.4 TB




API design

Define what APIs are expected from the system...


  • post(String message)
  • follow(String user)
  • unfollow(String user)
  • refresh()
    • Return a list of posts





Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...


  • User
    • UserID (primary key) 8B
    • userEmail 100B
    • name 50 B
    • creationDate 8B
  • Post
    • PostID (primary key) 8B
    • UserID 8B
    • message: 280B
  • Follow
    • FolloweeID 8B
    • FollowerID 8B





High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...


3 services and 2 main DB


  • PostService
    • Post messages
  • FollowService
    • Handle follow/unfollow users
  • RefreshService
    • Provide latest timeline


  • Metadata DB
    • Store posts
  • Graph DB
    • Store who follows who





Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...


Post

  • Client -> LB -> API gateway -> PostService -> MetadataDB -> RefreshService -> MetadataDB,GraphDB -> Client

Follow

  • Client -> LB -> API gateway -> FollowService -> GraphDB, cache

Refresh

  • Client -> LB -> API gateway -> RefreshService -> MetadataDB,GraphDB -> Client



Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...


All the services & DB are hosted in multiple regions and instances for scalability and fault tolerance.


  • RefreshService
    • User pagination (with like 100 latest posts) to create timeline so that return values will not be too large
    • Search the most recent posts of followees and the user from metadata and return






Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...


  • Graph DB is used to store "follow" info
  • Metadata could be SQL




Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.


  • Always active services are tracked with service monitors like Zookeeper and queries goes to active service
  • If request fails, impose retry strategy



Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?


  • Support uploading media
  • Display trends