System requirements


Functional:

  • compose and share new tweets
  • track updates from other accounts
  • favourite other tweets




Non-Functional:

  • if no accounts followed the user will see a collection of posts from top accounts
  • app moves fast, no lags when scrolling the "wall" + infinite pagination
  • when unfollowing an account, the user does not see that account's tweets on the wall anymore




Capacity estimation

This system should cater to millions of users




API design

Authentication:

  • sign-up
  • sign-in
  • password recovery

Account:

  • update profile
  • show profile

Tweet handling:

  • create tweet
  • edit tweet
  • share tweet
  • update favourite
  • list tweets





Database design


Account:

id, email, password, photo, bio, rank, partial_rank, timestamps


FollowedAccounts:

id, account_id, followed_accounts_ids, timestamps


FavouriteTweets:

id, account_id, favourite_tweets_ids, timestamps


Tweet

id, account_id, reply_tweet_ids, body, media, rank, edited_at, timestamps

* assumes no nested reply threads for V1


Ranking can be measured on an arbitrary scale where each 1000 tweet likes/account follows increases the rank by one unit


High-level design


Mobile app:

  • Account section
  • Home screen
  • Tweet detailed view


BE Services:

* all services associated with the business layer (Create, Update etc)

  • ListTweets
  • MarkAsFavoriteService




Request flows


A. Login -> auth API

email: string, password: string


B. Infinite Pagination

  • Home -> list followed accounts API (or top rated if none followed) - first load
listFrom: timestamp, listToLimit?: timestamp = now - 24 hrs
  • Scroll Home page -> list followed accounts API - pagination
listFrom: timestamp = last loaded timestamp, listToLimit?: timestamp


C. Marking/Unmarking a tweet as favourite

  • Mark tweet as favourite -> update favourite API
tweetId: string, favourite: boolean


D. Adding/Editing a tweet

  • Create/Edit tweet -> create/edit tweet API
body: string, media?: { filename: string, contentType: string, checksum: string, byteSize: number }



Detailed component design


MarkAsFavoriteService:

params: tweet, user
  • increase tweet rank
  • increase account partial rank (and overall rank if needed)
  • decision? - automatically mark associated account as favourite



ListTweetsService:

params: user

creates a list sorted by created_at/edited_at in descending order, from the following

  • new tweets from followed accounts
  • updates to favourite tweets
  • top ranking account tweets




Trade offs/Tech choices


For v1, not handling regional ranks, in other words we do not create different ranking based on the region of the world; while this will mean that some users might get tweets from accounts that are not relevant to the user's country/region, by not implementing this in v1 we will speed up the TTL of the app




Failure scenarios/bottlenecks


  • biggest issue foreseeable is handling hundreds of thousands of requests per minute
  • concurrency is also a possible bottleneck as thousands of users try to follow one account in a short interval of time
  • database backfills can also become an issue if for some reason we need to update db schema for millions of rows



Future improvements


  • Due to the large number of users, operation costs can escalate quickly especially during peak hours. As improvements, we move some of the processing around listing tweets by rank to jobs which return a list of ranked tweet ids, and when it's time to actually show the to the user, we just read the list tweets based on the tweet ids returned by the job
  • Add logic to rank accounts by region so that global customers can be catered with more relevant tweets