System requirements

Functional:

List functional requirements for the system (Ask interviewer if stuck)...

1. Display Posts: The Newsfeed should display posts from profiles and pages that the user follows in a chronological order, with the most recent posts appearing at the top.
2. Variety of Content: Support different types of content such as text posts, images, videos, links, and status updates.
3. Likes and Comments: Allow users to like and comment on posts.
4. Sharing: Enable users to share posts on their own timeline or with friends.
5. Customization: Provide options for users to customize their Newsfeed preferences, including following/unfollowing specific profiles or pages.
6. Personalized Recommendations: Incorporate algorithms to personalize the content shown based on user interactions, interests, and behavior.
7. Pagination: Implement pagination to load more posts as the user scrolls down the Newsfeed.
8. Real-time Updates: Notify users of new posts in real-time without the need for manual refresh.

Non-Functional:

List non-functional requirements for the system...

1. Performance: Ensure fast loading times and responsiveness, even during peak usage. Aim for a maximum response time of 2 seconds for loading the Newsfeed.
2. Scalability: Design the system to handle a large number of concurrent users and a high volume of posts. Plan for at least 1 billion active users and 10,000 posts per second.
3. Reliability: Aim for high availability with minimal downtime. Target a system uptime of at least 99.99%.
4. Security: Implement robust security measures to protect user data and privacy. Use encryption for data transmission and storage.
5. Compatibility: Ensure that the Newsfeed works seamlessly across different devices and platforms, including web browsers and mobile applications.
6. Ease of Use: Design an intuitive user interface that is easy to navigate and understand. Prioritize user experience to keep users engaged.
7. Maintainability: Develop the system with modular components and clean code to facilitate maintenance and future updates.
8. Monitoring and Analytics: Implement monitoring tools to track system performance, user engagement, and feedback for continuous improvement.

Capacity estimation

Given:

• Average Number of News per User: 400 posts
• Average News Text Size: 200 characters
• Assuming 4 bytes per character for storage (assuming Unicode characters for flexibility)

The calculation for the average daily storage size per user would be: [ 400 \text{ posts} \times 200 \text{ characters} \times 4 \text{ bytes} = 320,000 \text{ bytes} ]

This means that each user would generate approximately 320,000 bytes (or 320 KB) of data per day in news posts alone.

If we scale this to accommodate 1.8 billion daily active users on Facebook, the total daily storage required for all users would be: [ 320,000 \text{ bytes per user} \times 1.8 \text{ billion users} = 576,000,000,000,000 \text{ bytes} = 576 TB ]

So we need daily storage 576 TB

API design

1. getNewsFeed()
2. retrieve personalized news content.
3. input: Authentication token for user identification.
4. output: a list of posts(text).
5. getExploreNewsFeed()
6. fetch trending popular content from across platform exploration.
7. input: params like geo location, category interests, trending topic.
8. output: a collection of posts.
9. getReelsFeed()
10. retrieve a feed of short videos clips.
11. input: params like geo location, category interests, trending topic.
12. output: a collection of posts.

Database design

Database choice:

1. Feed(news, explore, reels)
2. NoSQL offer flexible schema and horizontal scalability.
3. user data
4. SQL, structured data like user profile
5. Post/Like/Comment
6. SQL, relationship data suite for relational data, with comlex comment/query.
7. caching. user profiles, feed data
8. in memory, redis, make i fast for recent active users.

High-level design

user service: user auth,reigster ,profile manage,.

post service:post creation,like and comment.

cache: improve performance around often access data user profiles, posys, comments

feed service: generate personalized content

• feed generation service, feed cache, feed rank service.

analytics service: track user interactions.

fanout service: Distribute posts and notfy followers of user.

database management system

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Fanout service:

pull model/fan out on load

• New data might not be available until user requests for it.
• Many times, there would be no new update in the newsfeed and an empty response will be served by our servers. This way our resources will be used just to send an empty response.

optimize it, we can use hybrid model, If users have comparatively less number of followers, we can use the push model. Pushing to less number of followers is not a problem. While if a given user with millions of followers uploads a new post, we can use a pull model for them. That is, followers of these users will be updated on the new post whenever the followers requests for the newsfeed. This way the server won't be busy sending the new post to all the millions of followers. Pushing to users with less number of followers is affordable.

feed ranking:

ranking algorithm: relevance. popularity...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

1. Combination of SQL/NoSQL to meet diverse data storage, trading off consistency and horizontal scalability.
2. microservices model, overhead manage multiple sub components.
3. 
   

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

1. Machine learning powered rank service.
2.