Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
on average, each user posts 10 photos, total photo storage = (1.1mb) * 10 * 100m, round down to 1 PT
Define what APIs are expected from the system...
POST /v1/register
GET /v1/authenticate
POST /v1/post
POST /v1/uploadPhoto/{:filePath}
POST /v1/deletePhoto/{:id}
GET /v1/getFeed/{:id}
POST /v1/sendMessage
POST /v1/comment/{:id}
POST /v1/like/{:id}
GET /v1/pushNotification/{:id}
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
User Metadata DB:
UserDetails
-user_id: UUID (PK)
-username: string
-hashed_password, string
-bio: string
-created_at: datetime
-is_active: boolean
-last_login: datetime
-profile_pic: string, URL
-following_counts: int
-followers_counts: int
Comments:
-comment_id: int
-comment: string
-user_id: foreign key referencing UserDetails
Follow:
-followee: foreign key referencing UserDetails
-follower: foreign key referencing UserDetails
Feed:
-feed_id: int
-user_id: user_id foreign key referencing UserDetails
-post_id: foreign key referencing Posts
-added_at: datetime
-seen: boolean, weather the user has seen the feed
Photos:
-photo_id: int
-photo_url: URL in S3
-thumbnial_url: URL in S3
-upload_date: datetime
Message:
-message_id: int
-content: string
-sent_date: string
-sent_to: user_id foreign key referencing UserDetails
-photo: foreign key referencing Photos
Posts:
-post_id: int
-photo_id: foreign key referencing Photos
-user_id: foreign key referencing UserDetails
-photo_url: URL to the photo in S3
-like_count: int
-is_private: boolean
-caption: string
-post_date: datetime
-is_celebrity #
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Post photo (non-celebrity):
Register:
Client -> API Gateway -> Profile Service -> User metadata DB -> client
Login:
Client -> API Gateway -> Profile Service -> User metadata DB -> client
Comment:
Client -> API Gateway -> Interaction Service -> Interaction ID -> client
Like:
Client -> API Gateway -> Interaction Service -> Interaction ID -> client
Client -> API Gateway -> Post Service -> Photo Service Upload Photo -> Message Queue -> Object Store -> Post Service-> Post DB -> Feed Service Fanout -> message queue -> Notification Service -> Message Queue -> Client
Post (Celebrity):
Client -> API Gateway -> Post Service -> Photo Service Upload Photo -> Message Queue -> Object Store -> Photo Cache -> Post Service-> Post DB, when the followers login, it will call Feed Service Fanout -> message queue -> Notification Service -> Message Queue -> Client
Instant Messaging:
Client webscoket request -> Message Service -> websocket connection established -> Dynamo DB,
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Photo Service will interact with the object store DB for upload/retrieval, and save the photos to cache (redis).
When Photo Service requests fail, it will retry a few times, if still fails, it will store photos in queues or cache, or store in local cache when the user is uploading photos, so when the service recovers it will work faster.
Feed Service will push or pull posts and fan out the followers.
Cloud Watch will monitor all of the micro services and DB system activities/performance and send alerts to developers if theres failures
CDN to store frequently accessed photos for fast delivery for global users
Explain any trade offs you have made and why you made certain tech choices...
CDN:
Pros: fast delivery, enhance system performance
Cons: more cost and dependent on S3
Message Queue:
Kafka:
Pros: high throughput and highly scalablity for data streaming, high fault tolerance
Cons: complexity, higher latency for small volume traffic compared to RabbitMQ
Notification Service:
Amazon SNS:
Pros: out of the box solution, lesser development effort
Cons: limited customization
Websocket:
Pros: real-time notifications for active users
Cons: complex to set up and requires maintenance
Post DB:
Dynamo DB:
Pros: high scalability, fast write/read
Cons: lower data consistency compared to relational DB, does not support complex queries
My SQL:
Pros: ACID high consistency, supports complex queries.
Cons: less scalability for high throughput, slower write/read
User Metadata DB: same as POST DB
Interaction DB:
Dynamo DB:
Pros: can store flat user data such as username, fast read/write, high scalability, less costly
Cons: does not support complex queries
Graph DB:
Pros: extramely fast for connected data like followee/follower
Cons: less scalability, more costly
Try to discuss as many failure scenarios/bottlenecks as possible.
Photo service failure
DB failure
API Gateway failure
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
Video uploading capability
Auth security
more worker DBs for fault tolerance and failover