Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
Assume we have 100000 daily active users and an average user has 5 interactions per day. Assume we store user activity progress on local device and sync it to the cloud every hour, and upon user completing an activity - we should safely assume every user has around 40 requests a day. That makes it 40 * 100k = 4M requests a day and around 40 requests/seconds
Traffic is not very high but it could surge when a large number of users sync data together (e.g. everyone sync every hour on the hour).To mitigate this, the system can try to spread out user's sync time - not syncing every hour on the hour, instead, get a random number between 0 and 3600 seconds for each user and sync that user's data at hour + randomNumber seconds.
This amount of traffic could be handle by one single server, however, in order to ensure availability, we should have multiple servers.
Assume each user activity would need 50 bytes of data, an average user has 3 activities a day, and each user's daily progress would require 50 bytes of data, the meta data of each user would require 100 bytes of data.
Every user would then need (50 * 3 + 50) * 365 + 100 = 200 * 400 = 80000 = 80KB data per user, and 80KB * 100K = 8GB of data in total.
This data can be stored in a single machine, however, in order to ensure data availability and avoid lost of data, we should have multiple backups of the database.
Define what APIs are expected from the system...
startActivity(activityType ActivityType) -> activitySessionId
Records starting of an activity. ActivityType should be an enum indicating the actual activity (e.g. running, jogging, swimming).
API returns a activitySessionId to record the activity.
endActivity(activitySessionId)
Records ending of the activity corresponding to the session id.
setGoal(targetDuration Duration)
Set the daily goal for activity duration.
syncProgress(currentDuration Duration)
Sync the current progress of the user.
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
User
id - auto increment, PK
first_name - varchar
last_name - varchar
current_target_id - FK
current_daily_progress - FK
user_daily_target
id - auto increment, PK
user_id - User table FK
daily_target_in_minutes - integer
start_date - date
end_date - date
user_daily_progress
id - auto increment, PK
daily target id - FK
progress_date - date
current_progress_in_minutes - integer
user_activity
id - auto increment, Pk
user_id - FK
started_at - dateTime
last_recorded_at - dateTime
activity_type - enum
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
The service itself should not be complicated and requires the normal components - client, server and database.
For analytics purposes, there are two additional components. A cron job (or jobs) reading from the database periodically and write to a seperate analytics datastorage.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
As shown in the request diagram, the request flow is as follows for the requests:
For startActivity, the client sends a request to server with an activityType that the user selected, the server then inserted a new row in user_activity table for this activitysession. Once the server has the activity id, it sends it as a response to the client.
For endActivity, the client sends a request to server with an activitySessionId to the client. The server then update the corresponding record in the user_activity table. The last_updated_at timestamp will be set to current timestamp.
For setGoal, the client sends a request to server with a targetDuration that is set by user using the app. The server will write to database in one transaction to complete the following steps: 1. update the user's current target and sets its end date to current date. 2. Insert into the user_daily_target a new row for the new target. 3. Update the current row for the user's user_daily_progress table to set the goal to the new goal. The server will then returns a response indicating goal is set successfully to the client.
For syncProgress, the client will record the progress of the user locally, and sync call this api every hour. The client sends a request with the currentDuration which represents the total time the user has spent in activities in the day to the server. The server will update its user_daily_progress with this. It can do a sanity check that the new value must be at least as big as the previous value, and should not be greater than the previous value + 60. Eventually the server sends a response to the client indicating that the progress is synced successfully.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Client: The client contains not only the UI but also a portion of user data, specifically, the user progress data. To prevent lost of data, the client should sync with server in the following situation: 1. every hour at a certain offset. 2. when user completes an activity, it should call endActivity as well as syncProgress. 3. when a user close the app.
Analytics datastore: The app itself seldom needs to access historical data, therefore, in order to keep database efficient, it should only contains latest 1 year of data. The historical data, however, could be valuable for analytics purposes, therefore, a cron job should fetch them periodically and store all data in this analytics datastore and store them in format that would be more efficient and convenient for analytics purposes.
Explain any trade offs you have made and why you made certain tech choices...
In this design, 1 hour periodic syncing of progress is chosen over a more aggressive approach - such as 1 second or 1 minute. The main advantage of this approach is to reduce the traffic going to the server, and subsequently the traffic going to the database. This will increase the risk of losing data, however, as we will also sync when the user is closing the app or when the user stops an activity, the risk of losing data is mitigated a bit. Besides, activity tracking data is not super critical compares to many other applications, very occasional data loss might be acceptable.
Try to discuss as many failure scenarios/bottlenecks as possible.
If client crashed or client device crashed, the worst scenario would be data lost for the client's most recent one hour of data or the data since the client last online. This is a tradeoff we have in order to keep the system simple and easy to maintain, so that we will not have to deal with a large amount of sync data and syncing them to database simultaneously.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
In the future, we could add more servers and increase the sync frequency of user progress. When the frequency is high, database could become a bottlenet. To handle more traffic, the database could be partitioned according to user ids. Besides, a request could be sent by a client in a fire and forget manor, as one or two requests failing should not affect overall performance if the sync frequency is high enough.