Loading...
DAU = 10K, with 3 activities logged in a day on an average = 30K activities/day, which is roughly 21 activities a minute.
Peak will 3x = 60 activities in a minutes or 1 activity/sec
The write speed is relatively low
Storage = 1MB/user/month = 10GB/month = 120GB/year = 600GB in 5 years
POST /users
Request:
{
name: string,
email_id: string,
dob: string,
gender: ENUM,
address: string
}
Response: {user_id: uuid, success: True}
PUT /users/{user_id}
Request
{
name: string,
email_id: string,
dob: string,
gender: ENUM,
address: string
}
Response: {user_id: uuid, success: True}
GET /users/{user_id}
Response
{
user_id: user_id
name: string,
email_id: string,
dob: string,
gender: ENUM,
address: string,
success: true
}
POST /activity
Request:
{
user_id: uuid,
activity_type: ENUM (selected from a limited option)
calories_burn: double (optional)
start_time: timestamp,
end_time: timestamp,
steps: int (optional)
}
Response:
{
activity_id: uuid,
success: boolean
}
GET /activity/summary/{user_id}/{time-range}
Response:
{
activities: [{sport: 'badminton', total_time: 120, total_cals burnt: 500, total_steps: 9789}, ....]
}
POST /goals
Request:
{
user_id: uuid,
activity_type: ENUM (selected from a limited option)
target_calories_burn: double (optional)
target_steps: int (optional)
target_date: string
}
This will return the list of activities, within this time period.
We can modify this later to support, only daily summary (upto 7 days), weekly summary (upto 8 weeks) and then monthly summary (Jan, Feb, March etc).
We can pre-aggregate the results and serve them to improve our read speed, as the users mostly care about their latest data in granular level, while aggregation for historical data is fine.
So, we have ios and android as clients, which call Load Balancer -> API Gateway.
We have 3 application Services, which users can call (Goals, Activity, User) and 2 internal Services (Notification, Activity Aggregator) which are run by cron jobs.
When a new user comes to our app, we will save all its information in the Postgress DB, and also add it to the queue to store in the Redis cluster (to show user profiles, which is mostly static). If there is an update to the user data, we will also update the cache async (but keep read your writes consistency)
When an user adds a Goal / activity, it is also persisted to Postgress DB.
We have cron services which run activity aggregator periodically to aggregate the daily / weekly / monthly data and write it back to Postgress DB for persistence and Redis for faster reads.
At our scale, where users are adding on an average 3 activities a day, we can asynchronously run the aggregation after every write, however on scale, we will be better off with a cron job, which we can tune to run every min/5 mins
We have a notification service, which will be run by a separate cron job, whose role is to check the users data (goals / activity) and create meaningful notifications for the user (All goals achieved for the day. Push hard, you still got time etc) -> add it to a queue and send a notification to the user
We will have an Integration Service that acts as an adapter for third-party platforms. It normalizes external data and feeds it into our Activity Service.
The backend design provides the client with the raw aggregated data. The graphs etc are rendered on the client side by using these data.
We will retry sending notifications incase of a failure, else it will end up in a Dead Letter Queue, which we can monitor for issues
Postgres fits our use case here, as the data is linked with each other. We need atomicity during the writes and read your own write consistency for the users.
Table:
User Table (uuid, name, email, address, created_at, updated_at)
Goals Table (uuid, user_id, activity, target_step, target_calories, target_date, created_at, last_updated_at) -> indexed at user_id
Activity Table (uuid, user_id, activity, steps, calories, start_time, end_time, created_at, last_updated_at) -> indexed at user_id
Activity Aggregation Table:
Daily Activity: (uuid, user_id, start_date, total_activiites, total_calories, total_steps)
Weekly Activity, Monthly Activity
All the tables are indexed by user_id, bcz our read pattern is based on per user.
Redis:
profile:{user_id} -> Value ({name, age, dob, gender})
aggregation:daily:{date}:{user_id} -> JSON
aggregation:week:{week_no}:{year}:{user_id} -> JSON
aggregation:month:{month_no}:{year}:{user_id} -> JSON
Redis data will be sharded by user id, so that the data of an user stays together in one shard
We will use Postgress Encyption at Rest to encrypt the data.
We will deep dive into Aggregator Service
Integrator Service will will fetch the newly added data / modified data and update the previously saved data.
This can be achieved by adding an event to the queue, when we add/modify/delete any data. The Activity Service will dump the data into the queue and aggregation will process it.
daily automated backups to cold storage (S3) with point-in-time recovery, tested restore process