Loading...
see basic stats
GET /api/v1/stats
analytic
GET /api/v1/analytics
see workout history, optionally filter to specific workouts
GET /api/v1/workouts?type=X&start-date=Y&end-date=Z
see goals, optionally filter to goals about a specific type of workout
GET /api/v1/goals?type=X
set goals
POST /api/v1/goals
{
type:
distance?
reps?
duration?
weight?
}
view a specific goal
GET /api/v1/goals/{id}
POST /api/v1/goals/{id}
DELETE /api/v1/goals/{id}
log a work
POST /api/v1/workouts
{
type:
duration:
exercises: [
{
id:
name:
type:
parameters: {
distance?
duration?
sets?
reps?
weight?
}
}, ... {...}
]
notes:
}
syncing data to/from other devices
GET /api/v1/workouts/import
{
timestamp: now
}
POST /api/v1/workouts/export
{
timestamp: now
}
GET /api/v1/goals/import
{
timestamp: now
}
POST /api/v1/goals/export
{
timestamp: now
}
We'll start by introducing an API Gateway. the API Gateway serves a couple of purposes the mains are Authentication and authorization, load balancing and rate limiting. So to handle bursts and of traffic we'll use a rate limiting strategy of bucket tokens. So that each user has a set amount of tokens per hour and when a user consumes all over their tokens and keeps sending requests, they gets a message of '429 too many requests' with The field in the header indicating when to retry again and a time for that.
We'll split the system into three different types of servers: a workout read server a workout write server and an analytics server. Each of these servers can scale independently of the other and each of those. Servers Sit behind a dedicated load balancer.
That way we distribute the load based on the requested query. And now since the servers are independent they are isolated, and when one functionality of the system fails other servers are unaffected and can still serve their function For example if the workouts write server fails reading previous workouts still works and viewing goals and analytics about the user still works.
To handle burst of writes for our servers will introduce a dedicated right message queue. So that messages would be consumed in a way that will not overwhelm the workouts write servers.
To handle Offline activities the various devices of the same user would keep data in a local storage. So that they would be kept and not lost. After a log of a workout was made, every set amount of time for example 1 minute The app would try to export that data via the suitable export endpoint. In case of some Network failure and other logs of workouts The data would be accumulated and would be tried to be exported at the same time.
So how does the synchronization process looks like so the user devices will temporarily store without data locally. Let's say a network error has occurred, and the device reconnects to the internet.
The device will then check for any uploads that needs to be done. Then it would send those to the write server. The server would then compare the idempotent keys of the requested log uploads, remove the existing duplicate logs that exists within the database. And then and only then It would write the new entries to the database.
To know that an import is needed we could introduce a server-side event from the workout server to the
Other devices of the user notifying that an Import is pending.
Other devices would then call the import endpoints in order to synchronize, again in a set amount of time, for example one minute, until the data is retrieved. at that point we would stop until needed again.
To avoid data duplication we'll use and idempotent key of user id, type of exercise and timestamp of that exercise. that way when the write servers receive the request to update the data they would start by checking if the idempotent key exists, otherwise they would write/update the data to the database.
For analytics will introduce a separate message queue for async work. Each time a user posts a workout log it will add a new task to the analytics message queue for the analytics that will be processed using an async worker in the background.
The async worker uses the workouts read servers to read freshly added data. it perform the various analytics works it needs and writes the results back to the original database(for goals and workout logs) and a dedicated analytics database that is suitable for analytics data(time-based, mostly writes).
Then when the user queries for analytics the data is already prepared. there is however a slight lag of processing new data. so to still show relevant data to the user we can introduce a Redis cache to hold previous data. The data might be a bit stale but it's still informs the user and of information instead of no information at all. And then the data would be invalidated with a TTL with a value of 5 minutes(or less, we can fine tune this value based on calculation time metrics). that way we achieve eventual consistency of the analytics server.
for the data layer we'll use:
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
We have 100 million daily active users with 20 queries per day per user. We know the
Reads to rights ratio is 20 to 1. Which means we have roughly 20K QPS for reads and 1K QPS for writes if we assume. Uniformly distribution throughout the day.
So we need to scale the various servers differently. we'll use multiple servers per server type and distribute the load using an layer 4 load balancer in a round robin strategy.
To support High availability we'll introduced sharding and the shard key is the user id. that way all related data to that user is kept at the same shard - goals and workout logs.
For durability we'll introduce secondary databases that will serve as replicas and snapshots that will occur every hour
for fault tolerance if the database fails we switch to the hot secondary database.in the case where everything fails. Will have logs to reconstruct recent activities/events from the current hour and add those Deltas to the latest snapshot.
In case of network failures we rely on local storage of the client devices to keep data until network connection resumes, which then triggers the synchronization strategy using imports/exports.
In the case of a spike of traffic we introduced a API Gateway to spread the load. we also have dedicated load balancers for the various servers, and we introduced a dedicated write message queue to handle a surge of write requests and not overwhelm the write server.
In the case the analytics worker fails no data should be written to either of the original database or the analytics database. so every work would be computed in the worker and then upon finishing the job It would be transactional, meaning the data is written to both the databases or to none of them. in case the worker fails the task still sits in the analytics message queue and would be picked up by another worker.
In the case of a analytics cache failure, the cache is only advisory, so the analytics server would query the analytics database Itself which will result in lower latency but it would still function the same because again the cash is only advisory. the cache will be built again by users querying for their information once the cache is back up.
the same could be said about a cache for the workout reads server.