Requirements

Functional Requirements:

• User should be able to upload their workouts and activities.
• User should be able to set goals.
• User should be able to track their progress for the goals.
• User should be able to view historical data or basic stats

Scale estimations

• 100M DAU ~ 10% records works out daily so 10M daily writes./
• ~120 writes/sec.
• 90% reads so ~1000 reads/sec

Non-Functional Requirements:

• Consistency > Availability - User's progress and goals should always be consistent and data should be synced to all devices
• Highly scalable - to support 1B unique workout records
• Low latency - System should be quick and efficient even during high traffic
• High read throughput - System should handle reads efficiently as system is read heavy.
• Workout records should be idempotent to ensure no duplicate creation

API Design

Data Entities:

Workout

Id

UserId

ActivityTypeId

Duration

Calories

Distance

Intensity - HIGH, LOW, MEDIUM

StartDate

EndDate

Goal

Id

UserId

GoalType

TargetValue

CurrentValue

StartDate

EndDate

UserStats

Id

UserId

TotalWorkouts

TotalDistance

TotalCalories

Lookup (optional / enum-based)

• ActivityType (id, name, code)
• GoalType (id, name, code)

POST /workout - creating/recording workouts

request - {

userId,

activityType,

duration,

distance,

calories,

startDate,

endDate

}

response - {workoutId}

POST /goal - used for creating goals

request -

{

goalType,

targetValue,

startDate,

endDate

}

response - {goalId}

GET /workouts?startDate=&endDate=&page=

GET /goals

response

LIST

GET /goals/{goal_id}/progress - returns progress of a goal

response - {

goalId,

targetValue,

currentValue,

progressPercent

}

GET /users/{userId}/stats

response : UserStats

High-Level Design

API Gateway

Acts as the entry point for all client requests.

Responsibilities:

• Authentication (if required)
• Rate limiting
• Request routing

Load Balancer

Distributes traffic across multiple instances of the Read and Write services to improve scalability and availability.

Rate limiting

Handled at API Gateway using token bucket per API key/IP to prevent abuse and control write traffic.

Workout Service:

Responsibilities:

• Create workouts
  • Uses idempotency key to ensure duplicate records are not created
• Fetches workout history
  • First page of history is stored in Redis, so if cache hit, return immediately
  • If cache miss, get data from DB, stgore in Redis and return.
• Publishes workout created event to message queue

Goal Service:

Responsibilities:

• Creates goals
• Fetches goal
• Fetches goal progress
  • Used redis here to fetch goal progress, if found return from redis,if cahce miss go to DB.
• Subscribes to the workout created event and updates the goal progress accordingly.

Stats service:

Responsibilties:

• Fetches user basic stats.
• Flow:
  • Check Redis: if hit, return immediately
  • If miss, fetch from DB table UserStats
  • Store it in redis
  • Return data
• Subscribes to the workout created event and updates the stats in redis if present and as well as in DB.

Redis:

• Acts a cache layer, use to cache frequently accessed data.
• We will store:"
  • User stats : userId -> stats
  • Recent workouts : last 10-30 workouts
  • Goal progress
  • First page of workout history
• All the above is shown is user dashboard, so all of this will be cached.
• TTL-based + cache-aside pattern

MessageQueue:

Responsibilties:

• Acts as a shared event bus.
• All workout realted event gets published here, Goal serviec consumer consumes those events and update goal progress.

Detailed Component Design

We will now deep dive into the following topics

1) Offline Sync + Idempotency Key:

• Whenever a user is offline, opens app and uploads/records workout, the system needs to ensure that workout is not lost.
• To achieve this ofcourse the simplest solution would be to just not allow user for any activity when they are not connected to internet, but if system needs that capability the approach we will take is:
  • When user is offline and uploads/records workout, that workout is stored locally in client side offline storage such as SQLLite/local DB.
  • Each workout gets a workoutID i.e. UUID
  • Then when user is back online, it bactches all unsynced workouts to the server.
  • The workoutId along with userId acts as an idempotencyKey, so we will have an uniqueConstraint on those workoutId and userID in table, this avoid duplicates being created.
  • In case if sync fails midway, the client sends same batch again, and sicne we have unique constraint, the already succeed records are ignored

2) Kafka Event delivery Consistency

• How do we ensure messages are not lost when goal service or stat service are down?
  • When kafka reciveces messages and those messages are consumed by workers, those workers might have issues and those message wouild fail and eventually reach max retires and gets lost forever.
  • To ensure this does not happen, we use DLQs. so once a meesage reaches max retry count, that message is sent to DLQ, in DLQ that message will live for 14 days.
  • We can setup alerts when any message hits in DLQ that gives us to act on issues right away.
  • Optionally, we can have an additional process that runs periodcally and clears our DLQ queue. As well as, messages from DLQ can be sent back to source queue manually after all issues are resolved.
• How do we ensure that when Kafka recieves duplicate events, those duplicate events is not processed twice, leading to wrong stat and goal progress updates.
  • The solution for this is having consumer check eventId.
  • So on consumer side, we store all eventIds processed in Redis.
  • When consumer processes a message:
    • Checks redis -> if event id exists -> skip
    • if event id doese not exist -> message gets processed

3) How will system handle high availability and fast response times

• Services can be scaled horizontally behind a load balancer. Redis reduced DB load by serving frequenctly accessed data. Kafka decouples writes from downstream processing, all this contributes towards fast response times