Loading...
99.9% availability: Users expect the app to work whenever they decide to exercise. Morning and evening peak hours concentrate 60% of daily traffic into 4 hours.
Offline-first operation: Cellular signal drops during trail runs, basement gym sessions, and subway commutes. The app must record full workouts locally and sync seamlessly when connectivity returns. Zero data loss during offline periods.
Low-latency reads: Dashboard loads and activity history queries must return in under 200ms. Users check stats frequently throughout the day.
Horizontal scalability: The system handles 10 million registered users, 1 million daily active users, and peak ingestion rates of 50,000 concurrent workout streams.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
activitiy upload:
POST /v1/activities
Headers: Authorization, X-Idempotency-Key (client UUID)
Body: {
type: "running" | "cycling" | "weightlifting",
start_time: ISO-8601,
duration_seconds: number,
distance_meters: number,
calories: number,
heart_rate_avg: number,
heart_rate_max: number,
gps_track: base64-encoded compressed binary (optional),
source: "manual" | "phone_gps" | "apple_watch" | "garmin"
}
Response: 201 Created { activity_id: string, status: "saved" }
The X-Idempotency-Key header prevents duplicate uploads when the client retries after a timeout. The server checks this key against a Redis set (24-hour TTL) before processing. If the key exists, it returns the original response without creating a duplicate activity.
GET /v1/users/:user_id/activities?limit=20&cursor=last_activity_id
Response: 200 OK {
activities: [{ activity_id, type, start_time, duration, distance, calories }],
next_cursor: string | null
}
GET /v1/activities/:activity_id
Response: 200 OK {
activity_id, user_id, type, start_time, duration_seconds,
distance_meters, calories, heart_rate_avg, heart_rate_max,
gps_track_url: string (presigned S3 URL, expires in 1 hour),
route_polyline: string (encoded for map display)
}
POST /v1/goals
Body: {
goal_type: "weekly_distance" | "weekly_frequency" | "monthly_calories",
target_value: number,
start_date: ISO-8601,
end_date: ISO-8601 | null (null for recurring)
}
Response: 201 Created { goal_id: string }
GET /v1/users/:user_id/goals
Response: 200 OK {
goals: [{
goal_id, goal_type, target_value, period,
current_progress: number,
period_start: ISO-8601,
period_end: ISO-8601,
achieved: boolean
}]
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
The architecture separates three distinct workloads: real-time GPS ingestion (high-concurrency, low-bandwidth writes), activity storage (batch writes with relational queries), and dashboard reads (high frequency, cache-friendly). Each workload has different scaling characteristics, and the event bus (Kafka) connects them without tight coupling.
High-level architecture: mobile clients connect through API Gateway to Activity, Goal, and Analytics services. Kafka event bus coordinates async processing. PostgreSQL, S3, and Redis serve different data workloads.
Level Expectations
Mid-level: draw the API Gateway, Activity Service, and a single database. Mention caching for dashboard reads.
Senior: separate the GPS ingestion path (WebSocket, Redis buffer, S3 flush) from the activity upload path (REST, PostgreSQL) and the dashboard read path (Redis cache). Explain why Kafka decouples these paths.
Staff: design the GPS ingestion scaling strategy for 500K concurrent workouts, reason about backpressure when Redis buffers fill faster than S3 flushes, and explain partition key choices for the Kafka activity topic.
Mobile app to API Gateway: The client sends workout data through the API Gateway, which handles SSL termination, authentication token validation, and rate limiting. The gateway routes requests to the appropriate backend service based on the URL path.
Activity Service: The core service for workout data. It receives activity uploads, validates the payload, stores the summary in PostgreSQL, uploads the GPS track to S3, invalidates the user's dashboard cache in Redis, and publishes an activity_created event to Kafka. This service is stateless and horizontally scalable.
Kafka Event Bus: When an activity is created, the event is consumed by multiple downstream services independently. The Goal Service checks if any goals were advanced. The Analytics Service updates aggregated stats. The Notification Service triggers alerts for milestones. This decoupling means adding a new consumer (such as a social feed service) requires zero changes to the Activity Service.
Dashboard reads: The most frequent read operation. The app requests the user's weekly summary, goal progress, and recent activity list. The API server checks Redis first. Cache hits (the common case for active users) return in under 5ms. Cache misses trigger PostgreSQL aggregation queries, which complete in 20-50ms and populate the cache for subsequent requests.
Activity detail reads: When a user views a specific workout's route map, the API server returns the summary from PostgreSQL and a presigned S3 URL for the GPS track. The client fetches the GPS data directly from S3.
GPS streaming: During an active workout, the phone maintains a WebSocket connection to the Ingestion Service. GPS points are buffered in Redis sorted sets and flushed to S3 in periodic batches (every 30 seconds or when the workout ends). This separation means a dropped WebSocket connection does not lose already-buffered data.
Two request flows define the system's behavior: the activity upload flow (writing a completed workout) and the dashboard read flow (displaying stats and history). Understanding each step reveals why the architecture is structured the way it is.
Activity upload: client sends workout to API Gateway, which routes to Activity Service. Service stores summary in PostgreSQL, uploads GPS to S3, publishes event to Kafka for downstream processing.
Step 1: Client sends activity payload: After the workout ends (or when the phone regains connectivity), the mobile app sends POST /v1/activities with the workout summary and compressed GPS track. The X-Idempotency-Key header is a UUID generated when the workout started.
Step 2: API Gateway validates the request: The gateway verifies the OAuth token, checks rate limits (max 10 uploads per minute per user), and routes to an Activity Service instance.
Step 3: Idempotency check: The Activity Service checks Redis for the idempotency key. If found, it returns the original response (the activity was already created). If not found, it adds the key to Redis with a 24-hour TTL and proceeds.
Step 4: Write to PostgreSQL: The service inserts the activity summary row (type, duration, distance, calories, heart rate stats) with the idempotency_key. The database unique constraint provides a second deduplication layer.
Step 5: Upload GPS track to S3: The compressed GPS track is uploaded to S3 with the key user_id/activity_id.gpx.gz. The S3 key is written to the activity row's gps_storage_key column.
Step 6: Invalidate dashboard cache: The service deletes the dashboard:user_id key from Redis, ensuring the next dashboard request reflects the new activity.
Step 7: Publish Kafka event: An activity_created event containing the activity_id and user_id is published to Kafka. This triggers downstream processing: the Goal Service evaluates progress, the Analytics Service updates aggregations, and the Notification Service checks for milestone alerts.
Step 8: Return 201 Created: The client receives the activity_id and confirmation. Total latency: 100-300ms (dominated by the S3 upload).
Interview Tip
Notice the ordering: PostgreSQL write, then S3 upload, then cache invalidation, then Kafka publish, then response. Each step depends on the previous one succeeding. If you swap the order (publish to Kafka before S3), downstream consumers would try to fetch a GPS track that does not exist yet.
Dashboard read: API server checks Redis cache first. On miss, queries PostgreSQL for weekly stats and recent activities, populates cache, and returns response.
Step 1: Client requests dashboard: The app sends GET /v1/users/:user_id/dashboard on launch and pull-to-refresh.
Step 2: Cache check: The server looks up dashboard:user_id in Redis. For active users who have not just completed a workout, this is typically a cache hit (5-minute TTL with event-driven invalidation).
Step 3a: Cache hit: Redis returns the cached dashboard data (weekly distance, activity count, goal progress, current streak). Response time: under 5ms.
Step 3b: Cache miss: The server queries PostgreSQL: aggregate this week's activities for the user (SUM distance, COUNT activities), fetch goal progress, and load the 5 most recent activity summaries. The composite index on (user_id, start_time DESC) makes these queries efficient. Response time: 20-50ms. The result is written to Redis with a 5-minute TTL.
Step 4: Return dashboard payload: The client renders the dashboard with weekly stats, goal progress bars, and the recent activity list.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
The data model splits across three storage systems, each chosen for the access pattern it serves. The interesting decision is not the schema itself but why a single database cannot serve all three workloads efficiently.
users
user_id UUID Primary Key
email VARCHAR Unique, indexed
name VARCHAR
password_hash VARCHAR
created_at TIMESTAMP
activities
activity_id UUID Primary Key
user_id UUID Foreign Key -> users
type VARCHAR (running, cycling, weightlifting)
start_time TIMESTAMP Indexed (user_id, start_time DESC)
duration_sec INTEGER
distance_m INTEGER
calories INTEGER
hr_avg SMALLINT
hr_max SMALLINT
gps_storage_key VARCHAR (S3 key for GPS track)
source VARCHAR (manual, phone_gps, apple_watch)
idempotency_key UUID Unique, indexed
created_at TIMESTAMP
goals
goal_id UUID Primary Key
user_id UUID Foreign Key -> users
goal_type VARCHAR (weekly_distance, weekly_frequency)
target_value INTEGER
start_date DATE
end_date DATE (nullable for recurring)
created_at TIMESTAMP
The composite index on (user_id, start_time DESC) enables the most common query: "show me my recent activities" without a full table scan. The idempotency_key column with a unique constraint prevents duplicate uploads at the database level, a second layer of defense after the Redis check.
Why PostgreSQL for this layer? Activity summaries involve relational queries: joining activities with goals to compute progress, filtering by date ranges, aggregating distance by week. These are SQL's strengths. The data volume (90 GB/year for summaries) is well within PostgreSQL's comfort zone.
Bucket: fitness-gps-tracks
Key pattern: {user_id}/{activity_id}.gpx.gz
GPS tracks are stored as compressed GPX files in S3. Each track is write-once (uploaded after workout completion) and read-rarely (only when a user views the route map). S3 provides effectively unlimited storage at a fraction of PostgreSQL's per-GB cost.
The key includes user_id as a prefix for two reasons: S3 distributes objects across partitions based on key prefix, and grouping by user enables efficient bulk exports ("download all my data").
Key: dashboard:{user_id}
Value: { weekly_distance, weekly_activities, goal_progress, streak }
TTL: 300 seconds
Key: idempotency:{key}
Value: { activity_id, status }
TTL: 86400 seconds (24 hours)
Key: active_stream:{activity_id}
Value: sorted set of GPS points (score = timestamp)
TTL: 14400 seconds (4 hours, max workout duration)
Redis serves three roles: caching dashboard data to avoid repeated PostgreSQL aggregations, storing idempotency keys for upload deduplication, and buffering live GPS stream data before it is flushed to permanent storage.
A single PostgreSQL instance could technically store everything: summaries, GPS tracks as BYTEA columns, and cache data in materialized views. The problem emerges at scale. GPS blobs in PostgreSQL inflate table size, slow VACUUM operations, and compete for shared buffer cache with the high-frequency summary queries. Separating hot relational data from cold binary blobs lets each storage layer operate in its sweet spot.
The core engineering challenge (the GATE of this problem) is building a reliable offline-to-online sync pipeline that handles intermittent connectivity, duplicate uploads, and concurrent multi-device edits without losing or corrupting workout data. Every component in this section exists to solve some aspect of that challenge.
Key Insight
The GATE insight: the fundamental difficulty is that the client, not the server, is the source of truth during a workout. The phone records data for an hour with no server contact, then must sync reliably when connectivity returns. This offline-first inversion drives the idempotency design, conflict resolution, and the entire sync protocol.
Offline sync: the mobile app records workouts locally in SQLite, queues them for upload when connectivity returns, and uses idempotency keys to prevent duplicates.
The mobile app maintains a local SQLite database that mirrors the server's activity schema. During a workout, all data is written to SQLite first, regardless of network availability. When the workout ends, the app adds the activity to an upload queue. A background sync worker processes this queue whenever the network is available.
Queue processing: The sync worker reads the oldest unsynced activity from SQLite, sends it to the API with the idempotency key (generated when the workout started), and marks it as synced only after receiving a 201 response. If the request fails (timeout, 5xx error), the worker retries with exponential backoff. If the server returns 409 Conflict (idempotency key already exists), the worker marks the activity as synced without creating a duplicate.
Multi-device conflict resolution: A user edits an activity title on their phone while their watch simultaneously uploads a heart rate correction. The server uses last-write-wins with per-field granularity: the phone's title change and the watch's heart rate update are independent fields, so both are applied. For conflicting edits to the same field (both devices change the title), the server accepts the write with the later client-side timestamp. The activity row includes a version number that increments on every update, enabling optimistic concurrency control.
GPS ingestion: mobile app streams points via WebSocket to the Ingestion Service, which buffers in Redis and flushes to S3 in batches.
The GPS Ingestion Service manages WebSocket connections for live workout tracking. Each active workout maintains a Redis sorted set where incoming GPS points are stored with their timestamp as the score.
Buffer and flush cycle: Every 30 seconds, a background job reads the sorted set for each active workout, serializes the points into a compressed GPX segment, appends it to the workout's S3 object, and trims the sorted set. This batching reduces S3 writes from 3,600 per hour (one per second) to 120 per hour (one per 30 seconds).
Connection recovery: When a WebSocket connection drops (common on mobile), the client buffers GPS points locally. Upon reconnection, it sends all buffered points in a single batch. The Redis sorted set handles out-of-order arrival naturally: points are ordered by timestamp score regardless of when they arrive.
Scaling: Each Ingestion Service instance handles roughly 10,000 concurrent WebSocket connections. At 50,000 peak concurrent workouts, 5 instances are needed. Auto-scaling monitors active connection count and launches new instances when utilization exceeds 70%.
Goal evaluation: Kafka delivers activity events to the Goal Service, which computes progress, updates the database, and triggers notifications for achieved goals.
The Goal Service consumes activity_created events from Kafka and evaluates each against the user's active goals. The evaluation process queries PostgreSQL for the user's goals and current-period activity totals, computes whether any goal was advanced or completed, and updates the goal progress record.
Batched evaluation: When a user logs multiple activities in quick succession (such as during a bulk import), the Goal Service deduplicates evaluations: if multiple events for the same user arrive within a 5-second window, they are processed as a single batch, querying the database once instead of N times.
Notification trigger: When a goal is achieved, the Goal Service publishes a goal_achieved event to Kafka. The Notification Service consumes this event and sends a push notification to the user's devices. The decoupling ensures that goal evaluation and notification delivery are independent: a slow notification provider does not delay goal processing.
Every architectural decision involves a trade-off. Understanding these trade-offs, and being able to articulate them in an interview, demonstrates deliberate design thinking.
PostgreSQL (our choice) provides ACID transactions, rich query capabilities (joins for goal evaluation, aggregations for dashboards), and strong consistency for activity records. The trade-off: vertical scaling limits throughput to roughly 10K writes/sec on a single primary. At 500K activities/day (6/sec average, 100/sec peak), PostgreSQL handles this comfortably.
Cassandra or DynamoDB would offer unlimited horizontal write scaling. But activity queries require joins (activities with goals), range aggregations (sum distance this week), and transactional consistency (idempotent uploads). NoSQL databases make all of these harder. The data volume (90 GB/year) does not justify the operational complexity of a distributed NoSQL cluster.
Our reasoning: use SQL until you outgrow it. At 10x the current scale (10M DAU, 5M activities/day), you would shard PostgreSQL by user_id or migrate hot paths to DynamoDB. Premature NoSQL adoption is a common interview mistake.
Common Pitfall
A common interview pitfall is choosing Cassandra or DynamoDB 'for scale' when the data volume is under 100 GB/year. Interviewers want to see you justify the complexity. At 90 GB/year, PostgreSQL with proper indexing handles the workload. Save NoSQL for when you have concrete evidence that SQL is the bottleneck.
Offline-first (our choice) means the client stores all data locally and syncs to the server asynchronously. This maximizes reliability during workouts but introduces complexity: sync queues, idempotency, conflict resolution, and storage management on the client.
Server-first means the client sends data to the server in real-time and relies on the server as the source of truth. Simpler to implement, but any connectivity loss during a workout results in data loss.
Our reasoning: for a fitness app, data loss during a workout is unacceptable. Users cannot re-run their morning 10K. The complexity of offline-first is justified by the domain requirement.
WebSocket (our choice) maintains a persistent connection for streaming GPS points every second. Efficient for high-frequency data, but requires connection state management on the server and complicates load balancing (sticky sessions or connection-aware routing).
HTTP polling would have the client POST GPS points every N seconds. Simpler server architecture (stateless), but at 1-second intervals the TCP handshake overhead wastes 10-20% of each interval on mobile networks.
Our reasoning: the 1-second GPS sampling rate makes WebSocket the clear winner. If we relaxed to 30-second intervals (like some hiking apps), HTTP polling would be viable and simpler.
The system's reliability depends on graceful degradation: each component can fail independently without losing user workout data. Every failure scenario maps to a specific architectural decision.
Scenario: S3 returns errors during a burst of activity uploads after a popular evening workout window.
What happens: The Activity Service cannot upload GPS tracks. Rather than failing the entire upload, the service writes the activity summary to PostgreSQL with gps_storage_key set to "pending." The GPS track is queued in a retry buffer (either in-memory with disk overflow or in Kafka). A background job retries S3 uploads with exponential backoff. When S3 recovers, pending tracks are uploaded and the activity record is updated.
Why this works: The user sees their workout summary immediately. The GPS route map shows "processing" briefly. No data is lost because the client retains the GPS track until the server confirms receipt.
Scenario: The Redis cluster loses a node, reducing cache capacity by one-third.
What happens: Cache hit rate drops from 95% to roughly 65%. Dashboard reads that would hit Redis now fall through to PostgreSQL. PostgreSQL read replicas absorb the additional load, increasing dashboard latency from 5ms (cached) to 30ms (database). The idempotency check falls back to the PostgreSQL unique constraint, adding 10ms to upload latency.
Why this works: Redis is a performance optimization, not a correctness requirement. The system functions correctly (just slower) without it. When Redis recovers, the cache repopulates organically through cache-aside on reads.
Interview Tip
A strong interview answer categorizes each component's failure mode: Redis failure degrades performance, Kafka failure delays downstream processing, S3 failure delays GPS availability. Only PostgreSQL failure would actually block activity uploads. This hierarchy of failure impacts shows you understand which components are on the critical path.
Scenario: Kafka becomes unreachable, preventing the Activity Service from publishing events.
What happens: Activity uploads still succeed (PostgreSQL write is the critical path). The Activity Service writes undelivered events to the PostgreSQL outbox table. Goal evaluations and analytics updates are delayed until Kafka recovers and the outbox is drained. Users see their new activities but goal progress may lag briefly.
Why this works: The outbox pattern decouples the critical path (activity storage) from the event distribution path. Kafka unavailability degrades timeliness, not correctness.
Scenario: A major running event causes 100,000 simultaneous WebSocket connections, 2x the normal peak.
What happens: Auto-scaling detects the connection count spike and launches additional Ingestion Service instances. During the scale-up window (1-2 minutes), some new connections are rejected with a 503 status. The mobile app falls back to local buffering and retries the WebSocket connection with exponential backoff. GPS data is never lost because the client always records locally first.
Why this works: The offline-first design means server-side capacity constraints never cause data loss. Users recording their marathon are unaffected; the WebSocket stream is an optimization for live tracking display, not a requirement for data capture.