Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
Total viewers per min - 10000, per hour 600000
Total number of videos - 100 per min, per hour 6000
peak traffic - 5times average
Very High read compared to write.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
Create a video
POST /views/create
Update view count of a viewed video.
POST /videos/{videoId}/views
body {user, timestamp}
Get total views of that video.
GET /videos/{videoId}/views
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Load Balancer - Manages load
API Gateway - Monitors user with the help of IP address, It also has rate limiter to identify request by bots.
SQS Queue - This manages the write server. In case of failure, the message is routed to Dead Letter queue where it is re-ingested with the help of cron-job or manual API triggering.
Write Server - This updates the views count and check for duplicate views. It also batch/aggregate before writing to DB.
Read Server - This returns total number of views from redis cache.
Redis cache - Cache
Database - use dynamodb
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
Need fast access and there are no strict relational requirement. So we use DynamoDB
Video Aggregator Row.
{
PK: VIDEO#123,
SK: METADATA,
Views: 1234,
Like: 123
}
User History
{
PK: User#123,
SK: Viewed#Timestamp,
VideoId: Vid_123,
Duration: 450
}
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
SQS consumer write service will first batch based on 5 sec timestamp and it will write to DB after that.
If there is failure during this DB write, then the whole batch is redirected to Dead Letter Queue and we have a cron job running, which will re-ingest the failed messages back to SQS. During replay, we will keep in mind, the timestamp of the message and if it is old message, then we will not increase the views for that message. Additionally, we can check user history to decide if the video has already been viewed by the user.
Based on User History Table, we will see the timestamp and duration. If the duration is very less <10secs, then we don't consider it as actual view count.
In case of Cache miss, we will read information from DB directly and will update the cache accordingly.
The cache for a particular video must be invalidated if there is any write operation happening for the same service. By default, TTL will be 1 hr max.
During the 5 sec timestamp batching, we will group by video_id and increase the count at once. We will also have optimistic locking in place for the count.
For rate limiting, we will use sliding window and this logic will live inside API Gateway.
There will be reconciliation based on number of messages in the DB and in the incremental count and those mesages will be reconcilled.
If the DB is throttling , then instead of writing to the DB, we will push the messages back to the SQS. THis will handle it.