Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
Here's the API to see the view count of a video:
GET v1/retrieve_view_count {
video_id: UUID,
user_id: UUID
}
Here's the API to document a video view event:
POST v1/write_watch_event {
video_id: UUID,
user_id: UUID,
watch_start_time: Timestamp,
watch_end_time: Timestamp,
video_metadata: String,
}
Here's the API to get video count trend analysis for content creators:
GET v1/view_video_counts_trend {
video_id: UUID
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
In our system, the first layer is an API gateway that does authentication, rate limiting, IP analysis, CAPCHA etc. to authenticate and authorize users. Only authorized users can watch a video.
On the write path, every time a video is watched, we stream the video from CDN to the user. Once the user finishes watching the video or exits watching the video, we send a request to the video count service, documenting details of the watch event. The watch event gets written directly to a redis cluster, as well as to cassandra database through Kafka.
On the caching layer, we use a redis cluster, the data model looks like:
key: video_id
value: video_view_count
To handle our high throughput, the redis cluster is horizontally scaled with leader-follower replication. The leader handles all the write traffic, while the follower handles read traffic. The writes to leader are asynchronously synced to the followers. This could lead to brief inconsistency due to the replication lag. However, it is an acceptable tradeoff for better availability and lower latency, as video count doesn't have to very accurate.
Due to the We also need to shard the data by video_id, so counts for different videos are stored in different leader-follower groups.
We use consistent hashing of video_id to ensure a even distribution of the traffic. If some videos are hot keys, causing traffic to disproportionately route to certain leader-follower groups, we will split the hot keys to several different groups, and aggregate them during read time. This could introduce latency for the video count of these hot videos. However, it is an acceptable tradeoff to ensure availability of the service.
For our cache, it is a write-back cache, which means that writes are written to the cache first, and asynchronously written to the database. This ensures that we have faster writes. However, if the cache crashes before data is pumped into the downstreams database, we risk losing data. This is a durability tradeoff that is acceptable in our use case.
For the database, we will use a no-SQL database, cassandra, that is highly scalable horizontally, and can handle huge write throughput. It is better for our use case than a relational database, because:
For the cassandra database, we store both the simple video count, as well as the detailed watch event.
For the video count:
table video_count {
video_id: UUID,
video_count: Integer
}
For the complete video watch event:
table video_watch_event {
video_id: UUID,
user_id: UUID,
watch_start_time: Timestamp,
watch_end_time: Timestamp,
video_metadata: String,
}
In our write path, when a write event happens, the video count service writes to the redis cluster, which then async propogates the data to the cassandra database. In cases of cache misses, the cassandra database will handle the reads with eventual consistency and low latency.
In the meantime, we also trigger a write event to be ingested on kafka, and consumed by a video event writer. The writer writes the complete watch event with metadata into a separate table in cassandra. This table is used by the video analytics service, to analyze watch trends, patterns etc.
On the read path, we go to video count service to fetch video count from redis and cassandra.
For content creators, they fetch the complete analysis of their video's watch patterns from our video analytics service.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
In our system, the first layer is an API gateway that does authentication, rate limiting, IP analysis, CAPCHA etc. to authenticate and authorize users. Only authorized users can watch a video.
On the write path, every time a video is watched, we stream the video from CDN to the user. Once the user finishes watching the video or exits watching the video, we send a request to the video count service, documenting details of the watch event. The watch event gets written directly to a redis cluster, as well as to cassandra database through Kafka.
On the caching layer, we use a redis cluster, the data model looks like:
key: video_id
value: video_view_count
To handle our high throughput, the redis cluster is horizontally scaled with leader-follower replication. The leader handles all the write traffic, while the follower handles read traffic. The writes to leader are asynchronously synced to the followers. This could lead to brief inconsistency due to the replication lag. However, it is an acceptable tradeoff for better availability and lower latency, as video count doesn't have to very accurate.
Due to the We also need to shard the data by video_id, so counts for different videos are stored in different leader-follower groups.
We use consistent hashing of video_id to ensure a even distribution of the traffic. If some videos are hot keys, causing traffic to disproportionately route to certain leader-follower groups, we will split the hot keys to several different groups, and aggregate them during read time. This could introduce latency for the video count of these hot videos. However, it is an acceptable tradeoff to ensure availability of the service.
For our cache, it is a write-back cache, which means that writes are written to the cache first, and asynchronously written to the database. This ensures that we have faster writes. However, if the cache crashes before data is pumped into the downstreams database, we risk losing data. This is a durability tradeoff that is acceptable in our use case.
For the database, we will use a no-SQL database, cassandra, that is highly scalable horizontally, and can handle huge write throughput. It is better for our use case than a relational database, because:
For the cassandra database, we store both the simple video count, as well as the detailed watch event.
For the video count:
table video_count {
video_id: UUID,
video_count: Integer
}
For the complete video watch event:
table video_watch_event {
video_id: UUID,
user_id: UUID,
watch_start_time: Timestamp,
watch_end_time: Timestamp,
video_metadata: String,
}
In our write path, when a write event happens, the video count service writes to the redis cluster, which then async propogates the data to the cassandra database. In cases of cache misses, the cassandra database will handle the reads with eventual consistency and low latency.
In the meantime, we also trigger a write event to be ingested on kafka, and consumed by a video event writer. The writer writes the complete watch event with metadata into a separate table in cassandra. This table is used by the video analytics service, to analyze watch trends, patterns etc.
On the read path, we go to video count service to fetch video count from redis and cassandra.
For content creators, they fetch the complete analysis of their video's watch patterns from our video analytics service.
One problem to discuss here is data integrity. If a video count write is failing, and we are triggering retries, we want to make sure that video counts are not double-counted. For the write requests, we will generate a idempotent key for each write request, and retries will have the same idempotent key. When server receives duplicate requests with the same key, it will reject the following requests.