Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
For capture, store and resize of the screenshots, these will happen in the local client application of the user, we won't need to call any API for any data from the server.
For uploading screenshots:
PUT v1/upload {
user_id: UUID,
image_metadata: String,
image_tags: String,
created_at: Timestamp,
image_content: Binary data
}
For downloading screenshots:
GET v1/download {
user_id: UUID,
image_id: UUID
}
For sharing screenshots:
POST v1/share {
user_id: UUID,
image_id: UUID,
users_list: List
is_public: Boolean
}
For assigning hashtags
POST v1/assign_hashtags {
user_id: UUID,
image_id: UUID,
hash_tags: String
}
For searching for screenshots
GET v1/search_screenshot {
user_id: UUID,
keywords: String
}
For registration:
PUT v1/register {
user_name: String,
user_email: String,
user_password_hashed: String
}
For login:
POST v1/login {
user_name: String,
user_email: String,
user_password_hashed: String
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
To handle peak read QPS of 1500 and peak write QPS of 300, we have a load balancer that distributed user traffic based on consistent hashing of the user_id. We also have an API gateway that does rate limiting and authentication. The rate limiting is per use basis, we limit to 50 requests per minute, using a token bucket algorithm.
For registration, we receive user's username, email, and their hashed password. We add a set prefix to user's password, hash it and then store the hash result in relational database, the db schema looks like:
table user {
userId: UUID,
userName: String,
userEmail: String,
userPasswordHashed: String,
}
During registration and login, we use HTTPS to ensure communication security. When user logs in, we check their entered userName and password to the record in the database. While registration and login traffic is usually smaller, we want to ensure high availability for this. So we create database replicas. In this case, all replicas will handle writes and each write needs to be acknowledged by all replicas, and read can be served by any replica. We consider 2 factors when choosing this design: latency/availability and consistency. In registration/login use case, we have a much higher read RPS than write, we also need strong consistency. If write needs to be acknowledged by all replicas, it will make write slower, and potentially make availability lower if any replica is not online. However, the reads will be much faster, and the results will be consistent regardless of replica. We won't need a caching layer for this flow. Since users usually just login once and use the app, there is not a lot of repeated requests, so not much value in caching. After use logs in, we return a JWT token to the user, to be used in following requests.
For capturing, resizing of the screenshot, as well as saving the screenshot to user's local device, these will only happen in user's local device. However, when users do these things, we trigger an client side event, that gets put on a kafka queue, gets consumed by analytics service, and gets written to analytics database. We can use cassandra for analytics database, which is eventually consistent but highly available/scalable. This way, we know what users do on their local device, and how many of them choose to upload the screenshot.
When users choose to upload their screenshots, we trigger 3 flows.
The first flow uploads the image to object storage. We upload the image to S3, and when users actually want to download the images, we will serve the download requests from a local CDN to reduce latency.
The second flow writes the image's metadata, including its tags to the image metadata relational database, with table schema of:
table image_metdata {
user_id: UUID,
image_id: UUID,
image_metadata: String,
image_tags: String,
is_public: Boolean,
allowlist_users: List
}
We have a caching layer, a redis cluster on top of the database table. The cache is a write back cache, which flushes changes to the database asynchronously. This create 2 problems, durability and consistency. If cache crashes before we flush the data to the database, we risk losing the data. Also, if other services query the database before we write the data to database, they might read stale data. However, these concerns are acceptable tradeoffs, since the item image metadata isn't critical data, and we value availability and latency more in this case.
Noticeably, the cache/database also stores the allowlist_users and whether the screenshot is public. When users upload an image, we default the allowlist_user to only the user themselves. If they choose to share to allowlist_users or to the public, we update the is_public and allowlist_users fields.
When users want to attach tags to a image, we also trigger the write path to update the redis cluster.
On the read path, when users want to view a particular screenshot, or when they want to view a feed of screenshots, we retrieve the screenshot metadata from the cache (if not there, then from the database), filter by the screenshots that this user can view, and retrieve the corresponding screenshot from the CDN, and show it to the user.
The third flow triggers an event that is put on a kafka queue, which gets consumed by an indexWriter. This is to support the search feature. Whenever users uploads screenshots, share screenshots or attach tags to screenshots, the screenshot feed service triggers the event. We write the complete screenshot metadata to the ElasticSearch index as well. Noticeably, since we are processing the events on a queue, updates users make may not become immediately available in the ElasticSearch cluster. This is an acceptable tradeoff, as we have eventual consistency but we can have lower search latency and higher search throughput. Search feature is a normally not a guarantee for the highest data freshness, so this tradeoff is acceptable.
For the write path, either uploading images or attaching tags, when requests fail or timed out, and users trigger retries, we risk creating duplicate data. To avoid this, we create an idempotent key when sending requests, and reuse the key for any retries or duplicate requests. So we won't risk creating duplicate image metadata or adding duplicate tags for an image.
For the screenshot feed service, if the database is processing too many requests, and we risk creating cascading failures (unlikely, given the load balancing, rate limiting and caching), we can implement circuit breaker that stops requests from overloading the database.
If the database or ElasticSearch becomes too hot, we can also create load shedding mechanism that lets it reject requests until CPU becomes normal.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
To handle peak read QPS of 1500 and peak write QPS of 300, we have a load balancer that distributed user traffic based on consistent hashing of the user_id. We also have an API gateway that does rate limiting and authentication. The rate limiting is per use basis, we limit to 50 requests per minute, using a token bucket algorithm.
For registration, we receive user's username, email, and their hashed password. We add a set prefix to user's password, hash it and then store the hash result in relational database, the db schema looks like:
table user {
userId: UUID,
userName: String,
userEmail: String,
userPasswordHashed: String,
}
During registration and login, we use HTTPS to ensure communication security. When user logs in, we check their entered userName and password to the record in the database. While registration and login traffic is usually smaller, we want to ensure high availability for this. So we create database replicas. In this case, all replicas will handle writes and each write needs to be acknowledged by all replicas, and read can be served by any replica. We consider 2 factors when choosing this design: latency/availability and consistency. In registration/login use case, we have a much higher read RPS than write, we also need strong consistency. If write needs to be acknowledged by all replicas, it will make write slower, and potentially make availability lower if any replica is not online. However, the reads will be much faster, and the results will be consistent regardless of replica. We won't need a caching layer for this flow. Since users usually just login once and use the app, there is not a lot of repeated requests, so not much value in caching. After use logs in, we return a JWT token to the user, to be used in following requests.
For capturing, resizing of the screenshot, as well as saving the screenshot to user's local device, these will only happen in user's local device. However, when users do these things, we trigger an client side event, that gets put on a kafka queue, gets consumed by analytics service, and gets written to analytics database. We can use cassandra for analytics database, which is eventually consistent but highly available/scalable. This way, we know what users do on their local device, and how many of them choose to upload the screenshot.
When users choose to upload their screenshots, we trigger 3 flows.
The first flow uploads the image to object storage. We upload the image to S3, and when users actually want to download the images, we will serve the download requests from a local CDN to reduce latency.
The second flow writes the image's metadata, including its tags to the image metadata relational database, with table schema of:
table image_metdata {
user_id: UUID,
image_id: UUID,
image_metadata: String,
image_tags: String,
is_public: Boolean,
}
We have a caching layer, a redis cluster on top of the database table. The cache is a write back cache, which flushes changes to the database asynchronously. This create 2 problems, durability and consistency. If cache crashes before we flush the data to the database, we risk losing the data. Also, if other services query the database before we write the data to database, they might read stale data. However, these concerns are acceptable tradeoffs, since the item image metadata isn't critical data, and we value availability and latency more in this case. In case of cache miss or cache entry expire, we will query the underlying database, and populate the cache. We will add a random jittering to cache entry TTL, to avoid the problem of thundering herd when popular keys expire.
We also have a separate user permission service, with a relational database and redis caching layer, sample schema is:
{
user_id: UUID,
image_id: UUID,
allowlisted: Boolean
}
When users upload an image, we default the allowlist_user to only the user themselves. If they choose to share to allowlist_users or to the public, we update the is_public and allowlist_users fields.
When users want to attach tags to a image, we also trigger the write path to update the redis cluster.
On the read path, when users want to view a particular screenshot, or when they want to view a feed of screenshots, we retrieve the screenshot metadata from the cache (if not there, then from the database), filter by the screenshots that this user can view, and retrieve the corresponding screenshot from the CDN, and show it to the user.
The third flow triggers an event that is put on a kafka queue, which gets consumed by an indexWriter. This is to support the search feature. Whenever users uploads screenshots, share screenshots or attach tags to screenshots, the screenshot feed service triggers the event. We write the complete screenshot metadata to the ElasticSearch index as well. Noticeably, since we are processing the events on a queue, updates users make may not become immediately available in the ElasticSearch cluster. This is an acceptable tradeoff, as we have eventual consistency but we can have lower search latency and higher search throughput. Search feature is a normally not a guarantee for the highest data freshness, so this tradeoff is acceptable.
For the write path, either uploading images or attaching tags, when requests fail or timed out, and users trigger retries, we risk creating duplicate data. To avoid this, we create an idempotent key when sending requests, and reuse the key for any retries or duplicate requests. So we won't risk creating duplicate image metadata or adding duplicate tags for an image.
For the screenshot feed service, if the database is processing too many requests, and we risk creating cascading failures (unlikely, given the load balancing, rate limiting and caching), we can implement circuit breaker that stops requests from overloading the database.
If the database or ElasticSearch becomes too hot, we can also create load shedding mechanism that lets it reject requests until CPU becomes normal.
Currently, the image size is limited to 1MB, but if we ever want to upload larger images, we will implement a block-based upload mechanism to enhance the reliability and efficiency of large file uploads. When a user initiates an upload, the file is divided into smaller blocks, typically 1 MB in size. Each block is uploaded independently, allowing for parallel processing and reducing overall upload time.
If a block fails to upload due to network issues or other errors, the system automatically retries the upload for that specific block rather than restarting the entire upload process. This is achieved by maintaining an idempotent key for each block, ensuring that duplicate uploads are avoided.
Additionally, we incorporate a hash-based comparison for each block to verify integrity upon upload completion. This mechanism ensures that the uploaded blocks match the original file, providing users with a seamless and reliable upload experience.