Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
Here are the APIs:
The APIs to create/edit/delete advertisements:
PUT v1/create_advertisement {
user_id: UUID,
title: String,
description: String,
category: String,
price: Double,
location: String
}
POST v1/edit_advertisement {
user_id: UUID,
post_id: UUID,
title: String,
description: String,
category: String,
price: Double,
location: String
}
DELETE v1/delete_advertisement {
user_id: UUID,
post_id: UUID
}
The APIs to view/filtering/searching listings:
GET v1/view_listings {
user_id: UUID,
categories_filters: List
sub_categories_filters: List
}
GET v1/search_listings {
user_id: UUID,
search_keyword: String,
search_filters: List
}
The API to upload images:
POST v1/upload_image {
user_id: UUID,
post_id: UUID,
image_metadata: String,
image_data: binary_data
}
APIs to register/login users:
POST /api/register
Content-Type: application/json
{
"username": "exampleUser",
"email": "[email protected]",
"password": "securePassword123"
}
POST /api/login
Content-Type: application/json
{
"username": "exampleUser",
"password": "securePassword123"
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
To support 1500 peak read RPS and 300 peak write RPS, we have a load balancer layer that distributes the traffic to different servers based on consistent hashing of user_id. We also have an API gateway that does authentication and rate limiting, which prevents a single user from sending too many requests to overload the servers.
Write path:
When users post, edit or delete an advertisement, the changes in data are propagated to 3 flows:
We write the data to a document database, since the schema for an advertisement changes often, as we often add/update additional metadata for an ad, we can use a document database like mongoDB. Data schema could look like:
{
user_id: UUID,
post_id: UUID,
title: String,
description: String,
category: String,
price: Double,
location: String
}
For the document database, as we need 5TB of storage, we can shard the database by hashing the post_id. We also create replicas for the shards, to increase availability. For the database, we will have a primary write replica, and the rest are follower read replicas. We will propagate changes from the primary write replica to the follower read replicas asynchronously. This could lead to some inconsistency where multiple users see different versions of the same advertisement. This eventual consistency model is an acceptable tradeoff for our use case as we get lower latency and better availability.
To mitigate the heavy load on the database, we build a caching layer (redis cluster) to return the repeated reads ultra fast. It will be a write through cache, such that we write to the cache and database synchronously as an atomic operation.. While this leads to higher write latency, we ensure durability in case of cache crashes.
When write events like create/edit/delete happens, we also trigger an event on Kafka queue. Every hour, the consumer of the queue, the index writer, batch writes the events to the ElasticSearch index. When we create the event for the advertisement, we also include categories and subcategories like jobs, housing, services, for sale to the event, so we eventually store them to search index, which allows users to search and filters. With queue and batch processing, we could create delay between when users create an advertisement and when that advertisement becomes searchable. However, since search doesn't always need the most updated information, this delay is acceptable. The tradeoff would be less ElasticSearch replicas needed to handle the huge write volume.
When users create advertisements, they also need to upload media information, such as images or even videos. We store the media uploaded in an object storage like S3, and with a CDN layer between storage and serving. This will ensure fast display of the media.
When users register, we store their username and appended + hashed password in a relational database. When they try to login, we match their password with what we store in database. All connections between user and the registration service should be sent over TLS.
Read path:
When users log in, we match their login credentials with our stored credentials. Once verified, the server sends signed JWT tokens, which the user will send in following requests.
For the advertisement feed, users requests are routed to feed service, which reads data from redis cluster and document database, as well as from CDN to display the advertisement. If users want to browse by categories and subcategories, we look at the advertisements returned, and group them accordingly in the server.
When users try to search for advertisement, the request is directed to ElasticSearchIndex, where users can apply filters in their queries.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
To support 1500 peak read RPS and 300 peak write RPS, we have a load balancer layer that distributes the traffic to different servers based on consistent hashing of user_id. We also have an API gateway that does authentication and rate limiting, which prevents a single user from sending too many requests to overload the servers.
Write path:
When users post, edit or delete an advertisement, the changes in data are propagated to 3 flows:
We write the data to a document database, since the schema for an advertisement changes often, as we often add/update additional metadata for an ad, we can use a document database like mongoDB. Data schema could look like:
{
user_id: UUID,
post_id: UUID,
title: String,
description: String,
category: String,
price: Double,
location: String
}
For the document database, as we need 5TB of storage, we can shard the database by hashing the post_id. We also create replicas for the shards, to increase availability. For the database, we will have a primary write replica, and the rest are follower read replicas. We will propagate changes from the primary write replica to the follower read replicas asynchronously. This could lead to some inconsistency where multiple users see different versions of the same advertisement. This eventual consistency model is an acceptable tradeoff for our use case as we get lower latency and better availability.
To mitigate the heavy load on the database, we build a caching layer (redis cluster) to return the repeated reads ultra fast. It will be a write through cache, such that we write to the cache and database synchronously as an atomic operation.. While this leads to higher write latency, we ensure durability in case of cache crashes.
When write events like create/edit/delete happens, we also trigger an event on Kafka queue. Every hour, the consumer of the queue, the index writer, batch writes the events to the ElasticSearch index. When we create the event for the advertisement, we also include categories and subcategories like jobs, housing, services, for sale to the event, so we eventually store them to search index, which allows users to search and filters. With queue and batch processing, we could create delay between when users create an advertisement and when that advertisement becomes searchable. However, since search doesn't always need the most updated information, this delay is acceptable. The tradeoff would be less ElasticSearch replicas needed to handle the huge write volume.
When users create advertisements, they also need to upload media information, such as images or even videos. We store the media uploaded in an object storage like S3, and with a CDN layer between storage and serving. This will ensure fast display of the media.
When users register, we store their username and appended + hashed password in a relational database. When they try to login, we match their password with what we store in database. All connections between user and the registration service should be sent over TLS.
Read path:
When users log in, we match their login credentials with our stored credentials. Once verified, the server sends signed JWT tokens, which the user will send in following requests.
For the advertisement feed, users requests are routed to feed service, which reads data from redis cluster and document database, as well as from CDN to display the advertisement. If users want to browse by categories and subcategories, we look at the advertisements returned, and group them accordingly in the server.
When users try to search for advertisement, the request is directed to ElasticSearchIndex, where users can apply filters in their queries.
Cache stampede protection:
For redis, when popular keys expire at the same time, there might be problems of cache stampede. To prevent this, we could either:
Duplicate listing:
When clients try to submit a request to create new listing, and it fails and they do retry, they could end up create duplicate listing. We create a composite idempotent key for each request, that key should be hashed result of the user_id, advertisement title, description, category etc.
When the feed service receives a request, it checks its idempotent key to see whether it already exists. If so, it will reject the request as duplicate.