Loading...
Future Improvements (Currently out of scope)
DAU = 10M
Concurrent active users could be 10-20% during peak time = 2M
Daily Postings = 1M/day = 1*10^6 / 10^5 = 10/sec
Daily Peak Postings = ~30 posts/sec
Active postings in a DB = 1M*30 = 30M posting (assuming 30 day expiry)
Reads = 1000/sec
Peak Reads = ~3000/sec
Storage:
Each posting will include title, description, price, category, pic URL link + metadata (~ 500bytes/post)
Storage in a day = 1M * 500 bytes = 0.5GB/day
Storage in a month = 15GB/month
Storage in a year = 180 GB/year = 900GB in 5 years
Images Storage:
Assuming 2 pics/listing at 1MB each
1M * 2 * 1MB = 2GB/day = 750GB/year [this will be a separate object store (S3]
Throughput = 30* 500 bytes = 15KB/sec [Peak]
Posting Ads:
Request:
Headers:
Authentication: Bearer
Idempotency Key: uuid,
Content: "application/json"
Accept: "application/json"
Body:
{
user_id: uuid, # This is based on the logged in user
title: string,
description: string,
category: ENUM,
price: double,
currency: ENUM
num_images: int
}
Response:
Headers:
Content: "application/json"
Accept: "application/json"
Body:
{
user_id: uuid,
title: string,
description: string,
category: ENUM,
price: double,
currency: ENUM
num_images: int,
images_upload_url: [url1, url2...]
created_at: timestamp
updated_at: timestamp
post_id: uuid
}
PUT v1/ads/{post_id}
Request and Response will be similar to POST (except new post_id will not be created)
[Only the original publisher of the ad can delete it]
DELETE v1/ads/{post_id}
Response: Status: 204 [Only the original publisher of the ad can delete it]
POST v1/ads/{post_id}/status:
{status: active/deactivate} # This will be used to activate / deactivate the ads [only the original publisher can do it]
GET v1/ads/{user_id}
Response:
Headers:
Content: "application/json"
Accept: "application/json"
If-Not-Match:
Body:
{
user_id: uuid,
posts: [
{
title: string,
description: string,
category: ENUM,
price: double,
currency: ENUM
num_images: int,
images_upload_url: [url1, url2...]
created_at: timestamp
updated_at: timestamp
post_id: uuid
},
{}, {}, {}
]
has_more: True,
next_cursor: uuid
}
GET v1/ads/search?q={cars, BMW}, min_price=1000, max_price=1000000, currency=USD, limit=10
Response:
Headers:
Content: "application/json"
Accept: "application/json"
If-Not-Match:
Body:
{
posts: [
{
user_id: uuid (This is the id of the person who posted the ad)
title: string,
description: string,
category: ENUM,
price: double,
currency: ENUM
num_images: int,
images_upload_url: [url1, url2...]
created_at: timestamp
updated_at: timestamp
post_id: uuid
},
{}, {}, {}
]
has_more: True,
next_cursor: uuid
}
POST /auth/login
Returns JWT Token
We have auth service that handles Login/JWT Token [Intentionally kept with less details as the focus is on Posting/Searching]
When an user wants to post an ad, the request will go through Load Balancer -> Gateway [Handle Rate Limiting] -> Post Service.
Post Service will store the metadata about the post in Postgres and emit an event to Kafka
Workers will subscribe to events in Kafka, write to Elastic Search (for search functionality), Notification Service, Monitoring Service and invalidate Redis (on put/delete actions)
Write Service ask S3 to give pre-signed URLs for users to upload images.
Users request to activate/deactivate the ads also follow the same path.
Read Path:
Search Service Reads primarily from Redis with fallback to Elastic Search (search results) and Postgres (metadata about the post)
Any outage related to Elastic Search/Redis doesn't block the Writes path (as it follows a separate path).
Similarly for writes, we have caching at CDN, Redis and then fallback to ES/Postgress DB, so our system is highly available and follows separation of concern principle
Postgres Table:
(
Post id: uuid # PK,
Title: string,
user_id: uuid # FK,
description: string,
price: double,
currency: ENUM
category: ENUM
created_at: timestamp,
last_updated_at: timestamp,
currently_active: bool,
location: string
)
Index at (user_id, post_id, category, created_at, location, price)
Listing Image table:
(
image_id: uuid #PK
url: string,
post_id: id #FK
)
Index at post_id
Redis:
products:{category}:{page_num} Values: [{}, {}, {}]
products:{category}:{min_price}:{max_price}:{page_num} Values: [{}, {}, {}]
Elastic Search:
We will insert the product, with Title, currency, category as keyword, description as text.
This will help us search any product quickly.
We will also use the edge-n-gram functionality of ES for searching top N products based on the search criteria.
POST commits: We follow an outbox pattern, where a single write updates the DB and an outbox table atomically (so either both succeed or both fails). A separate worker can read from the outbox table and add it to Kafka. This helps us in maintaining the writes atomicity, atleast once notification delivery and ensuring that all the ads are added to Elastic Search.
Another worker subscribes to the Kafka, which ads the ads to the Elastic Search for indexing and Searching. Since Kafka maintains a log, any outage in that service will not permanently impact the whole search mechanism. We can retry from the last known Position.
Any event that has crossed the retry limit will be put in a DLQ, which will be resolved after manual intervention.
If Elastic Search is down, we can still continue to serve the cached data from CDN and Redis.
If Postgres is down completely, then the write service will be impacted, however, the reads can still go through the cached data
To handle the backpressure in Kafka, we can add more workers.
To handle large read requirements, we can add more read replicas.