Loading...
Lets outline the fundamental actions we can take from a user perspective.
Ok lets take a high level view at the system compute at peak and storage per year requirements . Im going to assume we have a semi decent app. ~10% of the US population, which is about 30 million MAU. Each user lets say has 100kb data. For businesses lets say that the number is 10% of that. thats 3 million businesses. Each business, not including photos, has 100kb and lets say has reviews that are 1kb. So lets say the businesses average 1mb of reviews. Note, lets exclude the location search portion of this that could be saved for later
Storage/Memory
3 TB - for users
300 GB for businesses
3 TB - for reviews + ratings
thought: this could fit in a general purpose AWS RDS instance no problem. caveat,
Compute
This is definitely read heavy. Lets say at peak minute we have 10% of the users reading. Thats 3 million requests in a minute or 50k per second. Depending on the compute of the request and what our storage layer is, we can estimate that if a r5.xlarge with 4 cores can process 1k each.
thought: with r5.xlarge instances that can handle 5k each (4 cores at 1k rps if we use async event looping or something) then we need a load balancer to front about 10 instances. The bonus here is that we define a bound and in reality we can scale it up and down depending on traffic (by time or traffic pattern)
caveat: the bottleneck might be in the storage query later we can go into later
Ok lets quickly outline the API we could see here.
Creating a user
POST api/v1/user
{
name, dob, etc
}
POST api/v1/entity
{
address
name,
hours
category
}
Return
{
"result": success
"id": <>
}
GET api/entity/
{
address
name,
hours
category
}
GET api/entity
-> getting the entity reviews
GET api/entity/
{
"metadata": {}
"data": [{
name:
rating:
text:
}]
-> searching by location (need cache if sorted by personalization)
GET api/entity?latitude=<>&longitude=<>&cateogry=<>&sort=new&cursor=<>&limit=100
{
"metadata": {}
"data": [{
name:
rating:
hours:
}]
The REST server data schema is pretty typical, and we can use something like a sql database to get the relational between the reviews and businesses. its not super realtime write heavy and more read heavy so thats seems like an appropriate solution.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Lets talk about the proximity search flow because that is a little nuanced part of the system design.
-> Writes
First of all we need to map the address to a geolocation. Might be ok with using a separate API or mechanical turk to grab the coordinates. This shouldnt be too costly because its not a high traffic op. Ok so with the coordinates, we store it in the DB.
-> Reads
As explained below we will have the quadtree in memory. lets say each business id is 64 bytes. if we have 10 million businesses thats 600 million bytes which is 600mb. so the memory will be like 1 GB. This can be instantiated in the beginning. so as the user enters the search, we can go through the quadtree and select the appropriate leaf, then sort and filter by the criteria
Business Creation
User Account Storage
User Account Authentication
Proximity Search
Quadtree vs Geohash - instantiating in memory on server startup is something that needs to be thought through.
We can replicate the DB since the read and writes are split, so we have redundancy and not have a bottleneck on the peak traffic
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?