Loading...
-User can post ads
-User can view posted ads
-User can edit ads
-user can delete ads
-user can search ads
-System is available website is always up and running, prioritize over consistency (we can have posts be eventually consistent)
-System can handle a lot of viewing ads, and will be read heavy
-System can handle a lot of posts being added
-Posting ads and loading ads has a low latency, let's say no more than 5 seconds
8 millions users per week are reading ads and loading their feeds this means per hour about 47000 and 793 per minute, we should assume some hours are hotter than other thus at most our system should be able to handle 1600 reads per minute. Some of these people will be searching as well so the search system should be equally as scalable.
100,000 users per week posting ads means about 595 per day, while this is much less than loading feeds we should still make sure the system can handle this.
8000 a week is about 47 deletes an hour which is quite reasonable.
POST /user -> make user account
POST /ad {user_id, ad_info} -> posts ad
GET /ad/:id -> get's ad
PUT /ad/:id -> edits ad
DELETE /ad/:id -> deletes ad
GET /ad {query=query} -> queries ads
Ads
primary_key
user_id
text
location
hiring_user_id
Users
primary_key
username
ads can be indexed by location as people will tend to search for jobs by locations, depending on the depth of the system we could have a field in ads for job type of qualifications for easier indexing but that is subjective.
As for database type due to the read heavy nature of the system an SQL database like Postgres would be a great fit
In a high level design we have a service for making an account, searching ads, posting ads, updating and delete ads respectively. Each service interacts with the user or ad database respectively and modifies or reads it appropriately.
The DELETE PUT and POST user requests are all straight forwards
For Searching ads we want to use a full text search for queries, we opt to use elasticsearch for this, due to the read heavy nature of the system we might want to have a queue in the search service but I think it might be unneeded since it is just reads.
For making new ads we update the database as expected but we also update the search system, due to the nature of writes we might want to have a queue for that as well but might be unneeded.
Also we would have an API gateway at the start to serve as a load balancer, rate limiter, route the APIs.
POST /user: the request flows to the POST User Service where it updates the database with a new user entry
POST /ad: user makes a request -> goes to POST ad service -> service updates the elasticsearch system along with the database with the new ad -> returns confirmation
GET /ad and GET /ad/:id: -> goes to ad service -> uses the elasticsearch to search the query (or straight to ad if it is the GET /ad/:id request) -> returns the info to the client
DELETE /ad: -> goes to delete service -> updates database with delete command for ad
PUT /ad -> goes to service -> update db and elasticsearch correctly -> returns update
Elasticsearch -> full text search system, we might be able to index based on location or job type depending on structure of the query. We also can use elasticsearch to filter but we can also use a client side filter.
Elasticsearch also scales very well.
In order to maintain scalability and efficiency our services should scale horizontally which they do since they are stateless
In order to improve performance we should cache the ads database since some ads will be more popular than others. We can use Redis for this. Also makes sense to have CDN for different locations since our database in indexed by that.
Elasticsearch will be eventually consistent but might be a bit behind sometimes, this is an acceptable tradeoff for availability. The CDN adds complexity and cost but it worth it for performance.
Updates to database might not be immediately reflected in the search.
Future improvements are communication with the job poster and more effective way of dealing with updates to posts.