Loading...
Let's assume we are operating in a mega city with 1 million unique cars. So now out of that assume atleast 50% are using it monthly . So assuming a single person owns one car we have monthy active users at 500000 and out of that daily active users are 20% of that means 0.2 million . And out of this 100k daily users all of them searches for spots for an average 5 times in one day.
And they book atleast once per day so our calculations are as follows:
search-spot:5 per day per user so 0.5million per day means 350 rps but we consider the max capacity it must be atleast 5 times the avg per day capacity so total request per second lets consider 1000rps
book-spot:avg once per day so thats 140 rps
cancel-spot:0.1 avg per day since cancelling is a rare event about 14rps
Read to Write Ratio:
Since searching spot meaning ultimately reads hv high rps than booking spot we can assume a read to write ratio of 5:1 so caching may become useful here.
Database Storage:
To make the storage extremely simple we won't track the exact spots which have been reserved but instead track the total no of available spots available for parking. When a car comes we just pass them a QR code which just contains the data of the amount paid and the time till when the spot can be occupied. And in the backend we will just increment the counter of total cars parked in that spot . And the same thing happens when the car leaves the parking spot i.e. we decrement the counter . And in case after that time is over and we still can see the car parked we will charge them for the over extended time. We will discuss more on this later.The amount of storage requirements will be very less since we will just be storing the total no of cars parked .Lets look at the storage requirements:
6 Tables:Users ,Parking Spots ,Payments,Cameras,Car_info,Gates
500kb each table per row so 3mb per user if he uses one times.Now ,since we assumed each day 0.2 million active users mean 0.2-0.3 million payments if they use it on an average once .
Hence total storage will be 3TB for all the user metadata and for payments table on each day it will inc by about 0.15TB.
Means assuming we design this for 10 years use we would need total data of around 600TB.One more thing we will be using sql specifically postgresSql since this architecture requires use of relational structures and joins between tables.
The api design will be very extremely simple and consists of the main crud endpoints .The endpoints in the v1 of the system are
-/v1/book:Person books a timing slot for his car and clicks on booking.He makes the payment and the slot is booked and saved in the system.In the backend we check if the parking plot has more spaces available for the same given time period and if not other users can't book the same slot.
-/v1/payment:For the v1 this would be moslty a third party payment service provider like razorpay as i don't want to make it more complex.
-/show-booked:User can see all the spots he has booked for,the gate details for the parking plot and the timing for each one of them.After the time has passed we remove it from the active slots so that user can no longer see the slots.
-/cancel:User can cancel his slot but only if his booked slot timing has not yet started.He cant cancel after he has alr booked or after his slot ends.
-/register:Person will also hv the optional features to register or share with us his car details so that we can better suggest and assign him a more better space to park his vehicle.User also must tell us the type of his vehicle and the number of his vehicle and his user details while logging in for the first time so that we can properly map car to that specific person and number.
-/departure:When a person slot is finished or he goes for an early departure, the system scans and remove that from active slots.If the specific person time is over but he still hasnt departed he is given a max of 15 mins from his scheduled departure time to empty his spot.
For all of this endpoints after a user books a slot and his slot timings began , a countdown timer starts in the bg where if the person doesnt show up for atleast 50% of his booked time,after that his slot is auto cancelled and is assigned to someone else .
Users will be able to use our service using website and app.We will be using a level 7 load balancing which operates at the application layer and will dynamically routes requests to appropirate servies based on the api url.
We will also be using rate limiter in front of servers so as to not overwhelm the servers and to be protected against ddos attacks.We will be using token bucket algo for rate limiting as its the most simplest of them all and its storage requirements is very less.
We will hv services which cater to different user requirements-
-Payment services:Tracks payment flow every time after booking a slot a person initiates a payment row is created in the db with status "Pending" along wiht
-Notification service:Whenever a user succesfully books a slot after a payment a job is pushed to the message queue and a message is sent to the person email or phone about the payment.Similary whenever a person booking slot is near its end a notification goes to them 30 mins before that there slot is ending soon.
-User service:User service comprises of user accounts and authentication ,users land on this service while authentication at the start and they can edit their name or logout .
-Analytics service:This service is responsible for tracking analytics like how many times does an average user uses it in one week , which vehicles are the most used ones,which parking area has the highest demand .
-Booking service:This service responsible for bookings whenever a user sucessfully books a slot the slot is locked for that amount of time and is automatically released whenever the bookings slot completes its timings.If in case the payment fails for any reason we lock the specific slot for around 5 mins for the user to try another slot and if bookings isnt sucesfull we realease the slot.
-Message Queue:A kafka message queue can be used for storing and streaming different events to different microservices
The main database which we will be using is postgres sql for all the services and clickhouse or any other olap database for analtyics service.
The user service db will hv users table with user id ,name ,email which will be unique,phone and some metadata like created_at and updated_at.
Payment db will store the payments table ,it will hv the user id ,payment_id,amount paid,status -cancelled,pending or confirmed.
Booking db:This will only be written after payment is confirmed from the payment service it contains the slots ,after confirmations the slots are locked till the end.
Notification Db:This contains all the logs and messages sent in notifications each message will have a user_id+message+booking_id so that same message cant be processed twice ensuring idempotency.
To ensure higher availalibity of the db we will have replication of db .We will be followwing quorom read write strategy where alteast one follower node is in sync with the leader node architecture .All the writes will go to the leader node whereas reads go to the followers.This ensures in case of failures of leader node the follower node can immediately take over the writes.
We will hv eventual consistency in the user service .As payments and bookings cant be wrong or hv stale writes we will be using consistency+partition there.This ensures bookings and payments always never return stale queries.
For caching we will make use of redis since our read to write ratio is pretty high.Following the 80 20 rule where 80% of request accounts for 20% of data keys we will be caching users info who use the application multiples times a day .We will also using combination of analytics and demand during particular day cache slots who are very busy like during employees arriving or leaving hours from office since that's the time they will be using the system the most.This ensures users faces the least amount of latency.
Key features to discuss here is how do we not double charge the user and how do we prevent race conditions while booking .So every time a slot payment is tried we block that specific slot using db locks .This is pesimistic locking since race condiitons can occur during times of higher usage .This ensures that when two person try to book at the same time no race conditions can occur.Since we will have quite a few no of read replicas we will be using centralised database locks.Howerver pessimist locking can slow response times significantly so will be using a combination of pessmist and optimist locking mechanisms .So during not so peak hours we will be using optimist locking algorithm.We will can also use redis lock with set ttl time to live which will be 5mins .If the user succesfully completes the payment we mention that slot as booked and if not we re release that slot .
There might be slots which are overcrowded or slots which many people try to book at the same time for specific times of day like during some sporting event or concert or just mall or office parking.To prevent overbooking during these times we can try reducing the wait times during booking to lesser than 5 mins to like 3mins during extreme rush.Also we would implement a waiting queue system.For eg if user A arrives and is currenlty undergoing the booking process and at the same time user B and then user C arrives.We will maintain a queue where if user A booking fails then it auto goes to user B and then to user C and this continues.
To prevent overcharging the user twice we use the payment_id filled which is provided by the payment service provider.We can simply check if the given payment id has been processed earlier to prevent from overcharging the user.
Now coming to the user messages part each message will hv a unique id which will be combination of
user_id+message+booking_id , this ensures that no same message once sent can be queued again.This is enforced on db level so even if our workers malfunction this ensures that no message is queued twice .Messages are retried 3 times with exponential backoff before being marked as failure or are sent to dead letter queue so that the team can inspect them later.
For the analytics it will be running an asyncronous jobs to keep track of how much each user has made bookings ,the most busy parking spots ,most busy parking spots timings and what are the most used types of vehicles which are being parked.
For observaliblity we will be using promesthus and grafana for monitoring latency and metrics of the overall systems like p99,p95 and p90 latency.