Requirements

Functional Requirements:

• User can browse restaurants and their menus
• User can add items to food card
• User can securely complete payment to place their food delivery order
• User can track their delivery: Preparing order, Waiting for driver assigment, Driver picking up, Driver on the way, delivered
• User can rate the restaurant and driver after delivery

Non-Functional Requirements:

• Highly Scalable - Support ~10M orders daily
• Highly Available - 99.9% uptime, should not fail when user browsing menus
• Strong Consistency for secure payments
• High read throughput for restaurants and menus- ~500M reads daily and high write throughput for driver location updates
• Low latency for matching drivers <1s (does not include the time it takes for dispatching delivery order to top driver and waiting for acceptance)
• Eventual consistency for driver location updates

Capacity Estimation

• 100M DAU
• 10% places order - 10M daily orders -> 1K orders/sec -> peak = 5x -> 5K orders/sec
• 1M drivers active
• Location update every 5sec -> 1M/5 -> 200K updates/sec

API Design

Browsing restaurants and menus

GET /restaurants?filters=xyz

Return top restuaratns near user address

GET /restuarant/{id}

Fetch restaurant details and its menu

Delivery APIs

PUT /users/{id}/cart

req: {

item

}

add item to user cart

POST /delivery

req:

{

restaurantId,

userId,

items[],

paymentMethod

}

GET /delivery/{id}

Fetches delivery status and details

PUT /delivery/{id}

updates delivery status and details

Driver APIs

POST /drivers/{id}/status

req: {status} update druver status -> available, reservered, on delivery, offline

POST /drivers/{id}/location

req: {

lat, long, timestamp}

updates driver location

POST /delivery/{id}/accept

Accept delivery order

High-Level Design

API Gateway

Acts as the entry point for all client requests.

Responsibilities:

• Authentication (if required)
• Rate limiting - Applied using token bucket per IP to prevent fraudulent or bot views
• Request routing

Load Balancer

Distributes traffic across multiple instances of all services to improve scalability and availability.

User Service

• Rider CRUD service,
• Manages rider profile, including delivery history etc

Driver service

• Driver crud service
• Manages driver profile, including ratings, delivery history etc

Restaurant Service

• Consumes Get restaurants and get resturant by Id request and retrurns details
• For having high read throughput, the service uses Redis
• Restaurant details cached in redis
• For Get Restuarants:
  • request goes to geo index service to recieve geo spatial index of user location
  • Use that to find restaurants from redis, if cahce miss then fallback to DB
• For get restuarant by id:
  • Check redis directly by Id, if miss then fallback to DB

Delivery Service

• Delivery state machine
• Receives initial delivery request
• The follwing is atomic transaction that happens when delivery request comes in:
  • Create a delivery record with status PaymentPending
  • Send request to payment service
  • Update status of delivery to Requested if payument service returns success or Rejected/Failed when payment service returns failure
  • Proceed firther with delivery only if payment succeed
• Manges all end to end states from requesed to delivered
• Sends all stattus updates to kafka so downstream services can consume it and process it further
• Also sends a request to web socket gateway to create a websocket conneciton to user app for all status tracking for user

Location Service

• Handles Driver's spatial geo indexing
• Receives drivers updates every 5 sec
• Validates the request and emits LocationUpdated event to kafka

Payment Service

• Recievs payment request from delivery serivce
• Calculates total price, creates a payment record and sends request to the payment provider
• Request to payment provider is send with idemptotenKey -> userId, deliveryid with timestamp tos end to payment provider
• Updates payment record according to repsonse from payment provider
• All of these is an atomic transaction that means, if one failed entired transaction will fail
• In case if db update fails after paymetn is susccefull we will issue an compenstating refund transaciton to Payment provider

Dispatch Service

• Core Food delivery service brain
• Consumes DelviueryRequested event from Kafka
• Determines user GeoIndex cell
• Fetches all the drivers at that cell
• Ranks and filters those drivers, resulting in top k drivers
• offers delivery to top drivers

Geo Index Service

• Consumes the location updated event
• determines the geo cell from driver location and stores in redis
  • eg : d3sdsf3 -> driver1, driver2, driver3

Websocket gateway

• Manages all active websocket connections
• Sends driver's real-time location updates to the deliver that driver is on.

Kafka

• Event backbone
• Consumes all delviery related event
• Gurantes atleast once delivery
• Uses delivery as the partition key fro delviery events
• And for location update events uses driverid as parition key

PostreSQl

• Primary persistent DB for consistent data -> riders, drivers, payments, trips

Redis

• Primary store for Geo indexing
• Also stores restaurants cache
  • Stores top restauants by geo cell
    • eg: 343dsadd -> rest123, rest345 etc
  • Also stores restaruant details
    • eg: rest123 -> restauratne detail object

Database Design

We use hybrid apporach for Database, using SQL -> PostreSQL for storing strong consisten data where transactrions matter like Users, Drivers, Delivery, Restaurants, MenuItem, Payments etc and NOSqL -> redis for stroing geo index locations of driver

Restaurants

Id -> partition key

Name

Description

Type

Ratings

Location

MenuItem

Id

RestuarantId -> partition key

Name

Description

Price

Type -> appetizer, main course, dessert, drink etc

Image

User

Id -> partition key

Name

Email

Address

Driver

Id -> partition key

Rating

Status -> Avialable, Reserved, On Delivery, Offline

Delivery

Id -> partition key

userId

RestaurantId

DriverId

Status

TotalPrice

PaymentMethod

createdAt

DeliveryItems

Id

DeliveryId -> partition key

MenuItemId

quantity

UnitPrice

Delivery -> Delivery Items = 1 to many relationship

UserPaymentMethods

id

userId

method_id -> id from payment provider

providerId

Payments

id

tripid

amount

status -> PENDING, SUCCESS, FAILED

provider_id

Geo index -> stored in redis

Structure:

Driver -> cell mapping

eg:

34ascas -> driver1, driver2, driver3

Driver lastes location(comes from api)

driver1 -> lat, long, timesatmp

Detailed Component Design

We will now deep dive into these topics

Dispatching

Most imp and hardest problem.

Delivery Requested and Restuarant accepted order

• User orders a delivery -> Delivery service: stasu = Requested
• Restarant accepts the oder -> Delivery service: staztsus = Accepteed
• Emits PreparingOrder Event to kafka

Dispatch Service:

• Consumes the event 'PreparingOrder'
• Gets drivers in restaurant close proximity from deis
• Ranks the driver

• Determnises top 10 driver

Delivery offer workjflow:

• Intead of offer one driver, wait 5 sec offer next which is slow
• We use Batch dispatch i.e offer to top 3 drivers simultaneously
• First acceptance wins

Acceptance Race Probmlem:

• Driver A and B accetps at the same time, boht drivers could get assigned to same delivery
• To solve this we use row level locking

• This way driver A reqeust locks the delivery record and drier B has to wait for the lcck to be relased
• By the time lock is realsead the delviery staus is updated to Driver Assinged and thus Druiver B requests is rejected

Geo Spatial Indexing

• using direct sql to fetch nearyby users is a bad desing
• Instead we use GeoHash i.e. convert lat and long into a geo cell, eg: t5e33
• Then in redis

• When theres no driver aviable in certain geo cell, we expand serach by seraching in neighbour cells
• but this also introduces HotSpot problem.

HotSpot problme:

• Busy area like Toronot has 5000 drivers in same cell
• The redis shard becomes hot
• To solve this we use Dynamic GeoHash Precision
• Busy ares : smaller cells, non biusy area: largert cells

Deep Dive #6 Failure Handling

-> Driver Disconnect

Heartbeat missing:

Mark offline.

Remove from matching pool.

-> Dispatch Service Crash

Dispatch state stored in DB.

Kafka retains event.

New instance resumes.

Redis Crash

Redis cluster.

Automatic failover.

If full Redis loss:

Rebuild index.

Kafka Broker Crash

Replication Factor:

Leader election.

• No event loss.