System requirements

Functional:

• Support searching of hotels based on different criteria (e.g. location, budget, stars, dates, # of people, etc.)
• Support viewing hotel details (rooms, prices, perks, etc.)
• Support hotel booking (payment, confirmation)
• Out of scope
  • Recommendations
  • User profile, preferences management
  • Payment network
  • Hotel listing management

Non-Functional:

• For hotel search and viewing
  • Highly available (99.99%),
  • Eventual consistency with minimal latency (<500ms)
• For booking and payment
  • Strong consistency
  • Highly available (99.99%) but latency can be relaxed (up to 1 min)
• Fault tolerant and scalable
• Secure, complaint (privacy, PCI DSS)

Capacity estimation

Assume

• 100M user accounts
• 1M hotel listings
• 10M DAU (search, view hotels)
• 100k bookings per day

Storage

• Each user profile is 1k -> 100M * 1k = 100GB
• Each hotel listing is 10k -> 1M * 10K = 10GB
• Reservation is 1KB: 100k * 1K = 100MB per day, or 3GB per month, or 36 GB per year

Requests

• Assume each user searches 10 times per day, and view 10 listings
• Seach and view qps = 10M * 10 / 86400 = 1k qps
• Reservation qps = 100k / 86400 = 1 qps

API design

// User management and hotel listing management APIs are CRUD APIs are omitted.

// Search API

GET /hotels?location=& checkinDate={}&checkoutDate={}&adult={}&child={}&&pageSize={}&pageToken={} -> Hotel[]

// View hotel details

GET /hotels/ -> Hotel

// View hotel room details

GET /hotels//rooms/ -> Room

// Book room

POST /reservations -> Reservation

{

id, // output

hotelId,

roomId,

checkinDate,

checkOutDate,

adultNum,

childNum,

}

// Pay for reservation

POST /payments -> Payment

{

id, // output

reservationId,

paymentDetails, // {creditCardNumber, etc}

}

Database design

User

• id
• name
• ...

Hotel

• id
• name
• address
• ...

Room

• id
• hotelId
• name
• numBeds
• bedType
• maxOccupancy
• size
• amenities
• numAvailable
• pricePerNight
• ...

Reservation

• id
• userId
• roomId
• checkinDate
• checkoutDate
• numAdults
• numChildren
• status // held | paid
• reservedAt

Payment

• id
• reservationId
• paymentAmount
• paymentDetails
• status // unprocessed | processing | paid | rejected
• lastUpdatedAt

The core data is relational and needs strong consistency guarantee. We choose a relational database like postgresql, with ES that facilitates searching, and caching (like redis) to make reads more efficient.

High-level design

ViewService

• Handles request that view hotel and room details. Uses redis as a read through cache.

SearchService

• Handles search requests from user. Queries ES.

UpdateQueue and IndexService

• Receives CDC from database regarding room availability changes, (infrequent) room and hotel info updates, and updates the redis and ES.

ReservationService

• Handles reservation requests

PaymentService, queue and processor

• Handles payments in a async manner.

Request flows

Hotel and room search

• Client sends query to SearchService via APIGateway
• SearchService constructs ES query for ElasticSearch, and returns the results

Hotel and room views

• Client sends request to ViewService via API Gateway
• ViewService queries the redis cache for information. If not found, it queries the DB, and populates redis. Then it returns the result

Reservation

• Client makes a reservation call when they start the reservation flow
• The reservation is held for 10 minutes for users to complete payment
• Reservation service first validates the request (e.g. verify there is available room, room fits the number of people), creates a reservation row, and updates room table with less availability
• User is directed to payment page, where they enter payment information
• User submits payment information. The request is received by PaymentService and queued in PaymentQueue
• PaymentProcessor picks up the payment request. It first creates a payment row, and updates the reservation to "PaymentProcessing" state.
• It then contacts the payment network, which is a async call. It then queues another message on the payment queue (or use delayed delivery) so that the payment is checked periodically.
• When PaymentProcessor verifies the payment is complete, it updates the Payment and Reservation rows to indicates complete. It then puts a message on notification queue, where the notification informs the client that reservation is complete

Detailed component design

ES indexing

• We capture DB changes of hotel and room details, and the availability change in DB in CDC, which pushes the change message onto update queue
• The index service picks up the changes, and updates the information in ElasticSearch.

Reservation

• Reservations are held for 10 minutes. The start time is recorded in DB's row for the reservation.
• We check the reservation after expiry. Given the number of reservations (1 per sec), we can simply query active reservation every minute for expired ones. They we try to update the reservation to expired state.
• Potential conflicts with payment
  • If Payment was complete before we expire the reservation, we can check state in the transaction when we expire reservation.
  • If we expired reservation, then Payment came through, we overwrite the expiration with reservation succeeded.

Payment

• Payment are processed asynchronously by Payment network. Therefore we need to send the request, then periodically check status.
• Every day payment networks send journaled transaction history in bulk. We need a subcomponent to inspect this history and reconcile with the state in the system.

Trade offs/Tech choices

We use a relational DB because the need for strong consistency, and that data is managable by a single instance of DB.

We use ES for its query capabilities, especially keyword searches and geo-indexing.

Failure scenarios/bottlenecks

SearchService, ViewService, ReservationService, PaymentService, PaymentProcessor and NotificationServices are stateless and can be scaled horizontally.

The database (on the order of 300GB) is managable by a single DB cluster. We enable high availability and replicas and hot standbys.

For the UpdateQueue and PaymentProcessor queue, the load also isn't high and we may not need sharding.

To handle peak load (e.g. 100X or normal traffic for reservations), our queuing system should smooth out the peak, but we can also implement backpressure on the client.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?