Loading...
End users and sellers. For end users, they may use browser or mobile app to visit our service.
End user side:
There's also functions on ticket sellers side:
Ticketmaster revenues:
We'll focus on architecture of end user side for this design.
Service should be highly available. Tickets and Payments should be strictly consistent. For tradeoff, you don't want users to have purchased tickets but some are lost or double sold, or user made payments but not receiving tickets, or user payment failed but still got tickets. Therefore, strict consistency on sold tickets and payments take precedence of service availability.
Say DAU visit is 5 million. 10% make a purchase. On avg, eash user visits 5 pages
Read QPS = 5 million * 5 / 86400 = 290
Write QPS = 5 million / 86400 = 60
Peak traffic typically is 2-5x. Choose 3x.
Peak R/W QPS = 870 / 180
View/Browse:
Purchase:
Event:
event_id: one unique id per event
event_unique_id: one unique id per event per location per datetime
datetime: date and time of this event_unique_id
location_id: location of this event_unique_id
ticket_tiers: pricing and number of available tickers by tier
Order:
order_id
user_id
event_unique_id
ticket_ids
total_payment
is_canceled
is_refunded
Ticket:
ticket_id
order_id
event_id
event_unique_id
user_id
price
created_at
last_updated_at
is_used
is_canceled
is_refunded
Location:
location_id
description
address
seating_map
User profile:
user_id
last_name
first_name
middle_name
address
phone
payment_method: this needs to be hashed/encrypted
unused_tickets: we can keep a copy of unused tickets here for fast retrieval when user visits their profile
See high level diagram
View or Purchase Flow:
As described in the Functional Requirement section. Say user enters in a specific event, date and time:
View Purchased Tickets
Use Ticket
Cancel order and/or ticket
We have a batch job running in background to periodically check DB updates, and send out text/emails for order confirmations, newly purchased tickets, and reminders.
Servers:
At peak hours, many users may be looking at the same event.
To reduce latency, we need cache most recent popular events, typically in a LRU or LFU setup. These include the event profiles and their available tickets.
To ensure availability and consider disaster recovery, we need replicas. Leader should propagate updates to followers.
Server should immediately deduct an available ticket from its cache/memory as soon as it receives a user purchase intent (i.e. go to payment page, but not yet submit payment), and make it held for a period of time, say 10 minutes. User payment action triggered after the expiration will result a failure, because the tickets have been released back to available pool for other users to purchase. Otherwise, other users will not be able to purchase the held tickets.
Thus for payment, server needs to check if ticket is still available before sending payment info to third party payment servicer for verification.
We choose consistency over latency or availability here, as it involves financial transactions. For strong/strict consistency, we want the payment info, order and tickets being updated in all replicas before we confirm with user their purchase is made successfully.
To handle peak hours, we also want to route users to different servers. This may be based on consistent hashing on their user ids or auth tokens, to ensure randomness of traffic distribution, while also provides consistent user experience and better caching as the same user's traffic is route to the same server.
Database:
As data are not very well normalized, and write speed is critical during peak hours, we lean to choose append-only DB or key-value store for perf.
A few options on DB partitions:
Mitigations: in any of the options, we could cache popular events (typically those that just recently go on sale, or event time is approaching) to reduce latency for read. Note that write (i.e. purchase) should still choose consistency over latency.
Regardless of which partition method we use, we should always keep replicas of data, as we certainly don't want to lose anything, especially when financial transactions are involved. 3 replicas of 99.9% SLA under strong consistency yield to an avg down time of less than 32 millisecs per year.
To further reduce possibility of data loss, for every DB write, we could first write to their write-ahead log.
We have chosen consistency over availability or latency on writes, because of financial transactions being involved. To balance a little to availability and latency, instead of having all replicas to confirm before we finally confirm and commit change, we could use consensus and version vectors. By choosing W + R > N (num of write replica + num of read replica > num of total nodes), we guarantee that at least one node has the up-to-date data. With version vectors, we'll be able to tell which node has the latest data. This will give lower latency on making ticket purchases, however, add system complexity.
We may consider using CDN for caching popular events in different geographic regions, and let servers to route user read requests to their closest CDN server first. This is typically useful when a popular event's ticket sale initially goes live on the platform.
The most latency comes from where user makes a purchase. However, as it's financial transaction, we think it's acceptable. In real world, an event like Amazon Prime Day has millions to billions of transactions, when people may experience service interruption intermittently in peak time or for popular items; or something like a massive selloff in a stock exchange, retail users may not be able to send out orders.
There is a very small chance of data loss (< 32 ms per year from prior calculation). The only concerned scenario in our design is when a payment is made on user's payment method, but all servers or databases go down, such that the user does not receive confirmation and tickets. In this case, we'll involve the payment service provider, investigating their transaction logs with ours, find the gap, and make fixes.
For limited time, we didn't cover seller platform details.
On user side, we may consider adding reviews and comments for events, letting user to decide when they want to receive reminder notifications, etc.
We should also prevent attackers from abusing our system, e.g., blocking/holding lots of event tickets by making lots of purchase intents but not actually making purchase, or sending lots of invalid payment info, etc.