There could be 10k to 100k DAU.
The peak traffic is 3k to 5k QPS.
The payment info would be 10k * 5KB per day, storage: 1.5 GB every year.
Movie listing require CDN to store videos and images.
10k listing * 200MB = 2TB storage
At high level we have the following components:
Rate limiting at API Gateway: To balance efficiency and spread load, I suggest to use the sliding window + bucket algorithm for the rate limiting
DB reliability and scalability: To serve the peak traffic, DB should be replicated. Different table has different consistency requirement. i.e. seat table has a strong consistency requirement, so the replication algorithm should be synchronous update. And set the quorum to be high enough to avoid conflicts. The movie table has weaker consistency requirement, so we can implement the eventual consistency.
Orderid grows really fast and multiple order service instance might write at the same time, so we can use the snowflake algorithm to design the order id. The order id is consisted by 3 parts: service-id, timestamp, incremented id.
Writing to the order table ensures consistency over availability. It has a strong consistency requirement, so checking the availability and writing to the table has longer latency to ensure there is no conflict.
The order service and movie service are stateless, so failures can be handled as bringing up new services. The ongoing payment would not be written to the DB until the payment is successful, so it can be reverted.
A/b testing for the recommendation algorithm
Load testing
CI/CD
Different development environment: prod, canary, dev