Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
Assumptions: 100M DAU, 25M of them interested in a given sale event. Normal day: 10 browses + 1 order per DAU.
Baseline (non-sale) traffic
Sale-open burst (the number that actually drives the design)
Concurrent connections: 10M concurrent. If clients poll every 2 s instead of holding a websocket, that's 10M ÷ 2 = 5M req/s of pure polling — which is why push/websocket or long-poll beats polling here.
Storage
Bandwidth
Cache sizing (useful sanity check): all 5M SKUs × 5 KB = 25 GB — the entire catalog fits in a Redis cluster, so even the cold 99% of items can be served from memory.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
To register and login:
POST v1/register
POST v1/login
To browse available items:
GET v1/browse_feed
To browse a detailed item:
GET v1/view_item
To place an item order:
POST v1/place_item_order
This endpoint is rate limited, and comes with a idempotency key.
To initiate a payment:
POST v1/pay
To view order history:
GET v1/view_order_history
CRUD on item cart:
To view cart:
GET v1/item_cart
To update cart:
POST v1/update_cart
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
In this design, there are write path (catalog item ingestion) and read path (item browsing, flash sale participation)
For the write path, when the item sources update item catalog, we publish events on the Kafka topic. The item data writer consumes the events, and writes the item information to the item catalog database. A separate database writer also queries the inventory service, and update the item inventory counts. We use Kafka for:
For the read path, all requests first go through 3 components:
For login/register, the request is processed by the identity service, which reads from/writes to the identity database.
For feed browsing, the request is processed by feed service. Feed service retrieves latest items and relevant deals from the item catalog database. It stores some popular items/deals in feed cache, to alleviate traffic on database.
Once user decides to join the flash sale after browsing, the request is handled by the flash sale admission controller, with below steps:
How do we avoid double purchase of the last item?
When we update the inventory database, we use atomic conditional transaction, like:
UPDATE available_inventory_count SET available_inventory_count - 1 WHEN available_inventory_count - reserved_inventory_count > 0
UPDATE reserved_inventory_count SET reserved_inventory_count - 1 WHEN reserved_inventory_count > 0
It is implementing optimistic concurrency control, as we expect conflicts to be rare, since most requests have been rejected by the inventory redis.
What happens if one of the service is down, (for example, payment service), when we have already updated the other databases?
There are 2 ways to solve this problem, either through SAGA or through 2-phase commit.
In our case, 2 phase commit is a poor choice, since it requires a coordinator that coordinates across different databases, holding locks concurrently, and it works poorly in our high volume.
We can use SAGA, which initiates compensating transactions in case of failures of some of transactions.
For example:
T1: Create order SUCCESS
|
v
T2: Reserve inventory SUCCESS
|
v
T3: Charge customer FAILED
|
v
C2: Release inventory SUCCESS
|
v
C1: Cancel order SUCCESS
If T3 fails, we will initiate C2 and C1 as compensating transactions.
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
We need to store six types of data:
We will use a combination of distributed SQL, Cassandra, and Redis, based on their consistency requirements and access patterns.
Use a relational database such as PostgreSQL or distributed SQL such as CockroachDB.
users {
user_id: UUID (PK)
username: String
email: String (UNIQUE)
password_hash: String
role: String
metadata: JSONB
}
Access patterns:
user_id or email.Indexes:
user_idemailWhy SQL?
User identities require uniqueness constraints, transactional updates, and durable storage. These are naturally supported by relational databases.
The item catalog is read-heavy during flash sales, but also receives bulk updates from external item sources.
Cassandra is suitable because it provides horizontal scalability, high write throughput, and denormalized query-specific tables.
We will maintain two tables.
items_by_id {
item_id: UUID (partition key)
item_name: String
item_description: String
regular_price_cents: BigInt
metadata: Text
}
Supports:
GET /items/{item_id}
deals_by_sale {
sale_id: UUID (partition key)
bucket_id: Integer (partition key)
deal_start_time: Timestamp (clustering key)
item_id: UUID (clustering key)
deal_price_cents: BigInt
deal_end_time: Timestamp
deal_metadata: Text
}
Supports retrieving deals for a specific flash sale, ordered by start time.
The bucket_id distributes large sales across multiple partitions and prevents unbounded partitions. The Feed Service queries the relevant buckets and merges the results.
If the feed needs newest deals first, configure descending clustering order.
Tradeoffs:
The Feed Service will also use Redis or another caching layer to serve popular deals without repeatedly querying Cassandra.
This is the most challenging part of the database design.
Inventory must never be oversold, even if thousands of users simultaneously attempt to purchase the same SKU.
item_inventory {
sale_id: UUID
item_id: UUID
total_stock: Integer
available_stock: Integer
reserved_stock: Integer
sold_stock: Integer
PK (sale_id, item_id)
}
We also maintain reservation records:
inventory_reservations {
reservation_id: UUID (PK)
sale_id: UUID
item_id: UUID
order_id: UUID (UNIQUE)
quantity: Integer
status: ENUM
expires_at: Timestamp
}
Reservation statuses:
RESERVED
CONFIRMED
RELEASED
EXPIRED
A reservation reduces available inventory and increases reserved inventory. Payment confirmation converts reserved inventory to sold inventory. Failed or expired reservations release stock.
These mutations must be atomic and idempotent.
A single database row per SKU can become a bottleneck when tens of thousands of purchases compete for the same inventory.
We can use Redis as a fast admission and stock-token gate to reject requests cheaply before they reach SQL.
For example:
sale:123:item:456:tokens = 1000
An atomic Redis script can decrement the token count only if a token is available.
However, Redis token allocation is not the final inventory transaction.
For strict consistency, we must ensure that Redis failures cannot restore already-consumed tokens and allow additional purchases.
One safe approach is to allocate bounded inventory token batches through durable SQL transactions, then distribute those tokens to workers. Every successful reservation still creates an authoritative, durable record. Tokens must be uniquely owned, and lost or uncertain token allocations must be reconciled before reuse.
Alternatively, use atomic SQL reservations directly for a simpler initial implementation, and introduce inventory token allocation only if measurements show hot-row contention.
Tradeoff: Strong inventory consistency is more important than maximizing raw throughput. We should never acknowledge a successful reservation solely because an asynchronously replicated Redis counter was decremented.
orders {
order_id: UUID (PK)
user_id: UUID
sale_id: UUID
item_id: UUID
quantity: Integer
unit_price_cents: BigInt
total_amount_cents: BigInt
currency: String
status: ENUM
reservation_id: UUID
idempotency_key: UUID
created_at: Timestamp
updated_at: Timestamp
}
Order statuses:
REQUESTED
RESERVED
PENDING_PAYMENT
CONFIRMED
FAILED
CANCELLED
Access patterns:
order_id.Indexes:
order_id(user_id, created_at DESC)(user_id, idempotency_key)Sharding:
Shard orders by user_id to distribute storage and query traffic, and keep a user's order history together.
For order lookups by order_id, use a globally resolvable order ID or a mapping/index that identifies the owning shard.
All retries for a given order must be routed to the same logical record.
payments {
payment_id: UUID (PK)
order_id: UUID
user_id: UUID
provider: String
provider_payment_id: String
amount_cents: BigInt
currency: String
status: ENUM
idempotency_key: UUID
created_at: Timestamp
updated_at: Timestamp
}
Payment statuses:
PENDING
SUCCEEDED
FAILED
REFUNDED
Access patterns:
order_id.Indexes:
payment_id(order_id, created_at)(user_id, created_at DESC)Sharding:
Shard payments by user_id, matching the primary order-history access pattern.
Payments can still be retrieved by order_id using a secondary lookup index or order-to-shard mapping.
The Payment Service uses provider-side idempotency keys so retries do not cause duplicate charges.
Store payment tokens and provider references, not raw credit card details.
Shopping carts have different consistency requirements depending on whether adding an item reserves inventory.
For this system, adding an item to a cart does not reserve inventory.
A cart is simply a temporary collection of items the user intends to purchase.
cart:{user_id} -> Hash {
item_id: quantity
}
Redis is suitable because carts require fast reads and writes and can expire after inactivity.
Operations:
HSET: Add or update an item.HGETALL: Retrieve a user's cart.HDEL: Remove an item.EXPIRE: Set inactivity expiration.Inventory is reserved only when the user actually submits a purchase request.
If the product instead guarantees stock when an item enters a cart, cart operations must create durable inventory reservations with expiration and release handling. Redis cart expiration alone is insufficient.
Admission state is temporary and accessed frequently during traffic spikes.
Redis is suitable because it supports low-latency reads, atomic operations, and native TTL expiration.
Example keys:
sale:{sale_id}:user:{user_id}:active_attempt
-> request_id, TTL
sale:{sale_id}:admitted_count
-> Integer
sale:{sale_id}:waiting_room
-> Sorted Set
Access patterns:
Redis operations such as SET NX EX and Lua scripts provide atomic admission checks.
Admission state is not authoritative inventory or payment data. Redis failures must not create duplicate purchases or overselling.
For distributed SQL, we will use leader-based replication with a consensus protocol such as Raft, assuming the selected database supports this architecture.
A newly elected leader retains previously committed writes, assuming the consensus quorum and durability guarantees hold.
For Cassandra, we use replication across nodes and availability zones. Catalog reads may use consistency levels appropriate for eventually consistent browsing.
For Redis, we use replication, failover, and partitioning for availability and scalability. However, Redis replication alone does not guarantee that every recently acknowledged write survives failover. Critical admission state therefore requires recovery and reconciliation logic.
| DataStoragePrimary reason | ||
| Users | Distributed SQL | Transactions, uniqueness, durability |
| Item catalog | Cassandra | Scalable writes and query-specific reads |
| Inventory | Distributed SQL + optional Redis token gate | Strong consistency and hot-SKU protection |
| Inventory reservations | Distributed SQL | Durable, atomic stock allocation |
| Orders | Distributed SQL | Transactional state and idempotency |
| Payments | Distributed SQL | Durable financial records and deduplication |
| Shopping carts | Redis | Low-latency temporary storage |
| Admission state | Redis | Atomic limits, fairness state, TTL |
| Feed cache | Redis / CDN | Fast, scalable browsing |
The central architectural principle is to separate high-volume, temporary coordination state from durable business state.
Redis handles cheap admission decisions and temporary state, Cassandra supports scalable item browsing, while distributed SQL remains authoritative for inventory reservations, orders, and payments.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
We need to store six types of data:
We will use a combination of distributed SQL, Cassandra, and Redis, based on their consistency requirements and access patterns.
Use a relational database such as PostgreSQL or distributed SQL such as CockroachDB.
users {
user_id: UUID (PK)
username: String
email: String (UNIQUE)
password_hash: String
role: String
metadata: JSONB
}
Access patterns:
user_id or email.Indexes:
user_idemailWhy SQL?
User identities require uniqueness constraints, transactional updates, and durable storage. These are naturally supported by relational databases.
The item catalog is read-heavy during flash sales, but also receives bulk updates from external item sources.
Cassandra is suitable because it provides horizontal scalability, high write throughput, and denormalized query-specific tables.
We will maintain two tables.
items_by_id {
item_id: UUID (partition key)
item_name: String
item_description: String
regular_price_cents: BigInt
metadata: Text
}
Supports:
GET /items/{item_id}
deals_by_sale {
sale_id: UUID (partition key)
bucket_id: Integer (partition key)
deal_start_time: Timestamp (clustering key)
item_id: UUID (clustering key)
deal_price_cents: BigInt
deal_end_time: Timestamp
deal_metadata: Text
}
Supports retrieving deals for a specific flash sale, ordered by start time.
The bucket_id distributes large sales across multiple partitions and prevents unbounded partitions. The Feed Service queries the relevant buckets and merges the results.
If the feed needs newest deals first, configure descending clustering order.
Tradeoffs:
The Feed Service will also use Redis or another caching layer to serve popular deals without repeatedly querying Cassandra.
This is the most challenging part of the database design.
Inventory must never be oversold, even if thousands of users simultaneously attempt to purchase the same SKU.
item_inventory {
sale_id: UUID
item_id: UUID
total_stock: Integer
available_stock: Integer
reserved_stock: Integer
sold_stock: Integer
PK (sale_id, item_id)
}
We also maintain reservation records:
inventory_reservations {
reservation_id: UUID (PK)
sale_id: UUID
item_id: UUID
order_id: UUID (UNIQUE)
quantity: Integer
status: ENUM
expires_at: Timestamp
}
Reservation statuses:
RESERVED
CONFIRMED
RELEASED
EXPIRED
A reservation reduces available inventory and increases reserved inventory. Payment confirmation converts reserved inventory to sold inventory. Failed or expired reservations release stock.
These mutations must be atomic and idempotent.
A single database row per SKU can become a bottleneck when tens of thousands of purchases compete for the same inventory.
We can use Redis as a fast admission and stock-token gate to reject requests cheaply before they reach SQL.
For example:
sale:123:item:456:tokens = 1000
An atomic Redis script can decrement the token count only if a token is available.
However, Redis token allocation is not the final inventory transaction.
For strict consistency, we must ensure that Redis failures cannot restore already-consumed tokens and allow additional purchases.
One safe approach is to allocate bounded inventory token batches through durable SQL transactions, then distribute those tokens to workers. Every successful reservation still creates an authoritative, durable record. Tokens must be uniquely owned, and lost or uncertain token allocations must be reconciled before reuse.
Alternatively, use atomic SQL reservations directly for a simpler initial implementation, and introduce inventory token allocation only if measurements show hot-row contention.
Tradeoff: Strong inventory consistency is more important than maximizing raw throughput. We should never acknowledge a successful reservation solely because an asynchronously replicated Redis counter was decremented.
orders {
order_id: UUID (PK)
user_id: UUID
sale_id: UUID
item_id: UUID
quantity: Integer
unit_price_cents: BigInt
total_amount_cents: BigInt
currency: String
status: ENUM
reservation_id: UUID
idempotency_key: UUID
created_at: Timestamp
updated_at: Timestamp
}
Order statuses:
REQUESTED
RESERVED
PENDING_PAYMENT
CONFIRMED
FAILED
CANCELLED
Access patterns:
order_id.Indexes:
order_id(user_id, created_at DESC)(user_id, idempotency_key)Sharding:
Shard orders by user_id to distribute storage and query traffic, and keep a user's order history together.
For order lookups by order_id, use a globally resolvable order ID or a mapping/index that identifies the owning shard.
All retries for a given order must be routed to the same logical record.
payments {
payment_id: UUID (PK)
order_id: UUID
user_id: UUID
provider: String
provider_payment_id: String
amount_cents: BigInt
currency: String
status: ENUM
idempotency_key: UUID
created_at: Timestamp
updated_at: Timestamp
}
Payment statuses:
PENDING
SUCCEEDED
FAILED
REFUNDED
Access patterns:
order_id.Indexes:
payment_id(order_id, created_at)(user_id, created_at DESC)Sharding:
Shard payments by user_id, matching the primary order-history access pattern.
Payments can still be retrieved by order_id using a secondary lookup index or order-to-shard mapping.
The Payment Service uses provider-side idempotency keys so retries do not cause duplicate charges.
Store payment tokens and provider references, not raw credit card details.
Shopping carts have different consistency requirements depending on whether adding an item reserves inventory.
For this system, adding an item to a cart does not reserve inventory.
A cart is simply a temporary collection of items the user intends to purchase.
cart:{user_id} -> Hash {
item_id: quantity
}
Redis is suitable because carts require fast reads and writes and can expire after inactivity.
Operations:
HSET: Add or update an item.HGETALL: Retrieve a user's cart.HDEL: Remove an item.EXPIRE: Set inactivity expiration.Inventory is reserved only when the user actually submits a purchase request.
If the product instead guarantees stock when an item enters a cart, cart operations must create durable inventory reservations with expiration and release handling. Redis cart expiration alone is insufficient.
Admission state is temporary and accessed frequently during traffic spikes.
Redis is suitable because it supports low-latency reads, atomic operations, and native TTL expiration.
Example keys:
sale:{sale_id}:user:{user_id}:active_attempt
-> request_id, TTL
sale:{sale_id}:admitted_count
-> Integer
sale:{sale_id}:waiting_room
-> Sorted Set
Access patterns:
Redis operations such as SET NX EX and Lua scripts provide atomic admission checks.
Admission state is not authoritative inventory or payment data. Redis failures must not create duplicate purchases or overselling.
For distributed SQL, we will use leader-based replication with a consensus protocol such as Raft, assuming the selected database supports this architecture.
A newly elected leader retains previously committed writes, assuming the consensus quorum and durability guarantees hold.
For Cassandra, we use replication across nodes and availability zones. Catalog reads may use consistency levels appropriate for eventually consistent browsing.
For Redis, we use replication, failover, and partitioning for availability and scalability. However, Redis replication alone does not guarantee that every recently acknowledged write survives failover. Critical admission state therefore requires recovery and reconciliation logic.
| DataStoragePrimary reason | ||
| Users | Distributed SQL | Transactions, uniqueness, durability |
| Item catalog | Cassandra | Scalable writes and query-specific reads |
| Inventory | Distributed SQL + optional Redis token gate | Strong consistency and hot-SKU protection |
| Inventory reservations | Distributed SQL | Durable, atomic stock allocation |
| Orders | Distributed SQL | Transactional state and idempotency |
| Payments | Distributed SQL | Durable financial records and deduplication |
| Shopping carts | Redis | Low-latency temporary storage |
| Admission state | Redis | Atomic limits, fairness state, TTL |
| Feed cache | Redis / CDN | Fast, scalable browsing |
The central architectural principle is to separate high-volume, temporary coordination state from durable business state.
Redis handles cheap admission decisions and temporary state, Cassandra supports scalable item browsing, while distributed SQL remains authoritative for inventory reservations, orders, and payments.
In this design, there are write path (catalog item ingestion) and read path (item browsing, flash sale participation)
For the write path, when the item sources update item catalog, we publish events on the Kafka topic. The item data writer consumes the events, and writes the item information to the item catalog database. A separate database writer also queries the inventory service, and update the item inventory counts. We use Kafka for:
For the read path, all requests first go through 3 components:
For login/register, the request is processed by the identity service, which reads from/writes to the identity database.
For feed browsing, the request is processed by feed service. Feed service retrieves latest items and relevant deals from the item catalog database. It stores some popular items/deals in feed cache, to alleviate traffic on database.
Once user decides to join the flash sale after browsing, the request is handled by the flash sale admission controller, with below steps:
How do we avoid double purchase of the last item?
When we update the inventory database, we use atomic conditional transaction, like:
UPDATE available_inventory_count SET available_inventory_count - 1 WHEN available_inventory_count - reserved_inventory_count > 0
UPDATE reserved_inventory_count SET reserved_inventory_count - 1 WHEN reserved_inventory_count > 0
It is implementing optimistic concurrency control, as we expect conflicts to be rare, since most requests have been rejected by the inventory redis.
What happens if one of the service is down, (for example, payment service), when we have already updated the other databases?
There are 2 ways to solve this problem, either through SAGA or through 2-phase commit.
In our case, 2 phase commit is a poor choice, since it requires a coordinator that coordinates across different databases, holding locks concurrently, and it works poorly in our high volume.
We can use SAGA, which initiates compensating transactions in case of failures of some of transactions.
For example:
T1: Create order SUCCESS
|
v
T2: Reserve inventory SUCCESS
|
v
T3: Charge customer FAILED
|
v
C2: Release inventory SUCCESS
|
v
C1: Cancel order SUCCESS
If T3 fails, we will initiate C2 and C1 as compensating transactions.
The admission rate should be dynamically controlled based on order worker throughput, Kafka queue depth, and remaining inventory. We maintain a bounded purchase queue and keep excess users in a waiting room rather than Kafka.
We also use inventory-aware admission to avoid processing millions of requests for a small number of available items. Once inventory is sold out and no reservations can expire or be released, we close admission and notify waiting users immediately.
To guarantee fairness, we can randomly assign admission priority during a short registration window. We also define a maximum waiting duration so users aren't stuck indefinitely.