Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

• Users can create online stores and manage them.
• Users can track inventory counts of their online stores.
• Users can add item/inventory metadata to the item catalog. They can also manage the product and their variants.
• Users can browse available online stores.
• users can place orders at any store, pay for items and track their orders. Users can use multiple gateways for payments that are globally distributed, with different currencies.
• Users can communicate with the store owners regarding any complaints/requests.

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

• The system should be available 99.99% of time.
• The system should be easily scalable to more users.
• The p99 latency for read operations like browsing stores, items etc. should be less than 200ms. The p99 latency for write operations like placing orders, paying for items, managing stores should be less than 500ms.
• The order placement, payment etc. needs to be strongly consistent. The browsing, tracking et. can be eventually consistent.
• The system should handle payment/order placement securely using access controls/encryption.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

Assuming we have 100 million DAU, and peak QPS is twice of average QPS.

For read operations like browsing stores, tracking order status etc., assuming each user does 10 reads on average per day.

For write operations, assuming each user does 1 write on average per day.

Peak read QPS = 100 million DAU * 10 reads per day / 100k seconds per day * 2 = 20k

Peak write QPS = 2k

For storage, we need to store user information, store information and item inventory information.

Assuming 10 million stores, and each store on average has 5k items.

Assuming each store metadata is 1kb while each item metadata is 5kb, and each user metadata is 2kb:

Item metadata storage = 10 million * 5k * 1kb = 50TB

Store metadata storage = 10 million * 5kb = 50GB

User metadata storage = 100 million * 2kb = 200GB

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

For store owners to manage stores:

GET v1/fetch_store

POST v1/create_update_store

DEL v1/delete_store

For store owners to manage store inventory:

GET v1/fetch_store_item

POST v1/create_update_store_item

DEL v1/delete_store_item

For users to browse store and add items:

GET v1/browse_store

GET v1/browse_store_detail

GET v1/item_detail

POST v1/add_item_to_cart

For users to place order and pay, and track order status:

POST v1/submit_payment

POST v1/place_order

GET v1/view_order_status

For users to communicate with the store owner:

POST v1/initiate_chat

POST v1/leave_message

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

For all requests to the system, we first go through external load balancers, where traffic is distributed to different API gateway servers using consistent hashing.

Then API gateway enforces authorization, authentication and rate limiting.

In this system, we break down flows for shop owners and shoppers.

For shop owners, the several flows are:

1. Store upload/update. User creates a store through the store upload/update portal, we write to the store database once verification is done.
2. Inventory upload/update. After verified inventory update is uploaded, we trigger events on Kafka. The consumer groups are:

• Database writer. Upon ingesting the events, it writes to the inventory and catalog database. This buffering prevents overwhelming the databases for large bulk uploads.
• Analytics service: Upon ingesting the events, write to analytics database after data aggregation and cleaning.
• Notification service: Upon ingesting the events, send notifications to clients that are related to the store.

1. Store data analytics: We send requests to query the analytics database for latest data about a store.

For shoppers, the several flows are:

1. Feed service: We query the store database, catalog database and inventory database to load user feeds. We do cache user feed in redis to alleviate the loads. We query the image assets associated with stores/items from CDN, and if not available, S3 object storage. For engagement events, we also trigger events on Kafka. Upon ingestion by analytics service, write to the analytics database. This help store owners to analyze user behaviors later.
2. Payment/checkout flow: For users to place an order, after they have finalized the order details and go to the checkout phase:

• We create a order with "pending" status in the database. We also create a "pending" payment in the payment database, and an associated idempotent key with the payment.
• We then query the external payment providers, and route users to their payment providers based on user's selected payment method, geolocation etc.
• User pay
• Upon success, we update the payment/order database
• We return overall success status to the users This is overall an happy path. If the first 3 steps fail, we mark the payments as "failure" in database, and return failures to users. If the steps fail after user has paid, we retry with exponential backoff. If after a certain TTL, the request is still pending and database hasn't recovered. We trigger a rollback flow to refund user their paid amount and cancel the order.

1. User support flow: If users are not satisfied with email or phone number to contact store owners, we set up connections with websocket servers and allow them to live chat with store owners and our support agents.

This is an happy path

How to avoid race condition for last item?

In high concurrency situations for hot sales, if 2 users are attempting to buy the same item at the same time. We will use optimistic concurrency plus atomic operations to resolve conflicts.

An update to the inventory table is:

UPDATE inventory SET quantity = quantity - 1WHERE product_id = 'ITEM_999'AND quantity >= 1;

If Alice and Bob both tries to do this, the latter one will fail and get back a "sold out" message.

Since we shard the inventory count database by store_id, how do we avoid hot stores from overloading a single shard?

We can adopt a sub-sharding strategy. For inventory count of an item, the key is broken down from:

SKU-123 to

SKU-123-slot 1,

SKU-123-slot 2 etc.

And these keys can be distributed across different shards, and aggregated to give the actual product count.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

Here's the data we need to store:

1. User identifies
2. Stores information
3. Inventory information for items in stores
4. Catalog information for items in stores
5. Payement information
6. Order information
7. Analytics database

First, we want to talk about why we want to separate out catalog information and inventory information.

For catalog information, we store the metadata about a product item, like name, variants, color etc. It is read-heavy, write-light. We query information about product items all the item, while the product themselves don't change frequently. The schema is also very diverse across different products. Eventual consistency is okay.

For inventory, it stores the constantly changing information for a product, like price, inventory etc. So it is a write heavy system with relatively stale schema. Strong consistency is required.

The use case differences warrant different storage strategies.

For user identifier, store information, inventory information, payment information and order information, we will use distributed relational database.

The benefits are:

1. Strong ACID properties guarantees
2. The data is easily represented by relational data models
3. Join queries are supported if needed

The tradeoffs are:

1. Less horizontal scalability
2. Less flexible schema

For the use cases listed, the benefits outweigh the tradeoffs. The sample schemas are:

table users {

user_id: UUID,

user_name: String,

user_email: String,

user_phone: String,

user_password_hashed: String,

user_type: String,

user_address: String,

user_metadata: String

}

table stores {

store_id: UUID,

store_name: String,

store_description: String,

store_created_at: String,

store_type: String,

store_metadata: String

}

This table is indexed by store_id, so store look up is easy.

table user_store_owner {

store_id: UUID,

user_id: UUID,

user_role: String

}

table product_inventory {

store_id: UUID,

item_id: UUID,

item_count: Integer,

item_price: String,

item_inventory_metadata: String

}

This table has an index on store_id. It is to support the query of "showing me inventory information for my store's products"

table payments {

user_id: UUID,

payment_id: UUID,

order_id: UUID,

payment_time: Timestamp,

idempotent_key: String,

payment_amount: String,

payment_currency: String,

payment_status: String,

payment_metadata: String

}

table orders {

user_id: UUID,

store_id: UUID,

order_status: String,

order_contents: String,

order_time: Timestamp,

order_metadata: String

}

There is an index on store_id, used to pull orders for single store.

For inventory counts and orders/payments, we will need to shard the database as data sizes grow. For inventory count, we will shard by store_id so item inventory for the same store is put on the same shard. For orders/payments, we will also shard by store_id so information for the same store can be easily pulled.

If we want to increase availability and reduce latency, we also need to do database replication.

For data that requires strong consistency, like payments/product_inventory, we will do synchronous replication which ensures each replica is updated at the tradeoff of higher write latency.

For other data where eventual consistency is acceptable, we can do asynchronous replication.

We will have additional caching layers to support these tables:

1. An idempotent cache, that will be used to cache the idempotent key associated with each payment. This ensures that retries or double payment requests never double-charge.
2. A redis cache cluster. This will be used to cache popular information like "popular stores metadata", "user's browsing feed", "popular items metadata".

For the cache, we will use a read-aside cache structure:

1. For any reads, we read from cache. If not available, we read from database and update cache.
2. For any writes, we update the database and invalidate cache entries.

The benefits are:

1. Low read latency if cache hit rate is high
2. Data is durable as we write directly to the database
3. Implementation complexity is lower as it lives on the application layer not on the caching layer

The tradeoffs are:

1. Relatively high latency as we write to database instead of cache
2. Non-atomic operation of "writing to database -> invalidating cache" introduces eventual consistency
3. Needs to ensure high cache hit rate for benefits to surface

For catalog data, we will use a document database like mongoDB.

Catalog data schema updates frequently, so document database supports flexible schema very well.

A sample schema is:

{

"item_id": xxx,

"store_id": xxx,

"item_name": xxx,

"item_variants": xxx,

etc.

}

If the table becomes too large or read traffic becomes large, we will adopt the strategy of:

1. Sharding by store_id
2. Replication with asynchronous replication. Eventual consistency is acceptable in this case.

For analytics database, we will store the data in an OLAP database like snowflake.

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Here's the data we need to store:

1. User identifies
2. Stores information
3. Inventory information for items in stores
4. Catalog information for items in stores
5. Payement information
6. Order information
7. Analytics database

First, we want to talk about why we want to separate out catalog information and inventory information.

For catalog information, we store the metadata about a product item, like name, variants, color etc. It is read-heavy, write-light. We query information about product items all the item, while the product themselves don't change frequently. The schema is also very diverse across different products. Eventual consistency is okay.

For inventory, it stores the constantly changing information for a product, like price, inventory etc. So it is a write heavy system with relatively stale schema. Strong consistency is required.

The use case differences warrant different storage strategies.

For user identifier, store information, inventory information, payment information and order information, we will use distributed relational database.

The benefits are:

1. Strong ACID properties guarantees
2. The data is easily represented by relational data models
3. Join queries are supported if needed

The tradeoffs are:

1. Less horizontal scalability
2. Less flexible schema

For the use cases listed, the benefits outweigh the tradeoffs. The sample schemas are:

table users {

user_id: UUID,

user_name: String,

user_email: String,

user_phone: String,

user_password_hashed: String,

user_type: String,

user_address: String,

user_metadata: String

}

table stores {

store_id: UUID,

store_name: String,

store_description: String,

store_created_at: String,

store_type: String,

store_metadata: String

}

This table is indexed by store_id, so store look up is easy.

table user_store_owner {

store_id: UUID,

user_id: UUID,

user_role: String

}

table product_inventory {

store_id: UUID,

item_id: UUID,

item_count: Integer,

item_price: String,

item_inventory_metadata: String

}

This table has an index on store_id. It is to support the query of "showing me inventory information for my store's products"

table payments {

user_id: UUID,

payment_id: UUID,

order_id: UUID,

payment_time: Timestamp,

idempotent_key: String,

payment_amount: String,

payment_currency: String,

payment_status: String,

payment_metadata: String

}

table orders {

user_id: UUID,

store_id: UUID,

order_status: String,

order_contents: String,

order_time: Timestamp,

order_metadata: String

}

There is an index on store_id, used to pull orders for single store.

For inventory counts and orders/payments, we will need to shard the database as data sizes grow. For inventory count, we will shard by store_id so item inventory for the same store is put on the same shard. For orders/payments, we will also shard by store_id so information for the same store can be easily pulled.

If we want to increase availability and reduce latency, we also need to do database replication.

For data that requires strong consistency, like payments/product_inventory, we will do synchronous replication which ensures each replica is updated at the tradeoff of higher write latency.

For other data where eventual consistency is acceptable, we can do asynchronous replication.

We will have additional caching layers to support these tables:

1. An idempotent cache, that will be used to cache the idempotent key associated with each payment. This ensures that retries or double payment requests never double-charge.
2. A redis cache cluster. This will be used to cache popular information like "popular stores metadata", "user's browsing feed", "popular items metadata".

For the cache, we will use a read-aside cache structure:

1. For any reads, we read from cache. If not available, we read from database and update cache.
2. For any writes, we update the database and invalidate cache entries.

The benefits are:

1. Low read latency if cache hit rate is high
2. Data is durable as we write directly to the database
3. Implementation complexity is lower as it lives on the application layer not on the caching layer

The tradeoffs are:

1. Relatively high latency as we write to database instead of cache
2. Non-atomic operation of "writing to database -> invalidating cache" introduces eventual consistency
3. Needs to ensure high cache hit rate for benefits to surface

For catalog data, we will use a document database like mongoDB.

Catalog data schema updates frequently, so document database supports flexible schema very well.

A sample schema is:

{

"item_id": xxx,

"store_id": xxx,

"item_name": xxx,

"item_variants": xxx,

etc.

}

If the table becomes too large or read traffic becomes large, we will adopt the strategy of:

1. Sharding by store_id
2. Replication with asynchronous replication. Eventual consistency is acceptable in this case.

For analytics database, we will store the data in an OLAP database like snowflake.

For all requests to the system, we first go through external load balancers, where traffic is distributed to different API gateway servers using consistent hashing.

Then API gateway enforces authorization, authentication and rate limiting.

In this system, we break down flows for shop owners and shoppers.

For shop owners, the several flows are:

1. Store upload/update. User creates a store through the store upload/update portal, we write to the store database once verification is done.
2. Inventory upload/update. After verified inventory update is uploaded, we trigger events on Kafka. The consumer groups are:

• Database writer. Upon ingesting the events, it writes to the inventory and catalog database. This buffering prevents overwhelming the databases for large bulk uploads.
• Analytics service: Upon ingesting the events, write to analytics database after data aggregation and cleaning.
• Notification service: Upon ingesting the events, send notifications to clients that are related to the store.

1. Store data analytics: We send requests to query the analytics database for latest data about a store.

For shoppers, the several flows are:

1. Feed service: We query the store database, catalog database and inventory database to load user feeds. We do cache user feed in redis to alleviate the loads. We query the image assets associated with stores/items from CDN, and if not available, S3 object storage. For engagement events, we also trigger events on Kafka. Upon ingestion by analytics service, write to the analytics database. This help store owners to analyze user behaviors later.
2. Payment/checkout flow: For users to place an order, after they have finalized the order details and go to the checkout phase:

• We create a order with "pending" status in the database. We also create a "pending" payment in the payment database, and an associated idempotent key with the payment.
• We then query the external payment providers, and route users to their payment providers based on user's selected payment method, geolocation etc.
• User pay
• Upon success, we update the payment/order database
• We return overall success status to the users This is overall an happy path. If the first 3 steps fail, we mark the payments as "failure" in database, and return failures to users. If the steps fail after user has paid, we retry with exponential backoff. If after a certain TTL, the request is still pending and database hasn't recovered. We trigger a rollback flow to refund user their paid amount and cancel the order.

1. User support flow: If users are not satisfied with email or phone number to contact store owners, we set up connections with websocket servers and allow them to live chat with store owners and our support agents.

This is an happy path

How to avoid race condition for last item?

In high concurrency situations for hot sales, if 2 users are attempting to buy the same item at the same time. We will use optimistic concurrency plus atomic operations to resolve conflicts.

An update to the inventory table is:

UPDATE inventory SET quantity = quantity - 1WHERE product_id = 'ITEM_999'AND quantity >= 1;

If Alice and Bob both tries to do this, the latter one will fail and get back a "sold out" message.

Since we shard the inventory count database by store_id, how do we avoid hot stores from overloading a single shard?

We can adopt a sub-sharding strategy. For inventory count of an item, the key is broken down from:

SKU-123 to

SKU-123-slot 1,

SKU-123-slot 2 etc.

And these keys can be distributed across different shards, and aggregated to give the actual product count.

To prevent cascading failures in this system, we will implement circuit breakers on the servers side, and load shedding on the database side. So sudden increase of traffic to a database will be load shedded based on traffic priority, and servers will stop temporarily sending requests to database when they are degraded.

For search/discovery, we can add an additional ElasticSearch cluster and an associated search service. The search service will ingest item/store updates from Kafka, and write to the search index. Users can query search service to search for items in store or search stores.

Also, we can further extend the design so there is a corresponding service in front of database, so feed service don't query domain databases directly. For example, an order service managing order database.

Payment tokenization

PCI-DSS requirement: raw card data (PAN) must never touch our servers. Only tokenized references are stored.

Flow:

1. Customer enters card on our checkout page, which embeds the provider's hosted fields (Stripe Elements / PayPal Buttons). Card data goes browser → provider directly, never through our API gateway or order service.
2. Provider returns a short-lived single-use token (tok_...) to the frontend.
3. Frontend sends the token + Idempotency-Key to our payment service.
4. Payment service calls the provider's charge API with the token. Provider returns a customer/vault token (cus_... + pm_...) for future charges.
5. We persist only {order_id, provider, vault_token, last4, brand, status} — never the PAN or CVV.

Why it matters: compliance scope shrinks dramatically (we're token-only, not a card handler), and recurring/refund flows reuse the vault token instead of re-collecting card data.

2. Subdomain → store routing

Entry point: request hits store123.shopify.com.

1. DNS resolves *.shopify.com to the edge/load balancer.
2. At the gateway, extract the subdomain, strip .shopify.com, lowercase → lookup key.
3. Resolve store123 → store_id via:
   • Hash map in a distributed cache (Redis) for the common case — O(1), and the mapping is hot.
   • Trie when you need prefix/wildcard matching for custom domains (shop.mycoolbrand.com) — O(L) in the domain length.
4. Cache miss → look up stores by domain (indexed), populate cache.
5. Attach store_id to the request context; every downstream shard/query uses it.

Why a trie: custom domains share suffixes, so a trie resolves them without per-domain exact-match entries — hash map only handles the exact *.shopify.com case.