Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
We will estimate both peak read/write QPS, as well as storage.
Assuming 100 million DAU (readers) on the platform, on each day, a DAU on average does 5 browses/reads/searches etc. Assuming peak QPS is twice of average QPS.
Peak read QPS = 100 million * 5 / 3600 / 24 * 2 is approximately 10K
We need to record the position a user last read to sync position. This part we have very high QPS. Assuming each DAU reads for 30 minutes per day on average, and we have one write per 10s while reading: 100M × 30 min × 6 writes/min ≈ 18B writes/day ≈ ~200K avg / ~400K peak write QPS assuming on each day, the user reads and finishes reading for 2 times.
For storage, assuming we have 10 million books on the platform, and each books contents are 2Mb, with 1kb of metadata.
We will need storage for:
Object storage of 2Mb * 10 million = 20TB
Metadata storage of 1Kb * 10 million = 10GB
When users read the book, we download the book to their devices. We will need network bandwidth of 10k QPS * 2MB/book = 20GB/s at peak.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
For readers:
To purchase a book so they can download:
POST v1/purcahse_book {
book_id: UUID,
user_id: UUID,
payment_method: String,
purchased_at: Timestamp
}
To download a book so users can read it:
GET v1/download_book {
book_id: UUID,
user_id: UUID
}
This returns a encrypted link for users to download the book to their local device.
To get recommendations for books:
GET v1/get_book_recommendations {
user_id: UUID
}
To search for books:
GET v1/search_books {
user_id: UUID,
search_query: String,
filters: List
}
To publish latest reading progress of users:
POST v1/publish_progress {
user_id: UUID,
book_id: UUID,
page_number: Integer,
offset: Integer
}
For authors:
To upload books:
POST v1/upload_books {
book_title: String,
book_description: String,
book_author_ids: List
book_author_names: List
book_pages: Integer,
book_language: String,
book_genre: String,
book_metadata: String
}
For publishers:
To publish books:
POST v1/publish_books {
book_id: UUID,
publisher_id: UUID,
publisher_name: String,
book_price: Integer,
book_release_regions: List
}
For all, to register and login:
POST v1/register {
user_name: String,
user_email: String,
user_phone: String,
user_password_hashed: String,
user_role: String,
}
POST v1/login {
user_name: String,
user_email: String,
user_phone: String,
user_password_hashed: String,
user_role: String,
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
For any request to the system, we first go through an external load balancer, which routes request to API gateway instances. API gateway then handles authorization, authentication and rate limiting, before internal service mesh further distribute the traffic.
We first talk about the write path, authors and writers. For a writer to write a book, we go to the author service. We write to the redis cache and underlying database about the book metadata, and upload the book contents to S3 and CDNs. We also trigger an event on Kafka queue, which gets processed and writes the book metadata to ElasticSearch. At this stage, the contents are not published, so it's not active for search, recommendations or retrieval.
For publishers, we send a request to publisher service. We first go to an external certification service, that ceritifies:
Once all of these information is certified and aligned, we update the redis, relational database, and ElasticSearch to indicate that a book has been published.
For readers, we send request to recommendation service, which retrieve the personalized feed for the user. We query redis/database to get the user, book, author, publisher metadata for display. The recommendation service uses an internal ML model to generate and rank results.
Once the user places an order for a book, we send the request to an external payment provider to handle the payment. On purchase, the payment success handler should call the DRM service to mint a license (user_id, book_id, device_id, expiry), then hand back a CDN URL whose decryption key only that license unlocks. Upon success, we update the redis and relational database on the new purchase, and send back the user the signed URL for downloading the book from CDN.
Once downloaded, the user can use the app to read book. When they are reading them, we sync their progress every 10 seconds, and write their latest progress to cassandra database. If they log in from another device, we can read their progress directly from cassandra. So syncing happens in real time.
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
Here's the data we need to store:
For actual books, the contents are quite large, so they should be stored in S3 object storage, with globally distributed CDN servers as caching layer.
For book and users metadata, relational database is a good fit, for below benefits:
The tradeoffs are:
Here's some sample data models:
table readers {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table books {
book_id: UUID,
book_title: String,
book_description: String,
book_author_ids: List
book_author_names: List
book_pages: Integer,
book_language: String,
book_genre: String,
book_metadata: String
}
table authors {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table publishers {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table user__purchased_books {
user_id: UUID,
purchased_at: Timestamp,
book_id: UUID
}
table user__authored_books {
user_id: UUID,
authored_at: Timestamp,
book_id: UUID
}
table user__published_books {
user_id: UUID,
author_ids: List
authored_at: Timestamp,
book_id: UUID,
published_at: Timestamp
}
For the frequently accessed data, such as popular users/books/authors/publishers metadata, we cache them in redis. In our case, we use a write through cache, where any write writes synchronously to both the cache and the database. The benefits are:
If the number of users/books keep increasing, we will need to shard the different tables with different shard keys. For example, for user_purchased_books table, we will shard by user_id, so we can easily retrieve the list of books a user has purchased.
With sharding to handle large data size, we will also use replication to increase availability. For each shard, there will be several read replicas which we synchronously replicate data to. In case of master crash, we will promote a read replica to be the new write replica.
For books metadata, they are also written to ElasticSearch. So users can easily search by title, or filter by genre, title etc.
For the positions that user last read to, this has very high write throughput (400K peak QPS) and high availability requirement, so we will use a cassandra database, which is a globally distributed noSQL database, optimized for high write throughput and naturally support horizontal scaling.
The table can be:
table user_read_position {
user_id: UUID,
book_id: UUID,
page_number: Integer,
position_index: Integer
}
The user_id will be used as partition column where the book_id will be used as sorting column.
The tradeoff of using cassandra is eventual consistency, which is acceptable in our case. If a user's device A records that they read to page 100 position 800 of book X, it's okay if device B temporaily gets a stale data of page 99 position 97 of book X.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
Here's the data we need to store:
For actual books, the contents are quite large, so they should be stored in S3 object storage, with globally distributed CDN servers as caching layer.
For book and users metadata, relational database is a good fit, for below benefits:
The tradeoffs are:
Here's some sample data models:
table readers {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table books {
book_id: UUID,
book_title: String,
book_description: String,
book_author_ids: List
book_author_names: List
book_pages: Integer,
book_language: String,
book_genre: String,
book_metadata: String
}
table authors {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table publishers {
user_id: UUID,
user_name: String,
hashed_user_password: String,
user_email: String,
user_geo: String,
user_joined_at: String,
account_status: String,
user_metadata: String
}
table user__purchased_books {
user_id: UUID,
purchased_at: Timestamp,
book_id: UUID
}
table user__authored_books {
user_id: UUID,
authored_at: Timestamp,
book_id: UUID
}
table user__published_books {
user_id: UUID,
author_ids: List
authored_at: Timestamp,
book_id: UUID,
published_at: Timestamp
}
For the frequently accessed data, such as popular users/books/authors/publishers metadata, we cache them in redis. In our case, we use a write through cache, where any write writes synchronously to both the cache and the database. The benefits are:
If the number of users/books keep increasing, we will need to shard the different tables with different shard keys. For example, for user_purchased_books table, we will shard by user_id, so we can easily retrieve the list of books a user has purchased.
With sharding to handle large data size, we will also use replication to increase availability. For each shard, there will be several read replicas which we synchronously replicate data to. In case of master crash, we will promote a read replica to be the new write replica.
For books metadata, they are also written to ElasticSearch. So users can easily search by title, or filter by genre, title etc.
For the positions that user last read to, this has very high write throughput (400K peak QPS) and high availability requirement, so we will use a cassandra database, which is a globally distributed noSQL database, optimized for high write throughput and naturally support horizontal scaling.
The table can be:
table user_read_position {
user_id: UUID,
book_id: UUID,
page_number: Integer,
position_index: Integer
}
The user_id will be used as partition column where the book_id will be used as sorting column.
The tradeoff of using cassandra is eventual consistency, which is acceptable in our case. If a user's device A records that they read to page 100 position 800 of book X, it's okay if device B temporaily gets a stale data of page 99 position 97 of book X.
For any request to the system, we first go through an external load balancer, which routes request to API gateway instances. API gateway then handles authorization, authentication and rate limiting, before internal service mesh further distribute the traffic.
We first talk about the write path, authors and writers. For a writer to write a book, we go to the author service. We write to the redis cache and underlying database about the book metadata, and upload the book contents to S3 and CDNs. We also trigger an event on Kafka queue, which gets processed and writes the book metadata to ElasticSearch. At this stage, the contents are not published, so it's not active for search, recommendations or retrieval.
For publishers, we send a request to publisher service. We first go to an external certification service, that ceritifies:
Once all of these information is certified and aligned, we update the redis, relational database, and ElasticSearch to indicate that a book has been published.
For readers, we send request to recommendation service, which retrieve the personalized feed for the user. We query redis/database to get the user, book, author, publisher metadata for display. The recommendation service uses an internal ML model to generate and rank results.
Once the user places an order for a book, we send the request to an external payment provider to handle the payment. On purchase, the payment success handler should call the DRM service to mint a license (user_id, book_id, device_id, expiry), then hand back a CDN URL whose decryption key only that license unlocks. Upon success, we update the redis and relational database on the new purchase, and send back the user the signed URL for downloading the book from CDN.
Once downloaded, the user can use the app to read book. When they are reading them, we sync their progress every 10 seconds, and write their latest progress to cassandra database. If they log in from another device, we can read their progress directly from cassandra. So syncing happens in real time.
Never encrypt the book with the user's key directly — you'd have to store N encrypted copies of the same book. Instead:
So S3 stores ciphertext(book) + wrapped_CEK. Neither is usable alone.
Payment success → DRM service: 1. Verify entitlement exists (paid, user_id, book_id) 2. Unwrap CEK via KMS 3. Bind to device: encrypt CEK with the device's public key (device registered earlier with a keypair; private key lives in the reader app's secure enclave) 4. Sign license with the DRM service's private key (anti-forgery) 5. Store license → license store (Cassandra/Postgres)
The license record: license_id, user_id, book_id, device_id, wrapped_CEK, expiry, max_devices, offline_validity, signature.
The signed CDN URL is only released after the license exists — which your design already does.
1. App requests download → gets short-TTL signed CDN URL 2. App downloads ciphertext from CDN 3. App presents license_id + device cert → DRM returns wrapped_CEK 4. App unwraps CEK with its device private key 5. App decrypts the book with CEK
No reader licenses — but yes, it touches encryption at ingest. The publisher supplies the plaintext file; the ingest pipeline encrypts it with a fresh CEK on the platform's behalf (after your certification service clears originality/rights). The publisher never mints user licenses because they never consume encrypted content.
The one place publishers do set policy: DRM entitlements — e.g., "allow copy," "max 3 devices," "offline for 30 days." That's license policy input, not minting. Worth a line in your design: "Publisher sets DRM policy per book at upload; DRM service enforces it at license time."