High-Level Design
- encryption to enforce distribution
- ETL pipeline to do user analysis
- cross platform support
- list filters
No SQL database that can scale horizontally.
Author ebook management
- authors have a special dashboard where they can manage the content they own
- this has strict access control so only the author can adjust their books
- use the list_author_books api to get the list of books author can manipulate
- when registering a new book first the author uses create_ebook_content then they fill out a form and do the create_ebook_metadata
- when author is ready to publish their book they use the append_active_content_version(version) api. This will add the given version as an active version of the book. ebook_metadata maintains a list to track the history of active versions
Ebook encryption
- Use public and private RSA keys to encrypt and decrypt books. Book content is encrypted in rest and at transit. Only when the user is viewing the content through our reader to we decrypt it with the private key. The create and update ebook_content apis encrypt the content with the public key before saving the data in object storage
- User's reader decrypts it with private key to showing the content to the user. If the user just downloads raw files they will download encrypted files
Cross platform reading
- Offload buisness logic to the api backend as much as possible so that we can easily build thin clients across all the platforms we need to support
- User's need accounts desegregated from their device & ip address, they should have the same content on laptop vs iphone.
Ebook discovery
- Use the list_ebook_metadata(filter: str) api for users to discover new books. These filter strings will match a standard format such as "author=ryan&genre!=horror&sortby=title". Then the api will parse out this string and perform the neccessary filtering. This response will also be paginated
- Need to ensure we maintain indicies on fields we want to sort on. We should expect 1:100 write to read so we can afford expensive writes. We should index on every field.
User anlaytics
- Push any user actions to a user_activity API so they can be monitored
- THis should track things like the books a user is buying, how long they are spending time reading certain books, when thye read etc.
- Then this can be pushed through ETL into a machine learning ready service for analytics to be derived from.
- We can then action on these analytics. Notably for recommendations and for paying authors
E book streaming
- A full book would take too long to display to the user. Rather we will stream the book and mark sections of the book as ready. If the user is starting a book we will priortize the first page, but if they are on a page then we will start by streaming from that page and stream the pages before and after. I.E. page 10, then pages 9 & 11, then 8 & 12, etc.
- once streamed, we can download encrypted versions
- Cache the first 10 pages of the 20% most popular books
Archiving
- The sheer amount of books stored is the largest expense of operating this buisness. We should archive books that are unlikely to get published. While being mindful that users may continue to have non active versions of the book on their device
- Book content version that was never made active after 30 days
- By downloading encrypted versions of books onto user devices, we can consider putting outdated content versions that have been unactive for 30 days to be in a cheaper but slower storage tier