Loading...
TrackUserActions (clicks, page visited)
ReadActions
Availability --> as a customer, I want the dashboards system to be up at all times, plus, the tracking of actions to not suffer from downtime
Scalability --> I want to be able to ingest as many user actions as possible
Latency --> Mainly for the when we are reading data to fill the dashboard. <1 sec is good, preferably <200ms
Given a platform that targets stores growing organically:
Lets say we have: 10000 customers
For each customer: 1000 store visitors
Amount of writes to the system: 10 actions per store visit per customer
Total Writes = 100.000.000 actions per day
Each action is 1KB
Storage needed = 100GB a day * 2 years = 73TB
Lets round it to 100TB (with some buffer for growth) of storage that is needed for 2 years.
logAction:
Input --> actionId, actionType, timestamp
Output --> 201 created
readAllActions:
Input --> userId, actionType
Output --> list of all actions by type
User: id, store
Action entity: id, name, type, timestamp, owningUser
Relationship: a user owns none, one or many actions. An action is owned by a user.
Access Pattern: given a User --> retrieve all actions of given type
I would say since the system is write heavy, choosing a NoSQL DB to store data would be the right choice.
NoSQL DB would also give us the flexibility of adding new action types that might have different structures compared to existing ones.
NoSQL DB fits the NFR well since they can be horizontally scaled and we don't need strong consistency of the data.
I would pick Cassandra over MongoDB because Cassandra is optimized for write heavy workloads, all nodes are able to make write operations and we avoid having to create a sharded datastore if we used MongoDB given that only the master node can write.
As for reads, we would be better off if we use a database system that would allow for better querying capabilities. An Analytical warehouse with SQL support makes sense here.
So, the actions would be saved initially into Cassandra and we can have some sort of data management tool that asynchronously reads that data and inserts it into our warehouse.
Given that the application would need to run complex JOIN logic to be able to fullfill the asks of our different use cases (existing and future ones) data can be precomputed/materialized to whatever the expected shapes are. In this way, we would (initially at least) avoid any sort of caching needs.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?