Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) Provide type ahead suggestion
2) System should be able to update it after in case of any trending topics
List non-functional requirements for the system...
1) System shouuld highly scalable
2) System should be failut tolorent
Estimate the scale of the system you are going to design...
Let's assume the number of request made to this type ahead suggestion is 10^7 then for calculating the qps for this application we will divide it by t0^5 then the qps is 100
Define what APIs are expected from the system...
We will have two api's one is for getting the top text suggestion and other is to update the text suggestion
1) textSuggestion - > The following api gives the top suggested values for a particular prefix
2) UpdateSuggestion-> This api is used to update the prefix tree which the particular string
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
For typeahead system we will not be storing anything in the database except the user related information for storing user related information relation database will be sufficient.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
For storing all the values and their prefixes we willl be using a special data structure
Each word will be stored in the trie with their frequency and in order to save the time of traversing the whole trie we can precompute the top 10 most highly searched key and store it in the nodes.
In case of any trending words an offline job will run which will update the trie with the value and it's frequency as storing the trie in single server can cause bottle neck we will divide the trie in some character range for Example a server will store the range from A-D and other will store from E -H and so on for managing the which node stores which range we will use zookeeper as a cordinator service to manage the ranges.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
The user makes the call to webserver which directs the call to load balancer the load balancer implements some rate limiting algorithms to prevent ddos attacks. The load balancer then directs the call to api server in case of text Suggestion api's the call we be made to zookeeper which will redirect to a specific node and in case of update Suggestion the call will be stored in kafka and will be executed as an offline process in case of any trending topic we can store the prefix in a cache like redis which will directly serve the top 10 suggested instead of making call to zookeeper for monitoring logs and other application health we will be using Prometheus
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
The user makes the call to webserver which directs the call to load balancer the load balancer implements some rate limiting algorithims to prevent ddos attacks.The load balancer then directs the call to api server in case of textSuggestion api's the call we be made to zookeeper which will redirect to a specific node and in case of update Suggestion the call will be stored in kafka and will be executed as an offline process in case of any trending topic we can store the prefix in a cache like redis which will directly serve the top 10 suggested instead of making call to zookeper for monitoring logs and other applicatoin health we will be using promethes.
Explain any trade offs you have made and why you made certain tech choices...
A particular node can receive more requests then other nodes and it can cause serious bottlenecks to prevent this we can further divide the trie on more names based on the prefix Hotspot character and the next character. this will prevent bottlenecks
Try to discuss as many failure scenarios/bottlenecks as possible.
A particular node can receive more requests then other nodes and it can cause serious bottlenecks to prevent this we can further divide the trie on more names based on the prefix Hotspot character and the next character. this will prevent bottlenecks
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
We can try to incoporate machine learning in the system.