Loading...
One to hash the urls (POST) and one to retrieve it (GET). There will be a cache for the shortened url as there is a high number of reads.
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
The most perfect database choice for this would be a simple (persistent) key-value store. Since key-value stores are usually NoSQL, we don't have to worry about scaling issues too much.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Client calls to load balancer which calls the server. No need to for microservices as it is a simple hashing with an add URL and retrieve.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
I'd like the database to be replicating synchronously within the datacenter + async outside the datacenter. Version conflicts aren't a big concern at all since our software doesn't really contain a ton of potential duplicate issues (note one URL can be mapped to two shortened URLs with no problems)
The only concern here is the distributed generator of the unique IDs. For this, we can use something like a hash algorithm that would take /Facebook/com/post-id/1231 and return aH102. We would need to make sure that two opposite strings wouldn't generate the hash for the same resource otherwise we would have a problem. To prevent that, every time we generate a hash, we would check if this hash is already present in the database and mapped to another resource.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
There are some trade-offs I am making when designing the read/write flows around the cache. Some choices are who is adding things to the cache and who is just reading from the cache. We are trading off write latency (which is OK to be slow) with read latency (which we care that is fast).
Another trade we are making is around consistency. We are again trading off consistency for latency, we want the request to be serving reads as fast as possible without fully waiting for all replications to complete. The reason is again we care about latency and most likely link doesn't need to be immediately available within seconds of creation. Since our users would probably not share a link instantly.
Try to discuss as many failure scenarios/bottlenecks as possible.
the shortened url hasn't been replicated to all db nodes (especially the ones across the world);
we are getting a thundering herd problem - all of these requests are going to cache miss and all go to the disk which is slow and can cause additional problems/cost in our DB usage.
We can solve this problem by using a "push" based approach in cache for famous/reserve users - we first write in cache and then we write (and replicate) on disk. That would solve the replication/and thundering herd problem, but only if the cache daemon on the application server can have the ability to precisely determine the node id where the cache node with the resource is located.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?