Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
User authentication
Convert LongUrl to a shortUrl
Redirect the user to the shortUrl.
User can enter custom expiration timestamp for the shortUrl
Track url usage analytics
List non-functional requirements for the system...
Availability
Scalable
Fault tolerant
Low-latency
Estimate the scale of the system you are going to design...
50000 Daily Active Users
Every user makes 10 requests a day
500K requests per day
There can be peak traffic at certain time intervals.
On average, let's calculate traffic:
1 day = 24 * 60 * 60 seconds = 86400 seconds.
500K rpd = 5 requests per second, which isn't a lot.
1 server could serve all 5 requests if there's not a lot of backend computation, but for providing low-latency and fault tolerance, we can set up multiple servers in different availability zones.
Database side:
Let's assume each user wants to create 1 shortURL.
And let's assume that the read-to-write split of these requests is 4:1.
So 50k DAUs implies 1/5th of 50k are new short URLs, i.e., 10k new URLs daily.
Now annually, this is 10k * 365 = 3.65M records.
Each URL will be converted to 7 ASCII digits, so we need to store 7 bytes for shortURL, plus other fields such as creation timestamps, user id, expiration timestamps. So the total size of each record can be assumed as 1Kb.
For 3.65M records, it's 3.65GB in a year, which is again not a lot.
For storing the data for 5 years, it can accumulate upto 18.25GB.
Define what APIs are expected from the system...
Authentication APIs:
Main APIs:
Parameters:
user_id
longURL
expiration_timestamp (default: current_time + 2 months)
Return:
shortURL
Parameters:
user_id
shortURL
Return:
longURL
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
In the initial design, we can setup a simple authentication system, based on username, password, where we store the username, encrypted password in our database.
2 Tables:
We can optimize our URLLinks storage space by storing just the shortened 7 digit encoding string, instead of the complete link that has https:// domain details. We can add these details in the application logic.
Moreover, we can remove the http(s):// from the longURL string, and instead use a single bit field that identifies whether the link is secure or not.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Use Nginx for load balancing server and web server to serve static content efficiently.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Load Balancer:
We can setup a pair of load balancers (Active and Standby), so that in case of failure, we can immediately switch traffic to the standby server.
The load balancer routes requests based on custom routing technique.
We can decide it to be based on a weighted round robin approach, in which the weight is inversely proportional to the server load.
We can run Ngnix server as reverse proxy and load balancer.
Only the load balancer will have public IP, in order to secure the rest of our system from external misuse.
Application server:
Performs the user authentication.
Serves user read and write requests to create and redirect the shortened URL.
The IPs of these servers will be private and accessible only through NAT tables through a few ports (443, 80) to load balancer and other internal services.
Database and Cache:
We can use a key value database storage system to quickly retrieve the URL information.
Using key-value is better for availablility, low-latency and serving quick writes.
The cache can also be a key-value store in-memory data storage system such as Redis.
The database and cache are also going to be network protected, and accessible through the authorized services.
Explain any trade offs you have made and why you made certain tech choices...
NoSQL Db over SQL Db as we require high availability and scalable service that can be easily partitioned and replicated across many servers.
SQL DB is useful for performing ACID transactions, which we don't need in our system.
Try to discuss as many failure scenarios/bottlenecks as possible.
Currently the failure scenarios could be dealing with peak load of traffic. The load balancer and application servers can get overwhelmed, or if there are too many write requests or cache misses on read requests, then this can escalate to overloading the database.
We have to tried to setup an active and passive load balancing strategy, this will mitigate the SPOF issues on load balancing side.
Another potential failure scenario is with user authentication system, which can be prone to hacking attacks.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
To deal with peak traffic, we have to perform load testing and measure the impact on our current system and accordingly configure auto-scaling.
To secure authentication system, we can use JWT or OAuth2 authentication method to increase overall security.