Requirements

Functional Requirements:

• Create a short URL for a given long URL.
• Return the long URL associated with a given short URL.
• The short url should be 8-characters long, including texts and numbers.

Non-Functional Requirements:

• List the key non-functional requirements (eg low latency, scalability, reliability, etc.)...
• Consistency
• Scalability
• response time < 30 secs
• High availability
• Low redirect latency
• horizontal scalability

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

1. POST /api/url/create: Takes a long url as a request, and returns a short url as a response
2. GET /api/url/short: Takes a long url as a request, returns the short url connected to that long url
3. GET /api/url/long: Takes a shorturl as a request, returns the longurl connected to that short url

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Concept:

The goal of this project is to design a service that shortens long URLs into a preferrable 8-bits long URL which can reduce that considerable space in writing long URLs. A URL shortener shortens a long URL into a concise, easy to memorize, easy to share and aesthetically pleasing short URL.

Design:

The URL shortener takes a long URL as an input from a user and returns a short URL. When a user sends a request whose body contains a long URL, a service (API) takes the long URL and create a unique short URL, the created short URL is a path on my domain and it must be unique so as to prevent two long URLs being linked to the same short URL. for example,

USER LONG URL: "https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSGxwl4fhem6m2urbLPsROL7UHP2tjF4CowszALXe5IwDIMG7C2VdCqSyLCsNi4mrqWuKJgVE215SQCpX3TOTQSUgwhOnN_kBr2FB6omLvWww",

EXPECTED SHORT URL: https://my-domain.com/aXb324

Geographically, this program should be able to handle multiple users from different parts of the world. Users are expected to be able to communicate with this program via a mobile app, a desktop app, a web application through an API.

Active Users: 500,000 - 1,000,000 daily active users

1,000,000 - 3,000,000 monthly active users

Read vs. Write: 100 : 1

Queries per secs:

• Read: (1,000,000 users * 100 reads) / (24 * 60 * 60) = 1157 QPS
• Write: (1,000,000 users * 1 write) / (24 * 60 * 60) = 11.57 QPS

Storage: (Assuming every users have at least 5 long URLs stored on my DB for 3 years)

• Number of objects: 5 * 1,000,000 * (3 * 12 * 30) = 5.4B
• size of each objects = 2000 bytes
• Total object size = 2000 bytes * 5.4B = 10.8T

Throughput:

• Read: 1157 QPS * 2000 bytes = 2.314M = 2259 MB/Secs
• Write: 11.57 QPS * 2000 bytes = 23,140 = 22.59 MB/Secs
• Cache (following the 80%-20% rule): 2259 MB/S * (60mins * 60secs) * 0.2 = 1588 GB/hr

Security:

• SQL injection: SQL injection is the placement of malicious codes in SQL statements through a web page input. To tackle this problem, every input by the user is validated before executing.
• DOS and DDOS attacks: DDOS attacks are malicious attempt to disrupt a server, service, or network's normal traffic by overwhelming it with a flood of internet traffic from multiple, often compromised, sources. With the introduction of a rate limiting technology, multiple request from the same IP address, same user are blocked from interacting with the database.

Reliability:

• Using the replication technique, data stored on a server are replicated to another server, this secondary server will be updated daily and can act as a backup server in an event where the main server is down. The database can also be dumped into a HDD on a two-days basis.
• Having to get another server of the same size with the main server will bring about an increased cost and storage. Now, we are looking at a total storage of about 11T per DB.

Scalability:

• When daily users increase, there will be an adoption of vertical scaling. We increase the performance of our current servers by increasing the space, RAM, and processors.
• If that is not enough to handle the data traffic and users, then we scale horizontally, adding one or two servers with increased efficiency.

Component communication: There are different components in this program, and they communicate in different ways. The client component communicates with service layers by making REST API calls to the API endpoints. And the API endpoints communicate with the data layer by directly accessing the database.

A URL is submitted as a web page input which is send from the frontend as JSON data as a request to the appropriate API endpoint. The submitted data if it does not exist on the database is then transferred to the shortening service else, a response containing both the long URL and the short URL is returned via an API call from the DB to the user.

Components in this program communicates over the internet and uses HTTPS through REST API calls as the protocol. Requests and response are in form of JSON data.

Communication is synchronous and they wait for response to be sent before they continue to run.

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

• Shortening Service: The shortening service takes a text input (the long URL), and generate a short URL based on a shortening algorithm. As an output, the shortening URL returns an object in the form

{

"long_url": "https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSGxwl4fhem6m2urbLPsROL7UHP2tjF4CowszALXe5IwDIMG7C2VdCqSyLCsNi4mrqWuKJgVE215SQCpX3TOTQSUgwhOnN_kBr2FB6omLvWww",

"short_url": "my-domain.com/aXybC23

}

• Shortening algorithm: The shortening algorithm generates a 8-character string which will serve as a path that redirects the user back to the webpage whose long URL is stored. The shortening algorithm scans the English alphabets (both capital letters and small letters) and digits (from 0 - 9), generating a random string which will be a concatenation of 8 different characters fetched at random at different iteration. This alone guarantees a unique ID, but to be safe, any 8-character string generated will be scanned against the DB to ensure that no two long URLs are being routed to the same short URL
• Redirection: After a short URL have been stored in the DB. Whenever that short URL is being read by a web browser, the user is then rerouted back to the relating long URL.