Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

In Scope:

• We need to send a long URL and create a short URL to return to the user
• That short url once you opened it you will be redirect to the long URL

Out of Scope:

• Metrics/analytics
• Private URLs / Authentication
• Custom Aliases

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

Based in the CAP theorem:

As a base we will have P because it's required to have partition tolerance

and as well for the base of the project we will have A from Availability

so this system design will be based in a AP solution

we will need:

• High Availability
• Low latency in redirects
• horizontal scalability

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

Estimations:

Total Users: 10M

Daily Active Users: 500K (5% of total users)

Resources created daily -> 1 URL per day -> 500k writes/day

Read/write Ratio: 1:100 -> 50M reads/day

Register Sice: ~500B (key, original url, timestamp, overhead)

Trafic:

Writes: 500k / 86,400 = 6 QPS (queries per second)

Reads: 50M / 86,400 = 580 QPS avarage (queries per second) with spike ~3x = 1.7k QPS

High read required, low latency, high availability

Trafic Increase:

2x yearly in Daily Active Users

Storage:

With the trafic Increase in year 5 we will require around 2.8 TB of storage

In this case we will be expecting to have database sharding, because this table can be increasing all the time, we will never delete any data in the database

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

The base for this design based in the functional requirements are two base actions

create resource

read and redirect to original resource

POST /api/v1/short

Payload:

• originalUrl: string

Response:

• key: string
• originalUrl: string
• createdAt: string

Errors:

• 400 -> invalid URL

GET /api/v1/url/{key}

Response:

• Redirect 302 -> original Url

Errors:

• 404 -> Not Found

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

For this especific case to make all more clear I will be using AWS as the Cloud Provider

Client will be the browser/device which is going to make the request

the request will be received by the api gateway, can be for AWS as well

• this will help to manage rate limiting
• avoid suspicious requests

the Load Balancer will help directly to hit first to Redis to see if the resource is already cached, if not will hit to the available Api Service to handle the request

the Api service will be a EC2 container with multiple replicas to allow multi AZ

And the database will be cassandra for example in order to help with the sharding, high horizontal scalability, millions of registers

POST roadtrip:

• Client makes request
• Request pass through api gateway and load balancer
• the request is recieved by the Key creator service
• that creation event is going to be sand to kafka (this to allow hard write consistency)
• then the api service is going to received the new key to be saved in the database (this can be async)
• I will be using a TTL and write through strategy for the redis cache

GET roadtring:

• Client makes request
• Request pass through api gateway and load balancer
• the key hits to the cache layer first
• if the value is in cache, make redirect
• if not get it directly from the URL shorter api service
• redirect or 404

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

For this especific case I'm thinking that the database will be cassandra in order to help with the sharding, high horizontal scalability, millions of registers

The main entity will be something like 'shorted_url'

• short_code -> key, unique, string, primary key
• long_url -> string, not null
• created_at -> timestamp in utc, default now()

Out of this scope will be have an external entity that will be for the users in the platform

this in order to make the relation directly between the user and the short url created by the user

in that case I will add another property that will be created_by_user_id: uuid

the partition key that we will use is short_code

High considerations:

• Unique key creations
• Eventual consistency in redirects with low ms will be acceptable (404 on redirect)
• The create consistency will be strong, because it's required to once a code is generated that one will be persisted without duplications
• An External service will be the one that will have the full responsability of key creations, this in order to avoid duplications, the code can be created in multiple ways inside the service, one can be a hash based in the timestamp+url

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Lets explain a little bit about both flows

We received a register for create a Short Url, the load balancer will call the Key Creator Service

This service will create the Key for that short URL, that can use a hash algorithm based in the timestamp and the long url for example or using an idempontent key

This one will require to generate a unique value

methods that i will avoid for that key:

• UUID
• Math Random

What I'm thinking for this will be something like:

• Snowflake IDs
• autoincrement

then convert it to Base62 of 8 characters

Once the key is created I will send a kafka event with the key, original URL

With this we own eventually read consistency, high creation consistency

then a short url service consumer will received the kafka event in order to save that data in to the cassandra database

based in the partition key will determine in which database we will be saving that

Also in this case Kafka as well can save the value directly in cache with a small TTL (5min)

In the reads we will be receiving the request, the main idea is that the latency will be very slow so I expect something like <20ms based in the standar

So once de request arrives to the load balances we will check the redis cache

if exits return it and redirect

if not we will hit the URL shorter service

we will ask to cassandra based in the PK from the key

once we validate it exists we will be async adding it to the cache layer

and making the redirect

it's important that the cache added step will be async to avoid adding more latency to the request