System requirements

Functional:

• generates a shorten url from a url
• users can delete a shortened url
• counting clicks on a shortened url

Non-Functional:

• system will be highly available and highly consistent
• 100ms response time but with a longer url a few 100ms is fine.
• auth is required for url updates
• monitoring and logging
• easy to use

Capacity estimation

Estimate the scale of the system you are going to design...

• 1 read per second
• 1 url created every 4 seconds
• 4.6mb of storage per day of short urls
• 16.79 gb of short urls generated over 10 years

API design

POST "/shorturl/create"

{

original_url: ...

}

GET "/:token"

PUT "/shorturl/:shorturl_id"

{

original_url: ...

}

DELETE "/:token"

PUT "/shorturl/:shorturl_id"

{

original_url: ...

}

Database design

SHORTURL

• id - int
• original_url - string
• shortened_url - string
• user_id - int
• created_at - datetime
• clicks - int

User

• id
• password
• email
• first_name
• last_name

High-level design

Load balancer in front of services. Api gateway in front of services. Url service. Memcached. Kafka Stream. Database.

Request flows

User creates a shortened url... client hits api gateway, gateway sends data to url service, url service sends data to database to create record.

User clicks a shortened url link. client hits api gateway, api gateway directs to url service, url service checks memcache for url via the url token and the url gets sent to the client and client redirects using the shortened url. At the same time the url service makes a request to memcache, it also adds the click to our stream. Every minute our worker hits the stream and does a batch update to dynamo db adding the new clicks from the kafka stream.

For deleting a url, a user must be authenticated of course. Once authenticated user will click the delete button next to the url they want to delete, we make a request to the backend, backend checks session to see if the signed in user is the created of the tiny url in question and if so deletes the tiny url.

Detailed component design

We will have multiple URL services and when we have too much load we will spin up additional url services. When shortened url has been updated we invalidate the cache for that shortened url. When we have a cache miss we get the record from the database and update our memcache. Our worker will consume all records in our kafka stream every minute and do a batch update to our database.

Trade offs/Tech choices

Could have used a queue instead of a stream but we would have been processing one job in the queue at a time and it would be better to do a batch update instead of a bunch of singleton updates.

Failure scenarios/bottlenecks

Our database will be backed up and we can spin up a replica if it fails. Our stream will be logged so we can recreate the stream if our stream fails so we don't miss any clicks.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?