Loading...
Initial:
1) 100000 users, 5 urls per user, 10 accesses per months, therefore:
5 * 100000 = 500000 new urls per month
10 * 100000 = 1000000 accesses per month
Projected:
1) 10 * 10 ^6 = total number of users
2) Assuming a standart engagemtn rate 20% of those are daily, thus 10 * 10^6 * 0.2 = 2 * 10 ^ 6 DAU
3) Assuming that a user generates 1 new link per day, we can have:
2 * 10 ^ 6 new links per day and 2 * 10 ^ 6 * 365 new link per year.
For per second we can do: 2 * 10 ^ 6 / 24 * 60 * 60 = 23 write requests per second
4) Assuming that a user accesse 2 links per day we have: 2 * 2 * 10 ^ 6 accesses per day, or 2 * 2* 10 ^ 6 / 24 * 60 * 60 = 46 new links (read requests per second)
5) Taking into account that link size is 26 bytes, shortened link size is 13 bytes, auxilarry data is 20 bytes -> we would need 59 bytes per record. Which using our previous computation would be: 2 * 10 ^ 6 * 365 * 59 = 40GB of data per year
Considering the functional requirement below I propose the following API design:
// Creates a short url from A long one
1) shortUrl/v1 {
method: POST
body : {
url: String,
expiration: Int,
accessToken: String
}
return: On success Redirect(shortLink)
On failure BadRequest(Malformed URL)
on failure Unauthorized
}
// Batch create a short url from A long one. Doesn't redirect
2) shortUrl/v1/batch {
method: POST
postBody : {
url: String,
expiration: Int,
accessToken: String
}
return: On success Ok()
On failure BadRequest(List of Malformed URLs)
on failure Unauthorized
}
// Retrieves longUrl by shortUrl with analytics
3) shortUrl/v1/ {
method: GET
requestParam: accessToken
returnBody : {
url: String,
expiration: Int,
clicks: Int,
accessedBy: [String]
Referals: [String]
}
return: On success Ok(LongUrl + analytics + TTL)
On failure BadRequest(List of Malformed URLs)
on failure Unauthorized
}
4) shortUrl/v1/customAlias {
method: POST
body : {
url: String,
alias: String
expiration: Int,
alias: String,
accessToken: String
}
return: On success Ok()
On failure BadRequest(Malformed URL)
On failure BadRequest(Alias exists)
on failure Unauthorized
}
4) shortUrl/v1/ {
method: DELETE
body : {
shortURL: String,
accessToken: String
}
return: On success Ok()
On failure BadRequest(URL doesnt exist)
on failure Unauthorized
}
4) shortUrl/v1/customAlias {
method: DELETE
body : {
alias: String,
accessToken: String
}
return: On success Ok()
On failure BadRequest(Alias doesnt exists)
on failure Unauthorized
}
User: +int recordID
User: +int UserID
User: +String UserName
User: +String HashedPasssword
User: +String Email
User: +DateTime CreatedAt
User: +int LinkID
Link: +int recordID
Link: +int UserID
Link: +String shortUrl
Link: +String LongUrl
Link: +DateTime CreatedAt
Link: +long int clicks
Link: +[Coordinates] AccessedWhere
Link: +[String] Referals
Where User has foreign key LinkID and Link has foreign key UserID. and the relationship is one to many (one user -> many links)
Diagram is included
Components included:
1) Web or phone or API user
2) DNS service
3) Web Request load balancer
4) Auth service
5) Session data cache
6) Api servers
7) Memory cache for DB
8) Master (write ) database
9) Read replica database
Included in diagram
1) Load balancer: includes an algorithm to balance a load to api servers based on the current load in the system. Includes: rate limiter: to prevent DoS attack. connects to auth service to atuhenticate the user. Connects to session service to retrieve the existing session for the user.
2) Data layer: include in memory cache service that supports low latency data retrieval. Database level include a no-sql Mongo db solution setup as a read replica to increase read trhoughput and add to reliability. For cahche we choose memcahced as a great tool for key-value storage that can efficietly resolve same link conflicts. Mongodb is chose because we want availability over consistency. DB layer also includes a background cron job to clean expired links. Mongodb also provides schema flexibility when we need to add additional analytics. To resolve concurrent conflicts we can user a timeStamp and let the last write win to have a mnimium amount of retriees. that will make our system highly concurrent. For one user conflict resolution we can store a list of LongLinks corresponding to a conflicted short one and do al inear search in case we need to retrieve one.
3) Api servers are responsible for joining operations and encoding the longUrl into shortUrl. For encoding we choose MD-5 hash schemas as the one containing pretty long encoding space vs base 62
1) Cluster db solution is not used because it's too complicated for said problem , but can be used in the furute when number of user is too big
2) Session data is kept as separate service as raliability measure , but can be part of the load balancer, same as auth
1) Failure in load balancer - > another instance of balancer gets picked
2) DB fail: if its a read replica, we pick another one. If it's a master -> some read replica gets promoted
3) Api server fail -> another instance takes it's place
4) Memory cache failure -> another isntance takes it's place
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?