Loading...
1 million active users
1 million URL shortening per day. 200 per second
10 million URL redirections per day. 20.000 per second
Storage
URL record size estimation: 1 KB
1 year: 1 KB * 10^6 * 365 = 365 GB
3 years: more than 1 TB
shortenURL()
redirectURL()
URL record size estimation: 1 KB
A single field index with the Short URL is defined. As we have a CDN and a Cache, the number of queries to Mongo is reduced. To reduce memory ussage and improve write performance we don't use a compound index including the Full URL that would allow query coverage.
For high availability, a replica set with 3 nodes is used.
With the current estimations no sharding is initially needed to support a big number of documents or additiona read or write throughput.
If it's needed in the future, Short URL would be the sharding key.
Main components
Security
To avoid manipulations of the full URL, which may lead to redirection to dangerous sites, the following security measures will be enforced:
Algorithms
Short URL is generatod using a hasing algorithm using the URL and userid as input. This avoid collisions of two users sharing the same link.
In case of hash collision (already existing short URL for another user), a new hash is generated adding a sequential number starting with 0.
If a user submits an alread existing URL the system returns the short URL already existing.
Write path
Read path
Publish in CDN path
Load balancer
A L7 (application) load balancer is used to perform SSL / TLS termination, have more intelligent health checks, and to enable additional security measures like useing WAF.
Redis cache
In a system with a high ratio of queries / writes, it's important to have a fast cache to:
It is used to cache shortURL --> longURL mappings once published
Stores entries with a TTL so they are expired when no used, because it's expected that URL queries diminish over time. The TTL is updated every time an URL is retrieved (adaptative cache TTL).
Also LRU eviction policy is used in case to many entries are created and memory is not enough
A sorted set is used to identify short URLs that will be published in the CDN when they exceed a threshold. In that moment they are deleted from the sorted set
CDN
For celebrity problem, wehn short URLs are published in social networks, a CDN is used to:
After the number of requests of a shortURL exceeds a threshold, for example 100, it's promoted to the CDN
Web Servers
For providing scalability and availability, several web servers handle requests received in the Load Balancer
The architecture is stateless to increas scalability and avoid operational problems
Explain any trade offs you have made and why you made certain tech choices...
MongoDB is used instead of a RDBMS for storing URLs because:
Only highly queried URLs are stored in the CDN to reduce costs, as the number of shortened URLs is very high
Redis cache uses LRU to evict not commonly used short URLs
The main bottlenecks could be in the getRedirection requests son a CDN and a Redis cache are used before querying the MongoDB database
The URLs database could also be a bottleneck, and a component difficult to scale reducing availability. MongoDB with eventual consitency is selected, with partitioning and data replication, to:
To avoid Single Point Of Failure:
To avoid Redis SPOF Redis Sentinel will be used for automatic failover