Loading...
Assuming 200:1 read/write ratio
Number of unique shortened links generated per month = 100 million
100,000,000 / 31 = 3,225,806 per day
3,225,806 / 24 * 60 * 60 = 37.33 RPS.
But a peak load might be 3,225,806 in one second, as per BOTEC.
So assuming one server can handle 64000 RPS, 3,225,806 / 64000 = 505 - servers we need to process the write requests.
505 * 200 = 101,000 - server we need to process read requests.
Storage estimation:
Let's say our service will live for 100 years.
We have 100 million write request per month.
100,000,000 * 12 * 100 * 500(bytes per link) = 70TB for a 100 year period.
generateLink(longlink) - generates the short link
getLink(tinyLink) - returns a long link for a short one
We will use a key-value store for storing a short link as a key, and a long link as a value.
Metadata can be stored in a separate relational database to decouple the write/read requests from metadata.
User inputs the long url, it goes through the rate limiter, checks if the user hasnt reached request limit, the the request is passed to the Load balancer, which will pass the request to the appropriate application server, application server will update the key value storage, and rdb.
when users inputs tiny url into the browser it goes through the same way until it reaches the application server, which ffirst check if cache has the key, if not it asks key value store for the link, and updates the cache. after this the user is redirected to long url.
As the application servers are stateless, we dont can just use a cluster of servers which will be operating the user requests.
To scale well we will need our Key Value storage to be sharded, as it is NoSQL DB, it is quite easy to scale horizontally, we will have a cluster of Key Value storages which will be using consistent hashing to know where the links are located.
Also to avoid SPOF and provide high availability, we will replicate the shards, and we will use async replication, because the ratio of read write is 200:1.
In case of data inconsistency due to async replication:
1) during reads from multiple replicas:
we will resolve it by taking the newer one by looking at the update time.
2) it is not crucial if we serve a slightly outdated data to the users if replication hasn't happened on one of the replicas
We will also use the async replication for metadata, and sharding if the number of users increases substantially .
Distributed cache will be using LRU system, like Memcached, it suits our system, because many users will use short links at some short period of time.
We could use async replication to provide even faster write, but we would sacrifice the consistency which is one of our non functional requirements.
We could use the RDB instead of Key Value Store, but we would sacrifice the update and retrieval speed. We would get the ACID though.
In case of too many write requests, due to sync replication of Key Value Store, the read requests will have delay, which would affect our availability.
Also additional sync replication to RDB will affect our tiny url generation, because it will require users to wait until replication is done.