Loading...
System is more read heavy than write heavy. If a user creates a url, he will share the url with other people. So we have a unkown multiplication factor here. Daily Users I currently dont know, my system should be fast scalable on my launch or overscaled and then downscaled after more usage information.
Read/Write Ratio: 20/1
Capacity estimation is unkown. If the service gets hype, it will grow over time. This can be monitored and scaled on demand.
I need a REST API service that has the previous endpoints (Create Update Delete Redirect)
We should choose a scalable database technology, that maybe can scale out to multiple read replicas, one write master and multiple read replicas, since we have a higher read than write ratio. We could implement a cache if we want higher scale that has the most used/latest url redirections.
If we want real high scalability and low latency, we should consider a CDN with Read Replicas for fast Redirection. For a creation it is okay for the user to wait a few hundred miliseconds for creation, but since creations have no read dependencies, we could even replicate writes. If we add a update this will become harder, but we have a single user, so not too hard. There are no race conditions in this case.
Client -> Server
Server -> Cache
Server -> Database
The storage layer is the source of truth and holds all the data. We should consider backing up that data periodically. The data gets partitioned to multiple shards, most of the time the database decides the hash algortihm, in case we need to decise ourself, we should use a hash algorithm that distributes well and is fast.
If we need rate limiting, we could roll our custom solution, or we use a api gateway or cdn that does it already by ip or something.
The id generation should be collision free, so we should find a hash algorithm that has a very small collision rate at short hash length. For safety, we can ask the database at creation time if it has this id already, and if it does we reroll the id, with a dynamic part in the hash, like a timestamp or random id.
This should also give us non predictive tiny urls, which hardens security. We should include a secure random part in the hash.
The cache should be updated consistently, so we should have a small function that listens to a database change feed, and update the entries in the cache. This should give us fast and easy cache update and invalidation.
The cache will be filled on redirections, if a user gets redirected and we have a cache miss, we will retrieve the record from the db and inject it in the cache with a 5 min sliding window ttl to handle bursts.
For this the datamodel is simple.
The primary use case is redirection so:
Id(PrimaryKey): Id (the last part of the url)
OriginalUrl: OriginalUrlString
CreatedAt: Timestamp
We can add more metadata if we give the system more features, like management via a user:
User
AutodeleteTimestamp
...
The database type is not important. Here NoSQL is more natural since we have no "real" relations. There is possibility that multiple tinyurls will point to the same original url, but its not often. So for me NoSQL is fine.
I would pick a scalable NoSQL Database here. Partitioning could help here to help throughput, but there currently is no "good" partition key, since we dont have a "logical" partition.
One entry should be estimated at a 1-2 kilobytes max.
Storage Requirement Over Time: Lets expect 1.000.000 Entries for the start, so we start with 1.000.000*2kB = 2.000.000kB = 2.000 mB = 2 GB Storage, which is not expensive. We could implement monitoring strategies that warn when we have reached 80-90 Percent of the storage.
Database: Scales via Partitioning/Sharding. Choosing a Database which can have multiple read replicas or partition instances is helpful. For exmple Azure Cosmos DB or Cassandra DB
Cache: Caches redirected urls so if a burst of traffic comes we handle it gracefully.
Rest Api: Stateless, so we can just hit scale to multiple instances.