Loading...
should handle lots of data
will support many more reads than writes.
should track number of times each link is clicked
The system should reflect updates to tiny urls with in 30 seconds with a SLA of99.999%
The system should return redirect reads with in 10 milliseconds 99.999% of the time.
List non-functional requirements for the system...
Immeiate consistencey
don't need to plan for loosing an entire region.
1 trillion shortened urls
1 million read requests per second
1000 write requests per second
we can assuem each entry is
1024 bytes for ful url + 7 bytes for the tiney part + 4 bits for the counter data * 3 for teh tables = 3105 bytes per etnry
times 1 trillion entries is 3.1 * 10^15 bytes
or about 3 petabytes of space requried globally. if we have 10 regions,each would have 300 terabytes requred for space. However we also have a replication of 3,so that brings uback to 900 terabytes per region.
create alias will flow to the nearest region that the system is deployed to.
look up alias - will first hit the CDN, if there's a cache miss, it'll inspect the url and forward it to the region that own's the particualr tiny url.
delete alias - the CDN will route request to the deployment that owns the given tiny url.
update won't be supported
Casandra
Add a table
short_to_full
shortended url -> full length url, is_visable
full_to_short
full url-> short url
Read requests will flow from CloudFront into to casanda on cache misses. CloudFront will have a ttl of 1 day for cache entries. CloudFront will aggregate url hits and send them to the service in bunchs every 10 minutes. The service will save those to casandra.
This ensures that Casandra always anaytics data that's no more than 10 minutes out of date.
Casandra records have a ttl of 6 months. Updates, or analtyics dumps to the records refresh this ttl back to 6 months. So if a record is untouched for 6 months's it's delted.
On deltes, the service will reach out to cloudfront to delete the cached entry.
The service will be deployed to 3 AZs per rregion. This ensures that the data will be persisted if we loose up to 1 AZ at a time.
Internal system caching isn't necessary because the CDN will handle caching for us. Addingsomething like redis would add uncessary addtional complexity.
tiny urls will be randomly genrated. if a new url already exists in Casandra, the system will generate a new url and try the process again. once sucessfully, it will return the new url to the user.
the url will be 8 base 64 charaters long. This supports over 2 trillion entries.
Each region will own a slice of the address space to hand out urls for. This ensure that no cross region coordiaation is necessary.
I'd deploy the app to sevral regions around the globe. Each region would own a subset of the url address space. I'd have 3 replicas for each entry. I'd do quorum writes the 3 nodes on updates, and I'd do single node reads on read reqeusts.
A no sql db is harder to work with than using a relational db, but it will scale much better. the CDN will be out of sync with Casandra for a maxof a few minteus.
If we used somthying like mysql instead, we'd have some issues. Fist of all, mysql table stop working well once they get in to the millions of entires. Reads would be come slower and slwoer. We'd also run into backup issues. While we could asynchronlsy backup the writes, it's possible fresh datra woudl be lost.
Casandra, on the other hand, has a feature that ensrues writes are refelcted everywhere or no where at all.
The downside with some thign liek casandra is that multilple indexes dnt' work well, so we have to replicate the data in 2 tablesw and keep them in sync with our application cdoe.
A deleted entry will remain live in the CDN for a few minutes as the cache purge is carried out.
If an AZ is lost, we'd receate a new dpelloyemtn. Casandr should automaticlaly repair it'sefl assume only one AZ is lost.
I'd partition the services globally to own a differetn range of the url space that it can give out new urls for. this would ensure that a given AZ would be able to independntyly decide if a url is free without contacting other AZs.