Detailed component design
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
- Client submits API request to load balancer (AWS Network Load Balancer due to high traffic of reads/writes) which utilizes a weighted distribution policy among servers (due to some areas having more traffic than others i.e. West/East coast US vs Central US) then directs requests to API Gateway (AWS API Gateway) after verifying with a security service the given link is not malicious or corrupted.
- Write requests (POST) utilize a unique key assignment service with a one to many relationship so that each short_url generated is unique and there is no collision. During periods of high stress we have a queue set up through Kafka (pull-based model so not to overload servers at the cost of some potential latency). Writes are then uploaded to a Postgres database, we again use a relational database to enforce consistency through ACID properties
- Redirect requests (GET) are directed to a read queue due to the amount of excessive reads we have and utilize a pull-based queue to not overload server set up through Kafka. We then check the read cache which is housed in Memcached (We use memcached over redis due to the simplicity of storing only long and short urls) to see if requested data is available in cache, if not present we search the postgres database then return the data back to the client after saving the data to the cache which uses a LRU eviction policy (We choose LRU over LFU because users are more likely to use tinyurls from recently created links). We also push to the CDN so the client and other clients closer to them can access the data more readily instead of making a new read request. We can also employ the use of sticky sessions on the client side for when they want to access recently created tinyurls they have created through the use of cookies.
- For our database, we are also using synchronous replication to ensure consistency at the cost of a bit of latency. This is important because we want to ensuring there's no collision between two short url keys.
Database design
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
- For our database design - in order to ensure consistency we can utilize a SQL database such as Postgres which will aide in ensuring consistency through its ACID properties.
- Data is also not excessive to the point where we need to consider NoSQL options (~ 1.5 TB in 5 years)
- Use one to many hashing function for long_urls -> short_urls since multiple short_urls could be created from the same long_url but each short_url should be unique.
URL Table
Schema:
short_url (varchar) - key
long_url (varchar)
user_id (varchar)
created_at (datetime)