Loading...
-> The users can create easily a CDN providing its origin servers and define authentication and authorization restrictions.
-> The CDN servers are distributed across the glob serving content from most nearest geographical edge server.
-> The edge server will cache the content from origin server to be used for further requests.
-> Monitoring tool of the solution will dictate if the edge servers from a location should be scale up or down.
-> The CDN should authenticate and authorize the clients requests
-> Updating content at origin should allow the users to propagate and invalidate cache
-> The system should be highly available since are used by customers.
-> The system should be reliable and provide fast updates of the cache in the edge servers.
-> The system allows a eventual consistency model since the changes are propagated at the edges.
-> Reliability is one of the key non functional requirements since the system should be fault tolerant and monitored.
-> Security should be enforced so that only authenticated and authorized clients can retrieve the data.
Let's suppose that we have 100K customers with each having 5 origin servers each containing 1K items, each origin server holds 10Gb of data. ~10 MB per file.
Daily active users updating or creating these data are 100 K each doing 5 requests per day.
500 K requests of cache invalidation/day ~5 req/s.
Cache invalidation data are 100 bytes each holding information like: customerId (8), timestamp, path or regex expression (50). => 50Gb data daily.
10 million daily unique clients around the globe are requesting these data, each client do 100 requests with the probability of requesting 20% of the saved content -> 20% from a total of 500 000 000 items => 100 million items /day each item has 1MB => 100TB data/day stored in edge cache=> 1000 000 000 MB /day
with a bandwidth of 10GB/s.
100TB of items per day can be handled with 100 edge servers around the globe each handling 1TB of data.
-> POST /api/create-cdn with requests data
{
"customerId": uuid,
"cdn _name": string(100),
"regions": [string, string]
"HTTP_origin": link_url
}
with the response data, where resource_available represents the GET by Id url, the domain is the Front end domian.
{
"id" : uuid
"resource_available": link,
"domain" : string
}
-> GET /api/cache?server_id=uuid
returns a list of all resource cache in this server.
-> POST /api/cache/invalidate
{
"pattern": string,
"user_id: uuid,
"cdn_id": uuid -> the id of the CDN resource targeting.
} - response 200 OK.
-> PUT /api/content
{
"user_id": uuid,
"content_id": uuid,
"cdn_id": uuid
} -> 200 OK will invalidate the old cache for the content already cached
Metadata of the created cdn for customers can be stored in an SQL database
Index on content (content_id, region) to be able to easily spot the edge_server.
invalidated_cache pattern will be stored on the each edge server being propagated after publish an invalidated cache item.
For this kind of job I will choose multiple kind of databases: strong consistency for content metadata and where is cached, and Key-value : Redis for invalidate cache action.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?