Loading...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
consider traffic of 10 B /day
10 x 10^9 /( 24 x 3600 )users/s=10^7/(2.4x36)~ 100,000 qps
if flash sale, traffic would be 10 x general traffic = 10^6 qps
= 10Bx365 users/year = 3650 B users/year
Object size = 1 MB/req
Bandwidth req = 1 MB per req x 100000 req/s = 100000 MB/s = 100000 X 3600X24X365 MB/year = 10^7 x 36 x 24x365 MB/yr= 10^10 X 365 MB/yr = 3650 PB/yr
read/write ratio: 99.9%
update request: PUT /content payload: {
content
metadata
}
write request: POST /content payload: {
updated-content
updated-metadata
}
read request: GET /content
cache invalidation:
DELETE /content/cache
configure cache retention
PUT /content/cache -d {
ttl :
}
get req
client makes request,
request directed to dns
dns forwards req to cdn controller
cdn controller is the central manager for cdn edge servers , responsible for geography base request routing and traffic based scaling of edge servers
edge servers:
servers deployed in different geographic region, cotaining cached data
cdn controller will route the client request to right cdn server based on nearest geographic location, and other parameters like health of server, traffic already routing to the server etc.
load balancer further routes traffic for same geographic location edge servers based on set load balancing algo:
round robin
more resources
least traffic etc.
if the edge server contains the the data in cache , data returned from their to client
hence saving load on original server
if edge server doesn't contain data in cache or that cached data is expired, edge server pulls the content from original server, updates the cache based on different cache strategies and returns the requested content to client
put request:
on content modification request, the cached data is no more valid hence it needs to be invalidated so original server makes the api request to all edge servers to invalidate the cached data
post request
client makes post req, content is uploaded in s3, directly using the presigned urls returned by the original server, to avoid latency and bandwidth issues
metadata is stored in another db
original server can also set the ttl on cached data using the api defined, and also use other cache eviction strategies like LRU, LFU to ensure most relevant data is stored in cache while the garbage and invalidated data is cleared to save storage
req first being sent to edge servers rather that original server also saves it from direct security attacks
additional we can add rate limiter to further avoid ddos attacks
on top of this we can also have a separate analytic service setup for monitoring to analyze and recommend suggestions on best cache eviction strategies, geography based request routing etc.
DB choices:
for metadata storage mongo-db, because of unstructured data and fieslds depending on whether it is a file or image or a video
user details storage postgres db because the data to be stored is structured
massive amount of data to be stored: 3650 PB / yr
and in case of race conditions, it can handle requests well using atomic transactions in postgres db
for cache we use redis, for faster lookup and to be able to use redis provided feature to set ttl
moreover it is ideal for frequent content modification, invalidation and addition
postgres_db model:
user_id
geographic_data
redis:
key: content
ttl
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
CDN controller:
it is responsible for analysing the traffic and scaling up or down edge servers in different geographic locations
edge servers:
contain regionally relevant cache data
possible caching strategies:
write behind, write through
I would prefer writing the data to origin server first and then updating the cache, since write load will only be 0.1% and every latest added data need not be cache
cache eviction strategy: LFU, to reduce the read load on original server, apart from this ttl will be sent on each data being cached
caching also prevents origin server from becoming victim of security attacks by addressing req from edge server itself, as well as prevent origin server from getting overwhelmed with traffic
apart from this, even when original server is down, it would still help with addressing client req, without client even realizing about it.
original server addresses following requests:
add new data, modify existing data
apart from this it keeps track of modified data and ensures related cache data in each servers is invalidated to ensure strong consistency
idempotency on cache purge: in case cache invalidation req is retried, or tried on fresh data, that request is ignored, as the cache to be invalidated would be compared using unique ID, for which content hash sum or the file version can also be used
it also is reposible to address req to configure ttl on cache
Thundering herd scenerios: a cache data is requested which has already been cleared, flooding the users with requests, for this an another intermediate layer of cache will be used, so once one edge server req from origin server, intermiediate cache is updated, and other edge servers receive data from that intermediate cache instead of reahcing out to original server for same content
popular key:
by updating the cache, on first time get req is made for a content, we ensure that each edge server req for same content only once, intermediate cache can also be used to store more popular request data, identified by analytics service(pre-warming)
load balancer:
it can be based on round robin , or if the load on certain servers is higher than based on available resources in each edge server