Requirements

Functional Requirements:

• Store and serve frequently accessed web content from cache.
• Support cache invalidation when origin data changes.
• Evict entries when cache reaches capacity using a replacement policy.
• cached content stays consistent with the original data source.

Non-Functional Requirements:

• Low Latency response
• High Availability 99.99%
• Horizontal Scalability
• Strong consistency of data by following methods like LRU or LFU to evict the old data once the TTL is reached.

API Design

/api/v1/InsertCacheEntry

/api/v1/RemoveCacheEntry

/api/v1/GetCacheEntry

/api/v1/checkTTL

/api/v1/invalidateCache

/api/v1/GetStatistics

/api/v1/configure - helps configure cache size, eviction policy and ttl.

High-Level Design

We are going to use Redis as the main DB. And postgres as the metadata store.

Read Path: Client -> CDN -> API Gateway -> Load Balancer -> Cache Management -> cache proxy -> Redis -> Postgres

Write Path: Client -> CDN -> API Gateway -> Load Balancer -> Cache Management -> cache proxy -> Redis -> Postgres

If cache node fails the requests directly go to the actual db and after the cache is up it will start hitting the cache there by having graceful degradation.

we have a cache invalidation service which invalidates the cache keeping the cache fresh. It will use LRU approach and write through approach.

since we are using cdn they will be cached at the edge nearest to their location.

updates to the cache are taken care off by cache management service which will update the cache after they receive the invalidation is done.

Detailed Component Design

CDN - We host our frontend in this. I will be able to handle millions of request and is available on edge. It even acts as the first layer of cache where most of the read requests are taken care off. Also we can integrate our object store with this to serve the content files easily. this can help if external cache is either slow or unavailable.

API Gateway routes the traffic based on the api that is getting hit. This also helps in rate limiting/throttling of requests which prevents any DDoS and Brute force attacks.

Load Balancer this is configured with API gateway and is used to distribute the load across the servers hosting our services. Keeping the traffic uniform and capable of handling the spike in traffic.

we have a cache invalidation service which invalidates the cache keeping the cache fresh. They will be actively purged.

key redistribution will be done when scaling of the cache nodes using consistent hashing method.