Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
POST /api/v1/subscribe
POST /api/v1/deliver
GET /api/v1/status
POST /api/v1/schedule
POST /api/v1/metrics
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
API Gateway: To route traffic based on the endpoint getting hit. Used for authentication and for rate limiting the requests.
Load Balancer: To balance the load received so that high availability and reliability is achieved.
Kafka: It is used as a high throughput queue to store the requests that are received. With this even if there is a down time it is easy to recover as kafka will have the missed requests. User will receive 200 OK immediately in case of failure the request is enqueued safely.
Webhook Service: Will have the get, create, update, delete and get events apis.
Metrics: Will record all the metrics both in terms of traffic and for infrastructure.
Status: Sends out status based on the output received from the webhook apis.
Postgres - persistent storage to have all the data of the invocations, statuses and events that are taking place. Every invocation will have idempotency key to avoid redundant runs. We will be sharding the DB to avoid hot keys.
Delivery Service: Delivers the notification to the respective subsriber based on the sorted set in redis
Scheduler service: Used to schedule the notifications
Subscribe service: Used to subscribe to the topics to receive notifications
Redis: To cache the requests and take care of concurrent redundant requests and reduce load on the db and response time. and to store sorted sets so that delivery is fast.
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
Kafka: It is used as a high throughput queue to store the requests that are received. With this even if there is a down time it is easy to recover as kafka will have the missed requests. User will receive 200 OK immediately in case of failure the request is enqueued safely.
The messages will be partitioned based on userID to let it scale horizontally. We can have priority queues as well so that important alerts like OTPs and security alerts are let out first.
We need to push to respective adapters based on the device type token received.
Why go with this instead of just a standard message queue. Using this we can have a graceful degradation since it can enqueue requests unlike message queue which cannot.
Postgres: persistent storage to have all the data of the invocations, statuses and events that are taking place. Every invocation will have idempotency key to avoid redundant runs. We will be sharding the DB to avoid hot keys.
We will be using exponential backoff for retries to handle the failed events. In case the max retries are used up we will send it DLQ so that it can be investigated.
When a cache miss happens we will be reading directly from the DB and then cache the entry into redis so that subsequent requests hit the cache.