System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

1. show weather for specific location.
2. handling of realtime weather update.
3. support for multi-location and region in show weather.

Non-Functional:

List non-functional requirements for the system...

1. Availability
2. scalability
3. usability
4. quick response

API design

Define what APIs are expected from the system...

1). fetch weather.

api-/weather/fetch

method-GET

query:

lat:-

long:-

response

3) High-Level Design

Database design

Here, we are going to use a NoSQL database that will store vendor API or webhook responses. The data will be stored in the database with latitude, longitude, and a timestamp for historical retrieval.

data model

In high-level design, we are going to create a weather service that is responsible for orchestrating the flow. The weather service will call the third-party service, and then the third party will call the vendor for data. The weather service will then put the data into Redis and a database. Redis will be used for quick responses to recent weather pulls based on latitude and longitude keys, while the database will contain all historical data. Based on vendor webhooks, the weather service will publish events to Kafka, and then the notification system will send real-time bad weather forecasts to users.

flowchart TD

A[Client] --> |Weather Request| B[API Gateway]

B --> |Forward Request| C[Weather Service]

C --> |Cache Weather Data| D[Redis Cache]

C --> |Fetch Data| E[Third Party Service]

E --> |Get Weather| F[Weather Data Provider]

F --> |Weather Response| E

E --> |Send Data| C

C --> |Store Historical Data| G[NoSQL Database]

C --> |Publish Updates| H[Kafka]

H --> |Handle Notifications| I[Notification System]

I --> |Send Real-Time Alerts| A

Request flows

Client Request: The client initiates a request with latitude and longitude, which is routed through an API Gateway to the weather service.

Orchestration: The weather service orchestrates the flow by calling a third-party service to fetch weather data from a vendor.

Data Storage & Response: The weather service stores the fetched data in both a NoSQL database for historical records and Redis for fast, recent data retrieval before responding to the client.

4) Detailed component deep dives (3 components)

1.Weather-Service

This service's main responsibility is to orchestrate weather reporting functionalities.

• Caching: The response from the API is stored in Redis with a Time-to-Live (TTL) to reduce vendor API calls and support cache invalidation.
• Real-time Data Publishing: It's responsible for publishing events to Kafka for real-time processing and notifications.
• Scalability via Kubernetes: Weather-Service is containerized and uses Kubernetes for auto-scaling, handling varying loads efficiently.
• Outage Handling: In vendor outages, if data is in Redis, it returns cached data. If not, it queues requests for later processing.
• Cache Check:When a request for weather data is received, the service first checks Redis cache to see if data already exists.
• Vendor API Call: If data is not found, the service calls the third-party vendor API.

2. Notification-Service

Responsibilities and Workflow

• Subscribe to Kafka: Listens to Kafka topics published by Weather-Service for weather updates.
• Message Processing: Processes messages to generate real-time notifications.
• Notification Dispatch: Utilizes third-party services like Firebase, SMS gateways, and email services to send alerts to users.
• Scalability: Can leverage Kubernetes or container orchestration frameworks to scale based on message load.

flowchart TD F[Kafka Topic] -->|Subscribe| G[Notification Processor] G -->|Send Alert| H[Firebase] G -->|Send Alert| I[SMS Gateway] G -->|Send Alert| J[Email Service] subgraph Notification Service G H I J end

3. Third-Party Service

Responsibilities and Workflow

• Vendor Integration: Handles API integrations, making the system vendor-agnostic.
• Factory Design Pattern: Utilizes this pattern to dynamically select the appropriate vendor implementation at runtime.
• Data Transformation: Converts vendor-specific data into a standardized format used internally.
• Circuit Breaker Implementation: Ensures system reliability by preventing cascading failures from vendor outages.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Notification Systems: Polling vs. Push

• Polling:
  • Pros: Simpler to implement, no need for maintaining a connection.
  • Cons: Higher latency, inefficient on resources.
• Push Notifications (e.g., Websockets, Firebase Cloud Messaging):
  • Pros: Real-time updates, efficient use of resources.
  • Cons: More complex infrastructure.

Failure scenarios/bottlenecks

when our Thirdparty service providers have down time and we need to provide ssls simility to them to our client.