System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

show weather for specific location.
handling of realtime weather update.
support for multi-location and region in show weather.

Non-Functional:

List non-functional requirements for the system...

Availability
scalability
usability
quick response

Capacity estimation

Estimate the scale of the system you are going to design...

Number of user : 100000 peak hours

API Request Rate: 5 request per session

Data Size: 5KB /response

Estimation Snapshot:

Peak Hour Requests:
- 100,000 users x 5 requests/session = 500,000 requests during peak.
Data Transfer during Peak:
- 500,000 requests x 5KB = 2.5GB.
Database Operations:
- 400,000 read operations
- 100,000 write operations
Yearly Storage Requirement:
- 365GB

API design

Define what APIs are expected from the system...

1). fetch weather.

api-/weather/fetch

method-GET

query:

lat:-

long:-

response

{
   "coord": {
      "lon": 7.367,
      "lat": 45.133
   },
   "weather": [
      {
         "id": 501,
         "main": "Rain",
         "description": "moderate rain",
         "icon": "10d"
      }
   ],
   "base": "stations",
   "main": {
      "temp": 284.2,
      "feels_like": 282.93,
      "temp_min": 283.06,
      "temp_max": 286.82,
      "pressure": 1021,
      "humidity": 60,
      "sea_level": 1021,
      "grnd_level": 910
   },
   "visibility": 10000,
   "wind": {
      "speed": 4.09,
      "deg": 121,
      "gust": 3.47
   },
   "rain": {
      "1h": 2.73
   },
   "clouds": {
      "all": 83
   },
   "dt": 1726660758,
   "sys": {
      "type": 1,
      "id": 6736,
      "country": "IT",
      "sunrise": 1726636384,
      "sunset": 1726680975
   },
   "timezone": 7200,
   "id": 3165523,
   "name": "Province of Turin",
   "cod": 200
}

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

Here, we are going to use a NoSQL database that will store vendor API or webhook responses. The data will be stored in the database with latitude, longitude, and a timestamp for historical retrieval.

data model

{
"_id": "5553a998e4b02cf7151190c9", // MongoDB's unique ID for the document
"location": {
"city": "New York City",
"state": "NY",
"country": "USA",
"latitude": 40.7128,
"longitude": -74.0060
},
"current_weather": {
"timestamp": "2025-09-06T11:00:00Z",
"temperature": 20.5,
"humidity": 65,
"wind_speed": 15,
"wind_direction": "NW",
"condition": "Partly Cloudy",
"icon_code": "02d"
},
"daily_forecasts": [
{
"date": "2025-09-07",
"temp_max": 22,
"temp_min": 15,
"condition": "Rainy",
"icon_code": "09d"
},
{
"date": "2025-09-08",
"temp_max": 25,
"temp_min": 17,
"condition": "Sunny",
"icon_code": "01d"
},
// ... up to 7 days
],
"hourly_forecasts": [
{
"timestamp": "2025-09-06T12:00:00Z",
"temperature": 21.0,
"condition": "Cloudy",
"icon_code": "03d"
},
{
"timestamp": "2025-09-06T13:00:00Z",
"temperature": 21.5,
"condition": "Cloudy",
"icon_code": "03d"
},
// ... up to 24-48 hours
],
"historical_data": [
{
"timestamp": "2025-09-05T11:00:00Z",
"temperature": 18.2,
"humidity": 60,
"condition": "Cloudy"
},
{
"timestamp": "2025-09-05T10:00:00Z",
"temperature": 17.5,
"humidity": 58,
"condition": "Cloudy"
},
// ... many more entries
]
}

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

In high-level design, we are going to create a weather service that is responsible for orchestrating the flow. The weather service will call the third-party service, and then the third party will call the vendor for data. The weather service will then put the data into Redis and a database. Redis will be used for quick responses to recent weather pulls based on latitude and longitude keys, while the database will contain all historical data. Based on vendor webhooks, the weather service will publish events to Kafka, and then the notification system will send real-time bad weather forecasts to users.

flowchart TD
A[Client] --> |Weather Request| B[API Gateway]
B --> |Forward Request| C[Weather Service]
C --> |Cache Weather Data| D[Redis Cache]
C --> |Fetch Data| E[Third Party Service]
E --> |Get Weather| F[Weather Data Provider]
F --> |Weather Response| E
E --> |Send Data| C
C --> |Store Historical Data| G[NoSQL Database]
C --> |Publish Updates| H[Kafka]
H --> |Handle Notifications| I[Notification System]
I --> |Send Real-Time Alerts| A

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Client Request: The client initiates a request with latitude and longitude, which is routed through an API Gateway to the weather service.

Orchestration: The weather service orchestrates the flow by calling a third-party service to fetch weather data from a vendor.

Data Storage & Response: The weather service stores the fetched data in both a NoSQL database for historical records and Redis for fast, recent data retrieval before responding to the client.

sequenceDiagram
client->>+apiGateway: 1. Initiate call with lat long
apiGateway->>weather_service: 2. Forward call to weather service
weather_service->>+thirdparty: 3. Request weather data
thirdparty->>+weather_data_provider: 4. Fetch weather data
weather_data_provider->>+thirdparty: 5. Return weather data
thirdparty->>+weather_service: 6. Pass weather data
weather_service-->>+NOSQL: 7. Store data for historical retrieval
weather_service -->>+redis: 8. Cache data for quick access
weather_service ->>+kafka: 9. Publish updates
weather_service -->>|Websockets| client: 10. Real-time data delivery

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

1.Weather-Service

This service's main responsibility is to orchestrate weather reporting functionalities.

Caching: The response from the API is stored in Redis with a Time-to-Live (TTL) to reduce vendor API calls and support cache invalidation.
Real-time Data Publishing: It's responsible for publishing events to Kafka for real-time processing and notifications.
Scalability via Kubernetes: Weather-Service is containerized and uses Kubernetes for auto-scaling, handling varying loads efficiently.
Outage Handling: In vendor outages, if data is in Redis, it returns cached data. If not, it queues requests for later processing.
Cache Check:When a request for weather data is received, the service first checks Redis cache to see if data already exists.
Vendor API Call: If data is not found, the service calls the third-party vendor API.

2. Notification-Service

Responsibilities and Workflow

Subscribe to Kafka: Listens to Kafka topics published by Weather-Service for weather updates.
Message Processing: Processes messages to generate real-time notifications.
Notification Dispatch: Utilizes third-party services like Firebase, SMS gateways, and email services to send alerts to users.
Scalability: Can leverage Kubernetes or container orchestration frameworks to scale based on message load.

3. Third-Party Service

Responsibilities and Workflow

Vendor Integration: Handles API integrations, making the system vendor-agnostic.
Factory Design Pattern: Utilizes this pattern to dynamically select the appropriate vendor implementation at runtime.
Data Transformation: Converts vendor-specific data into a standardized format used internally.
Circuit Breaker Implementation: Ensures system reliability by preventing cascading failures from vendor outages.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Notification Systems: Polling vs. Push

Polling:
- Pros: Simpler to implement, no need for maintaining a connection.
- Cons: Higher latency, inefficient on resources.
Push Notifications (e.g., Websockets, Firebase Cloud Messaging):
- Pros: Real-time updates, efficient use of resources.
- Cons: More complex infrastructure.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

when our Thirdparty service providers have down time and we need to provide sla similar to them to our client.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

Solutions

galaxy2351 - 03/20/2024

0

nebula_jubilee499 - 06/15/2024

0

celestial_lotus529 - 09/06/2025

0