Requirements

Functional Requirements:

• User chooses a location to get the weather information (Temperature, Humidity, Chance of Raining, Sunny or not, ...)
• User to be notified immediately if there is a crisis (Storm, Hurricane, High-Wind speed, ...)

Non-Functional Requirements:

• Available and Partition Tolerant application (Not necessary the consistency here, since the information can be delayed for seconds or something)
• 1 Million user using the App daily (Like Google Weather or Apple Weather) -> ~12 requests per second

API Design

Endpoints

POST /v1/subscribe/{city}

• Purpose: Subscribe a user to receive weather notifications for a specific city.
• Request Body:

• Responses:
  • 201 Created → subscription successful
  • 400 Bad Request → invalid input
  • 409 Conflict → subscription already exists

GET /v1/weather/{city}/{dayOfTheWeek}

• Purpose: Retrieve weather forecast for a city on a specific day.
• Responses:

• Error Codes:
  • 404 Not Found → city or day not found
  • 500 Internal Server Error → service failure

Benefits:

• Makes implementation easier for clients.
• Provides clarity on expected data structures and error handling.

2. Consideration of Edge Cases

1. Ambiguous or Conflicting Weather Data
   • If multiple providers provide differing forecasts for the same city and day:
     • Use weighted averaging for numeric metrics (temperature, humidity, wind).
     • For hazard detection, adopt max severity rule (e.g., if one provider reports HIGH_WIND, treat it as urgent).
2. Duplicate Alerts
   • Each forecast/event has a unique event_id.
   • Deduplicate in Redis with TTL to prevent sending multiple notifications for the same event.
3. Missed or Delayed Updates
   • If a subscriber misses a push (device offline), maintain recent events buffer in Redis (last 24h) to allow delivery upon reconnection.
4. High Subscriber Volume in a Hot City
   • Shard subscribers to multiple queues to prevent a single queue from becoming a bottleneck.
   • Batch notifications to push providers respecting rate limits.

High-Level Design

Streaming/Messaging is going to be used.

• RabbitMQ as the messaging broker (Featuring the Priority Queue feature to prioritise dangerous events
• A Simple Datastore of any kind to store the user's subscriptions of a city
• Whenever a data comes from the service responsible of gathering the weather forecasts, RabbitMQ subscribes it and fanout the necessary data to the intended customers
• Whenever data comes, it is stored and persisted on Redis to serve the second API (GET v1/weather/{city}/{dayOfTheWeek}
  • keys will be city::day

Detailed Component Design

1. Weather Subscription Service

• Purpose: Store user subscriptions per city (POST /v1/subscribe/{city}) for fast lookup.
• Scaling: Partition by city, cache hot cities in Redis, horizontally scalable.
• Tradeoffs: NoSQL favors availability/partition tolerance; SQL needs sharding for large cities.

2. RabbitMQ (Messaging Broker)

• Purpose: Stream weather events to delivery workers; prioritize crisis alerts.
• Scaling: Shard queues by city or hash; multiple consumers per shard; clustered for HA.
• Tradeoffs: RabbitMQ supports message priority; Kafka provides higher throughput but no native priority.

3. Weather Forecast Service / Ingestion

• Purpose: Normalize provider data, detect hazards, publish to RabbitMQ, update Redis.
• Scaling: Stateless, horizontally scalable; deduplicate events in Redis.
• Tradeoffs: Simple per-event hazard rules vs. complex multi-day storm tracking requiring state.

4. Redis Cache

• Purpose: Serve fast GET API responses and cache subscriber lists.
• Scaling: Redis cluster with TTL-based eviction for freshness; shard large sets.
• Tradeoffs: Fast reads with eventual consistency; memory limits require careful sharding.

5. Delivery Service (Airship / FCM / APNs)

• Purpose: Push notifications to subscribed users, prioritizing urgent events.
• Scaling: Horizontally scalable, batch notifications to push provider limits.
• Tradeoffs: Push provider rate limits; urgent events handled first via priority queues.

Flow Summary:

1. User subscribes → stored in DB + Redis.
2. Provider sends data → Ingestion normalizes + hazard detection → RabbitMQ + Redis.
3. Delivery workers fetch subscribers → send notifications.
4. GET API reads forecast from Redis.