Requirements

Functional Requirements:

• Users can query current weather conditions by city, zip code, or exact coordinates.
• The system provides customizable weather forecasts (e.g., hourly, 7-day).
• The system sends push notifications or live data alerts for severe weather changes.

Non-Functional Requirements:

• High Availability (99.99%): Weather data is critical; downtime is unacceptable.
• Low Latency (< 200ms): Read operations must be lightning-fast.
• Scalability: The system must handle extreme, sudden traffic spikes (e.g., during hurricanes).
• Fault Tolerance: The system must resiliently handle external vendor outages without cascading failures.
• Data Integrity: The system must guarantee the accuracy and freshness of the ingested meteorological data.

API Design

We'll use RESTful APIs for standard queries and WebSocket/Server-Sent Events (SSE) for real-time live data updates.

• Get Current Weather:
• GET /v1/weather/current?lat={lat}&lon={lon}
• Response: 200 OK with JSON payload containing temperature, humidity, conditions, etc.
• Get Forecast:
• GET /v1/weather/forecast?city={city_name}&days={n}
• Response: 200 OK with a time-series array of forecasted conditions.
• Subscribe to Alerts:
• POST /v1/alerts/subscribe
• Payload: { "user_id": "string", "location_geohash": "string", "device_token": "string" }
• Response: 201 Created

High-Level Design

To achieve our low latency, high availability, and newly added reliability goals, the architecture is decoupled into distinct read, write, and validation paths.

• Edge & API Layer: A CDN caches common API responses, while an API Gateway manages routing and rate limiting.
• Read Path (Weather Service): A heavily scaled microservice that prioritizes a distributed in-memory cache (Redis) before falling back to the Time-Series Database.
• Write & Validation Path (Ingestion Service): Dedicated workers pull data from 3rd-party meteorological agencies. This path now includes robust validation and circuit-breaking logic.
• Event Bus (Kafka): High-throughput message queue for streaming ingested data.
• Processing & Alerting: Consumer services update databases, refresh caches, and trigger notifications for severe weather.

Detailed Component Design

1. Resilience & Outage Mitigation (Circuit Breaker & Fallbacks)

• Circuit Breaker Pattern: We wrap our external API calls (to NOAA, OpenWeather) with a Circuit Breaker (e.g., Resilience4j). If a vendor experiences an outage or elevated latency, the circuit trips to "open" mode, instantly halting requests to prevent resource exhaustion and cascading failures across our system.
• Stale Cache Graceful Degradation: During a vendor outage, the Weather Service degrades gracefully. Instead of returning a 500 Internal Server Error, it serves a "stale cache" by temporarily extending the Redis TTL for affected Geohashes, ensuring users still see the last known good data.
• Queued Retries with Exponential Backoff: When external APIs fail, the Ingestion Service pushes the failed fetch task to a Dead Letter Queue (DLQ) or retry topic. The system will attempt to fetch the data again using exponential backoff until the vendor recovers.

2. Data Accuracy & Freshness

• Multi-Source Consensus: To prevent inaccurate reporting from a single faulty sensor or provider, the Ingestion Service fetches data from multiple authoritative sources.
• Anomaly Detection & Validation: Before raw data enters the main Kafka topic, it passes through a validation layer. If a temperature reading spikes unnaturally (e.g., reporting 150°F in winter), the system flags it as an anomaly, drops the payload, and relies on consensus data from other providers to maintain accuracy.

3. Data Storage & Caching Strategy

• Geohash Caching: Latitude and longitude are converted to Geohashes. Redis stores the current conditions using the Geohash as the key (weather:{geohash}:current) with a standard 10-minute TTL for freshness.
• Time-Series Database (TSDB): Apache Cassandra stores historical data and forecasts. Data is partitioned by Geohash and sorted by timestamp, making range queries highly efficient:  to locate the partition and  to fetch the time range.

4. Real-Time Alerting Pipeline

• Subscription Management: User alert preferences are stored in a highly consistent relational DB (PostgreSQL).
• Asynchronous Push: When the Stream Processor detects a severe weather event, it publishes to a dedicated Kafka topic. An Alert Worker fans out the payload to APNs/FCM for immediate mobile push delivery.