Requirements

Functional Requirements:

• Users can query current weather conditions (temperature, humidity, wind, etc.) by city, zip code, or exact coordinates.
• The system provides customizable weather forecasts (e.g., hourly, 7-day).
• The system sends push notifications or live data alerts for severe weather changes.

Non-Functional Requirements:

• High Availability (99.99%): Weather data is critical for daily planning and safety; downtime is unacceptable.
• Low Latency (< 200ms): Read operations must be incredibly fast to ensure a seamless user experience across web and mobile clients.
• Scalability: The system must seamlessly handle extreme, sudden traffic spikes (e.g., during a hurricane or blizzard).
• Read-Heavy Optimization: The read-to-write ratio is heavily skewed. Millions of users read data, while writes only occur when ingestion pipelines pull updates from meteorological sources.

API Design

We'll use RESTful APIs for standard queries and WebSocket/Server-Sent Events (SSE) for real-time live data updates.

• Get Current Weather:
• GET /v1/weather/current?lat={lat}&lon={lon}
• Response: 200 OK with JSON payload containing temperature, humidity, conditions, etc.
• Get Forecast:
• GET /v1/weather/forecast?city={city_name}&days={n}
• Response: 200 OK with a time-series array of forecasted conditions.
• Subscribe to Alerts:
• POST /v1/alerts/subscribe
• Payload: { "user_id": "string", "location_geohash": "string", "device_token": "string" }
• Response: 201 Created

High-Level Design

To achieve our low latency and high availability goals, we must decouple the read path from the write/ingestion path.

• Edge Layer: A CDN (like Cloudflare or Cloud CDN) caches API responses for common queries (e.g., "Weather in New York") at the edge, absorbing up to 80% of read traffic.
• API Gateway: Routes requests, handles rate limiting, and manages authentication.
• Read Path (Weather Service): A lightweight, horizontally scaled microservice that fetches weather data. It first checks a distributed in-memory cache (Redis). If there's a cache miss, it queries the underlying Time-Series Database.
• Write Path (Ingestion Service): Dedicated workers periodically fetch data from authoritative 3rd-party meteorological agencies (NOAA, OpenWeather).
• Event Bus: Ingested data is published to a high-throughput message queue (Apache Kafka or Google Cloud Pub/Sub).
• Processing & Storage: Consumer services process the raw Kafka streams, update the Time-Series DB, invalidate/update the Redis cache, and trigger the Notification Service if thresholds are breached.

Detailed Component Design

1. Data Ingestion & Caching Strategy (The Core)

• Weather data is inherently tied to locations. We will convert latitude and longitude into Geohashes (e.g., a precision of 5 characters covers roughly 5km x 5km).
• When the Ingestion Service pulls new data, it maps the coordinates to a Geohash and updates our Redis Cluster using the Geohash as the key: weather:{geohash}:current.
• We apply a Time-To-Live (TTL) of 10-15 minutes to these cache keys, ensuring data freshness while preventing our databases from being hammered by redundant queries.

2. Time-Series Database (TSDB)

• For historical data and forecasting models, a standard relational DB is insufficient. We need a TSDB like InfluxDB or Apache Cassandra.
• Cassandra is excellent here due to its wide-column store nature. We can partition the data by geohash and sort by timestamp, making range queries for a 7-day forecast highly efficient O(1) to locate the partition, O(log N) to fetch the time range.

3. Real-Time Alerting Pipeline

• Subscribers' preferences (Geohash locations they care about + device tokens) are stored in a highly consistent relational database (like PostgreSQL or Cloud Spanner).
• When the Ingestion Service detects a severe weather flag from a provider, it publishes an event to a severe_weather_alerts Kafka topic.
• An Alert Worker consumes this topic, queries the database for all users subscribed to the affected Geohash, and fans out the payload to a Notification Service (FCM/APNs) to deliver the push alerts asynchronously.