Requirements

Functional Requirements:

• Allow users to query current weather conditions (temperature, humidity, wind, etc.) for a specific city, coordinates, or region.
• Provide weather forecasts for upcoming days (e.g., 7-day or hourly forecast) so users can plan ahead.
• Enable users to look up weather by city name, zip code, or latitude/longitude coordinates.
• Push notifications, alerts, or live data updates to users when weather changes or severe events occur.

Non-Functional Requirements:

• Since we are developing push notifications, alerts, it seems this is more about hosting a service for everyday people as opposed to a computationally expensive back load problem to try to predict weather as accurately as possible for news our other specialties.
• Scalability, we want to be able to serve a large number of users
• Availability & latency > consistency. Weather predictions can change constantly, but its ok if the forecast is not the most up to date version
• Feasibility. There are lots of apps that can do this, this is a market where where the margins are driven small by competition
• Performance: the faster we can serve users the better
• This is also going to be a system where reads happen much more than writes. Writes are fixed rather than variable. Writes will also be simple while reads are variable

API Design

• get_current_conditions: users must specify either city, zip code or coordinates
• get_forecast: users must specify the time range & city, zip code or coordinates
• scan_event: this can either be polling or server sent events, but must have a way to send that a weather change has occurred. Key is to know what the location the user is interested in, either we can monitor their current location or store the 10 most recent locations they queried in the past 10 days

High-Level Design

1. Bottom up, we first need to either recieve or compute both current weather conditions & forecasts across our area of operation. For simplicity i'll assume we recieve a stream of events that have weather conditions, coordinate lcoations & a start time they will be relevant for. These will be put operations, where if a more recent forecasting is put then that
2. We have an api service that translates users requests into sql queries. This has to: a. translate areas into spans of geographic coordinates (i.e. what coordinates do we need to check weather for if user asks for Miami's weather) and needs to calculate forecasts to time ranges.
   1. We should not assume that we get pretty data where we have a weather forecast for every hour for the time ranges users request. We will have some of the forecast data we want & will be missing others. I.E. when user asks for time from 6-10am we might have 6, 6:30 and 8:45.
   2. Really we need two services: a Reader that translates the users request into an efficient query that fetches all data we need
   3. An analyzer, that uses the fetched data to return something meaningful. we may have some algorithm, or a family of algorithms, for how to reconcile all the forecasting data
   4. an api service to accept the user's request
   5. a caching layer fro the 20 & of queries that take up 80% of volume. I.E. Miami weather will be asked a lot more than less popular cities. also during inclement weather conditions this will be
3. We have a client web or app that surfaces this in a pretty way

// Implementing notifications

1. When events come in, we should have a service that checks for large deviations from previous expectations or inclement weather. When it notices these, it then sends that to a notification service that sends server sent events to the needed users.
2. i would make this service read the database after the events have already been written to, since we don't need these to be low latency. Its ok if this is delayed, we don't want to delay how soon the database gets updated for rare events

Detailed Component Design

Database

The database can likely be a simple key-value database like dynamo db. Weather events do not have sql like relationships or tree shaped objects of document databases. This helps us optimize for performance too. We should allow sorting by timestamp & location. Additionall, weather events should have a TTL, perhaps of a year. There's no need for us to store these long term.

User data will not have a TTL. Arugably users could be in a different database because of their different shape, but for feasability it will be easier to have one data layer solution. The key things to store for user beyond standard personal & security info will be to track the locations they care about for notification purposes. We may ask users to tell us directly, store their most recent location queries, or track their current/recent locations.

This database will not need to scale writes unless we cahnge how we fetch data or add new regions to our service. We should design instead to heavily optimize for reads. This will involve denormalizing data, caching & a focus on indices. We can scale horizontally since its a key-value db

Reader

A key thing is that we do not want the database to care about the different options a user can select a location. This app could evolve to add an increasing number of options such as: county, city, state, region, zipcodes, etc. So the reader needs to translate from those inputs to the raw longtitude-latitdude coordinates that we care about. The reader should also have a cache system, as users will likely ask for similar coordinate info in slightly different ways. I.E. calculating Miami Dade county's weather uses the same data as calculating Miami's weather. This service will focus on query optimization, caching and I/O operations. Once it fetches the data it will send them to a seperate anaylzer that will compute the data.

The reader will want to be optimized for I/O networking operations, while the analyzer is optimized for compute internsive operations (multi-threading)

Weather event listener

We could implement this a few ways. It could read from our weather service as the same time as our database, read before the database (if we wanted to optimize to be the first ones to give you alerts), could recieve events from our database, or could scan our database for the latest changes.

Of these I prefer scanning our database. 1. it is simple, it only has to look at the updated_at field on weather_data rows and the database will already be read optimized. And it ensures that we only alert on events that are in our database. It would be embarassing for users to recieve a storm notice before our app shows (its storming) data. I also do not think that seconds matter for an operation like this. The weather event listener will scan the latest updates and do the following logic:

a. is this indicating severe weather we have not already notified about.

b. if this a large deviation from what we previously predicted this weather to be?

To answer question a we will likely want an additional table in the database indicating the broadstrokes of weather alerts we've already done. This is tricky to get right, as we don't want to respam alerts when a 3 day storm is going to last 1 additional hour. but we do want to send alerts if its going to last an additional day or hit a new city. We will likely have to track the events we've alerted on by the range of coordinates we've given an alert to and the time thresholds we don't want to alert for. I.E. for a 3 day storm in Miami, we may have our threshold for notification be if the storm is for one fewer or one additional day.

We may also want tiers of notifications: emergency, warning etc.

Answering b is tricky as well as we need to set thresholds for what is a large deviation and decide whether we compare it to the last recorded event, or the record of a certain time. If the weather gained 3 degrees 10 times today, then each deviation is small but a user that checked the forecast yesterday would see a 30 degree difference!