Requirements

Functional Requirements:

• Allow users to query current weather conditions (temperature, humidity, wind, etc.) for a specific city, coordinates, or region.
• Provide weather forecasts for upcoming days (e.g., 7-day or hourly forecast) so users can plan ahead.
• Enable users to look up weather by city name, zip code, or latitude/longitude coordinates.
• Push notifications, alerts, or live data updates to users when weather changes or severe events occur.

Non-Functional Requirements:

• High availability
• High scalability
• Low latency
• Data duability

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

• QueryWeatherCondition(location, timeframe, granularity), it allows user to query weather condition in a given timeframe using a given granularity. Timeframe can be 3 days, 7 days. (We need a max value apparently). Granularity is the form of data, in hourly or daily. The response is a slice of result, each struct has a start time, end time and one field indicating the weather condition
• Converted() this coverts user input to location, as user can input zipcode, city name or lat and long.
• Push() this is a push notification to users
• Subscribe(userId, locationId), subcribes a user to a location.

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Entry:

• Load balancer: load balance traffic
• API Gateway: auth, throttling, rate limiting

Service:

• location service that coverts input to location
• weather service that returns weather condition
• event queue that queues push event
• push service that pushes notifications to users
• Data aggregation that aggregates data to a certain granularity. For instance, if a user requests response in daily format, but raw data is in hourly. We need to aggregate data.
• Data analytics for providing analysis tasks.

Storage:

• RDB that stores all weather data
• CDN that stores hot weather information
• Cache like Redis to reduce traffic to the main RDB. LRU might be a better eviction strategy as users don't like searching old weather data, LRU should help evict old data more. Each cache should have a TTL, can be hourly or even daily.

HA:

• Multiple replica on services and middlewares
• Multiple AZs
• RDB should use read-write replica
• Queues and other middlewares should persist data

Scalability:

• Most services should be stateless, location service, weather service, push service
• Auto-scaling across AZs

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

1. Weather service is the core of the design. However, it could be done. We can build a circuit breaker aound it, it has three states, closed, open and half-open. Closed means circuit breaker is not open, traffic is normal. Open means traffic is stopped, Half-open means only a subset of traffic can continue. This would help us to prevent cascading failures as applications won't retry when breaker is open.
2. Idepotent writes. Concurrent writes could create duplicate records. At the database level, we need to implement UNIQUE constraint, it is the safety. Most of the use cases in this scenario is read which should be safe. The write scenarios seem to only involve subscription endpoint, which should be enough to use the UNIQUE constraint.
3. We can also cache data for popular location ID to avoid overwelming weather service.
4. CDN, Cache and RDB serve data to the application. CDN can be deployed at the nearest locations and stores the hottest weather data. For instance, New York City in the next 7 days. If a cache miss, we got to Cache. Many cache services can be used, like Redis. We can configure a TTL to hourly or daily depending on the use case or cities. An LRU eviciton plan should be better as users wouldn't access old weather data most of the time. Cache needs to be deployed in clusters, for instance a Redis cluster, data needs to be persisted, and syn betwee nodes need to be handled. RDB needs to have read-write replicas, data needs to be persisted. Data sharding should also be implemented as data grows. Consistent hashing is a good technique for this task, it makes horizontal scaling easily handled. Virtual nodes on top of that can minimize distruptions on adding or removing nodes.
5. When a cache miss occures, it goes to read from RDB, then write back to cache. When an update event occurs, RDB pushes an event to Cache that invalidates the data entry.
6. Location service, Weather service are stateless services, they are deployed in clusters and load balanced. They should be easily scaled across AZs.
7. Weather data is a time-series data, thus a time-series database would be helpful for queriing, or act as a cache layer. This may overlap with #1, depending on implementation or product design. Time-series database like Influx can be more beneficial in certain cases. Each weather station stores data per second or per microsecond, they aggregate data and push to the main influxdb per minute.
8. Push service waits events from event queue and push to certain users.