System requirements

Functional:

• User can use application via web or mobile app
• WeatherAPI is used to retrieve information
• User's location is used to show initial weather information
• There exists a search which lets the user search for cities to show the weather information in that particular location
• Data to be shown: temperature (24h forecast) and humidity

Non-Functional:

• Low latency: The user's locaation shouldn't be a degrading factor for performance
• High scalability: The app should work as expected under high load
• Accurate and reliable weather information
• Security and compliance: user's location data needs to be asked for explicitly
• Usability / UX: The interface should be simple and easy to use for the user

Capacity estimation

• Let's consider a city with 1m population for our estimation
• we assume a 1% usage -> 10.000 users
• DAU: 10% - 1000 users in that particular city

API design

We'll use a RESTful API for our system. There's basically one endpoint that would suffice the requirements. Both current location's weather information, as well as a searched city can be retrieved from a single endpoint.

This endpoint looks as:

GET /weather/:city

The :city param is the geohash of the particular city.

The response looks as follows:

[{time: timestamp, temperature: int, humidity: int}]

Database design

So we assume that the data for the weather forecast will be retrieved from an API, WeatherAPI in this case.

As calling the API is both expensive in terms of cost and performance, we'd need to have a proper mechanism to cache the retrieved data on our side.

For this purpose let's assume that we store the data per city and define that city as a geohash to be stored as a key.

The value of each city (geohash) will be a JSON object looking like:

[{time: timestamp, temperature: int, humidity: int}]

So it will be an array of objects. It will be a 24 hr forecast for each day, meaning that we'll store 24 objects. Each timestamp is representing an hr forecast and the object consists also of temperature and humidity information.

So the redis data will be stored as

KEY - city geohash , VALUE: [{time: timestamp, temperature: int, humidity: int}]

High-level design

flowchart TD

n0["Client"];

n1["Weather Service"];

n2["3rd Party Weather API"];

n3[("Redis Cache")];

n0 --> n1;

n1 --> n2;

n1 --> n3;

n1 --> n0;

Request flows

The client calls the Weather Service with a particular location request.

The Weather Service checks the Cache if the geohash for the city exists in the redis cache. If so the value is returned. Otherwise it's retrieved from the 3rd party and written to the cache again. Then the result is returned to the user

Detailed component design

Our high level design outlines the important components, but it's not enough for a scalable architecture. Here's a a proposal for the detailed design:

flowchart TD

n0["Mobile"];

n1["Weather Service"];

n4["Web"];

n5["Load Balancer"];

n7[("Redis Replica Americas")];

n8[("Redis Replica Europe")];

n9[("Redis Replica Asia")];

n10[("Redis Replica Pacifics")];

n11["Gateway"];

n12["3rd Party Weather API"];

n4 --> n5;

n0 --> n5;

n5 --> n1;

n1 --> n11;

n11 --> n7;

n11 --> n8;

n11 --> n9;

n11 --> n10;

n1 --> n12;

Trade offs/Tech choices

• First we introduce a load balancer which can distribute the traffic from both mobile and web clients to the weather service
• We deploy replicas of the redis instance based on location -> this reduces latency for users and also minimizes the redundance of data as each location would be unique in the replicas
• A gateway is introduced which handles the logic of redirecting the weather services request for a particular location geohash to the right redis replica
• The data of the cache is updated asynchronously every night for every replica -> during the update we might have eventual consistency, but we're showing the latest timestamp's data until the data is updated
• The data from the Weather API is normalized and adapted to an internal itnerface we defined, so we stay agnostic to any kind of 3rd party changes in the future

Failure scenarios/bottlenecks

• Stale weather data until the update finished - when it fails after multiple retries, it wouldn't be able to update for a long time and the users wouldn't see up to date data
• The redis replicas are only per region once deployed. This can be a bottleneck
• When users in a region are searching for other regions, one replica can get unexpected load

Future improvements

• Making the cache update mechanism more resilient against failures to prevent that users see stale data