Loading...
Availability - this service has to be highly available.
Response time - functionality has to have a low response time, e.g., less than 100ms.
Scalability - we will get more and more requests to return the weather forecasts, so the service has to be highly scalable.
The system has a few providers of weather data - integration with third-party weather services.
20,000 requests per second for searching the weather forecast.
2,000 requests per second were made to get the summary weather reporters.
I assume the search whether the request contains 200 bytes of data and the summary weather report is about 10 kbytes. It requires the bandwidth is about 24 Mbits to pass the data.
Let's think the wether data is updated every 2 weeks from the wether third-party services and every update takes 2 Mbytes, it needs to 54 GBytes to store the wether data for 5 years.
forecast(apiKey, country, state, district, city, fromTimestamp, toTimestamp) returns the weather forecast for the given country,state, district and city for specified period.
weatherSummary(apiKey, country, state, district, city, fromTimestamp, toTimestamp) generates the report for region for the previous seasons.
updateWeather(apiKey, country, state, district, city,timestamp,weather data) updates the weather data by third-party weather providers.
The primary data model for this system is three tablets: one table is for the forecast, for the weather summary and raw data from the weather provider would be stacked up to an object store.
We would use MongoDB to store the data.
Forecast table:
id(10 bytes)
country (100 bytes)
state (100 bytes)
district (100 bytes)
city (100 bytes)
whether data (2 kbytes) temperature, precipitation, humidity.
Weather summary:
id(10 bytes)
country (100 bytes)
state (100 bytes)
district (100 bytes)
city (100 bytes)
whether data (1 kbytes)
The raw data would have the provider-based schema.
See the diagram from the high-level architecture.
API Gateway provides DDoS protection and TLS termination, and forwards request to the right service nodes.
The forecast service processes the request to search wheather for a specified country, state, district, city, and period. This service reads data from MongoDB which data would prepared by the Map Reduce pipeline. The most requested data would be cached in Redis. The summary service replies with weather summary data for a specified country, state, district, city, and past period:1 month, 3 months, and 6 months. The most requested data would be cached in Redis. The weather providers upload the raw data to Kafka and the Map Reduce pipeline processes that data for the forecast and summary and stores that to Mongo DB.
The forecast(apiKey, country, state, district, city, fromTimestamp, toTimestamp) reads the weather data from MongoDB using indexes to speed up the search in DB.
weatherSummary(apiKey, country, state, district, city, fromTimestamp, toTimestamp) reads the weather summary from MongoDB for the previous time ranges.
track(apiKey, trainingId, timestamp, user metrics) sends the workout data to the Tracking service from the client application. The tracking service persists those data: the user speed, geolocation, heartbeats, and distance to No SQL. In this case, No SQL is the best solution to make writing. We may use AWS Dynamo to store that data.
updateWeather(apiKey, country, state, district, city, timestamp,weather data) used by weather providers to feed data to the system. Then that data would be in Kafka and the Map Reduce prepares the data for the forecast and summary services MongoDB.
Performance and scalability of forecast and summary endpoints are extremely important for this system. As such, we employ No SQL to scalable user load evenly. Also, we may Kubenates horizontal autoscaler to regulate node number. No SQL uses replication and partitioning to be fault-tolerant. Using composite indexes (country, state, district, city, timestamp) would help forecast and summary services to process the weather data quickly.
forecast endpoint would be frequently requested and it's the critical functionality, and it should be scalable under load. Most frequent data would be cached in Redis.
Summary endpoint would return the weather data summary for the given past interval. Most frequent data would be cached in Redis.
The client application should be developed in such a way, that if it loses the connection and restores it, it shows the last downloaded data. We may use WebSockets to connect the client application.
Some partitions would be more requested and (city, state, distinct, city, timestamp) key doesn't give an even distribution so we're going to create a hash of (city, state, distinct, city, timestamp) and use it as the partition key.
Mobile applications would represent a comfortable interface for customers. Web client and mobile application clients calls REST API to get the last the weather data.
Forecast and summary services would be stateless and if they fall down, they would be restarted.
The Map-reduce pipeline may be restarted if it fails and that data would be consumed from Kafka again.
All the components - Load Balancers, Web Servers, Cache and Database should have multiple instances for improved availability. There should be robust monitoring and alerting systems on them.
All nodes can fail. Let's look at important failure cases.
If the forecast fails (hardware failure, crash, software bug, network partition, slowness ...), it would directly impact the most time-sensitive operation of this system, i.e., forecast(). To mitigate this, we should always run multiple forecast service nodes. It is s stateless service, so we can have multiple nodes of the same service. We can use an orchestration framework,e.g. Kubernates, to track which nodes are alive, which are likely dead (i.e. not responding to livenesses endpoint for some time), and which nodes should be taking requests.
This service uses Mongo DB that supports master and slave nodes. If leader is down, the one of slave nodes would be promoted to leader. To decrease the inactivity timeout for Mongo DB, we may support the leader-to-leader replication, when the standby leader stores updated data and may get promoted immediately to a new leader.
Losing the summary service would also impact API functionality. However, the client application may mitigate the failures by showing outdated data and trying to connect to the backend. To mitigate this, we should always run multiple forecast service nodes. It is s stateless service, so we can have multiple nodes of the same service. We can use an orchestration framework,e.g. Kubernates, to track which nodes are alive, which are likely dead (i.e. not responding to livenesses endpoint for some time), and which nodes should be taking requests.
Failing in Map Reduce pipeline is not crucial functionality. And when it's up, the MR starts processing data from Kafka again.
If Kafka fails the system doesn't get the new updated data for some time. We may loose some data but it's not crucial. Also, Kafka is a highly available and fault-tolerant steam system andreplicates the partitions to other brokers.
Scalability - forecast and summary services:
As the number of read requests to forecast and summary API increases, it might put too much pressure on the database, causing slowness, errors, or even crashes.
To avoid this, we can introduce a Redis caches to store frequent responses and buffer a message to the queue. API Gateway would push a message in the message queue, representing the request. The forecast and summary services would pull from the queue, process the request, and notify the API Gateway client requests processed with a hook. The system can inform the client with long polling.
Need to design temperature alerting service according to customers preferences.