Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Define what APIs are expected from the system...
API: getCurrentWeather
GET /weather
request: locationInfo {longitude, latitude}
response: weatherInfo, temperatureInfo, precipitationInfo, humidity, windSpeed
API: getWeatherPrediction
request: locationInfo
response:
List
The list should contain 7 entries, each for a day in the next week.
The WeatherPrediction object should include a date object represents the date that the weather prediction is for, as well as certain weather information including temperature and weather prediction
GET /v1/location/search
parameters:
query: the search name, should be the city name.
limit: the upper limit of numbers of locations to return (could set a default)
lang: language of result name (default en-us).
response:
list of location objects:
place_id: id
lat: latitude
long: longitude
country: country name
state / province
display_name: display name
GET /v1/location/autocomplete
parameters:
text: string, the string that user is typing
lat: latitude of user's location (use as a bias for returning location)
long: longitude of user's location
response:
suggestions list of objects:
place_id
place_name
lat
long
POST /v1/location/subscribe
parameters:
place_id:
condition: (rain, heavy rain, high temperature)
response:
succeeded.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
The basic system is very simple as it consists of only client, API Gateway (for rate limiting and load balancing), server and the third party vendor. There is no internal data storage needed for the functionality.
We should add a cache in the system to store the already fetched weather data for each location, this would help alleviates traffic to vendor and potentially reduce cost.
However, to store Business Intelligence data (such as number of clicks, etc), there are three more components needed: A queue to store data processing jobs, job handler worker machines, and a data warehouse. The purpose of queue and worker is to handle the data asynchronously, so that the latency of handling data would not affect user traffic. The data warehouse is used to store BI data.
There should be a notification system for users to get updates for weather events they are interested in (such as extreme weather alerts, hot wave, cold weather...). The client can initiate this by calling subscribe API in webservice. The Web Service should update this preference to user management service (which will store this preference somewhere, in some data store). We expect third party vendor should provide notification services. Assuming it uses a push model, we will have a notification service which contains a webhook to get such alerts. The notification service should insert some messages for each notification to a weatherUpdateMessageQueue. A weatherUpdateNotificationHandler should read from the queue, and also read user preferences and user information (such as geolocation) from userManagementService. This handler should then decide which user should receive a notification and then deliver it. For this system, typically delivery method could be Firebase Cloud Messaging / Apple Push Notification Service (for mobile apps). One key note is that the weather updates could arrive out of order or stale, there should be a timestamp check at the message queue handler.
An additional historical storage can be used to store historical weather updates. This can be updated through the weatherUpdateNoticationHandler. Apart from sending notification to the client, it can also handle updates to the storage (or alternatively use an async method such as another message queue to update the storage).
In order to handle peak traffic, the web service and notification service should use elastic scaling - handled by ASG.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Cache:
The system is mostly doing reading as read data from vendor. Therefore, we can use a read-through cache in the system. The server should read the cache first, if the data is not in the cache, the cache system read data from vendor and have the result cached. This cache can also be used as a fallback mechanism when vendor is experiencing downtime. Stale weather data can be served as responses for live users.
BI Data Queue:
We use a queue in the system to store metrics for future usage. The queue could be a kafka or sqs job queue. This helps decouples the BI data handling and real time traffic, so that the process of handling BI data should not affects users latency
Vendor / cache / notification service:
The service will use vendor data, however, in order to make sure vendor could be switched later, there should be an adapter layer in web service and notification service that will handle data model adaptions. An internal data model should be used within the system, and the adapter handles transforming of data from vendor to internal systems.
There should be a circuit breaker wrapper for the vendor calls. If the vendor calls fail like over 50% in a certain time period (say 5 minutes), the system should break the circuit and fail fast. It should wait for sometime before slowly ramping up the traffic again (time to ramp up should be dependent on the vendor's SLA).
Historical storage / data warehouse: In order to avoid duplicate records that resulting from retries or network issues, we should usually expect the vendor to have some id about each event, and use the vendor + id to uniquely identify each event. Only upsert an event if it is not yet already upserted.
Colescing Locks: When the cache of a city is expired, instead of letting all requests to hit the vendor, we should have a "lock" so that only one of them will query the vendor. All other requests should wait for the response to be broadcast.
Cold start: When the system is initialized, pre-heated the cache with top cities / locations so that we do not get a lot of thundering herds.
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Not applicable as we do not store data internally for functionality, we might store data for BI purposes.