Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
all /auth for user authnetication.
/weather?city="users_city",period="1day(hourly)" - This route is the default on the dasboard when user opens the app we show him the weather
/weather?city="users_city",period="1day(hourly)/7days(daily forecasts)" - This route is called on dashboard when user wants to see also 7 days forecast with daily temperatures.
/weather?city="users_city",period="1day(hourly)/7days(daily forecasts)",zip_code="01010",latitude="value",longitude="value" - This is called when user wants to search a different city so he can switch and
/notifications - user can set for what city and for what events he wants to get notification.
/location - to resolve current location of user.
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Stateless authentication.
Api gateway and load balancer to distribute the load evenly it has info about the alive servers and its loads and route the traffic based on this information.
kuda based scaling for peaks in incomming traffic.
Database SQL - Postgis database with geo location data. Events. notifications.
Main tables.
forecast_data table - all data aggregated by location we get from third party, it makes sense maybe to split them by city boundaries. partition the data in table by boundaries and then by time partitioning. all data should be aggregated by month-week based on granularity and data amount.
city_data - a table with spatial indexing for efficient querying to fast query what cities are we interested in based on users location - what polygons are in the radius of 5km of this place? then use this cities to search for the data and show user data for the main city he is nearest to.
Microservices -
Event + Notification microservice with worker to evaulate events and tell if we should create a notification - this microservice takes care of real-time notification system that notify user when there is any event happening near to user. We set a static trashholds for temperature, wind speed, humidity etc and if anywhere the value goes over the trash hold we create event. Event is linked to the location so all users that calls for a specific location will get the notification.
Main weather microservice - this takes care of all the data aggregation, location translation from lang long to city. orchestrating the jobs that will scrape the data for specific locations etc.
We also have jobs that will invoke functions to get data for the main cities we are interested in - the smaller cities will fetch data on demand.
caching in the system works the way that the hourly data that already "expired" are cached for 24hours in cache. because we show data for last 24hours and forecast for next 24hours. because the new data can change on hourly basis we keep it in cache for 1 hour and refresh it every hour.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
main weather microservice - it aggregates the data there is 24h fetch for all cities we defined as main that will sync its data.
then there is a on-demand function that will fetch the data store it in database and cache - it can cache values that already happend to 24h becuase we show last 24h in data. Cache miss will be a route to DB and ensure we have this value cached - if its 24h historical data its cached for the 24 hours automatically on first request. Upsert is used to store data because it can change over the time. (future data need to be rewrited with last-write wins approach)
Postgis database - Postgis database with geo location data. Events. notifications.
Main tables.
forecast_data table - all data aggregated by location we get from third party, it makes sense maybe to split them by city boundaries. partition the data in table by boundaries and then by time partitioning. all data should be aggregated by month-week based on granularity and data amount.
city_data - a table with spatial indexing for efficient querying to fast query what cities are we interested in based on users location - what polygons are in the radius of 5km of this place? then use this cities to search for the data and show user data for the main city he is nearest to.
Event + notification service - Events are generated by workers that will evaluate if any of the values in data for next 5hours is over the thrashhold we define as static map in code. generate an event and notification worker will distribute notification to users (there could be 1milion of users for czech republic and there could be strong wind so we need to distribute the notifications). It will use mobile push notification or email service.
We will have two providers with canonical schema. If one vendor is out we call the second/third.
If there are no vendors available use a predicting mechanism to tell for specific location what could be the weather humidity etc based on historical data and then when providers are back use upsert to refresh the data.
i will implement the vendor-agnostic integration the way that I will have client - all clients will be splited by vendor each vendor will pull data - normalize to canonical schema and save to database. WE always want to use one main source if its down we use another if its down too we predict the data and use our local algorithms to show atleast some data to user.