System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

1. refresh the data. every 15 min.(cache time-based expiry, adding user triggered overriding the cache mechanism)
2. provide user zip code and fetch weather information. Allow to search weather of other cities.
3. people could use city name to fetch weather information. internally, the city name transformed to zip code, and input the zip code to third party API to fetch relevant weather information. City has multiple zip codes, providing selections.
4. provide weather data based on GPS location
5. Send notifications for user if there will be extreme weather soon.
6. provide offline function for user to access previously fetched data without internet connectivity.
7. allow user to provide feedback for the app
8. store historical data, integrated AI technology to predict
9. In weather event, use WebSockets to provide real-time data

Non-Functional:

List non-functional requirements for the system...

1. API Rate Limits: prevent overload to third party API. Utilize caching for repeated requests, and use background fetches to reduce direct API calls.
2. The system will return response to user within one second. Apply load testing tools like JMeter
3. ease use: allow users to access basic functionalities without login in. Integrated user-friendly guidance system.
4. employ Priority Queue to manage user API request. It prioritizes the high-traffic areas

Capacity estimation

Estimate the scale of the system you are going to design...

1. expect 10k users call API simultaneously
2. limit 1,000 requests per hour per user
3. upper bound of 10 million requests per hour
4. users primarily request new weather data
5. most user request weather information under 10 times a day. In extreme weather conditions, user requests rate will increase to like 10 times an hour.

API design

Define what APIs are expected from the system...

getWeatherByZipCode(string zipCode): we provided zipCode selections for user, ensuring the input zipCode is valid. Implement Rate Limit policy

citynameToZipCode(string city_name): give zipCode lists according to the city_name, providing for user selection

getWeatherByCityName(string city_name): dealing one city name mapping to different cities, list the city

collectFeedback(userId, feedback): select user feedback for the weather app, help us refine the functions continuously. log asynchronously to avoid disrupting API request.

predictByAI(location, timeFrame): integrated AI technology to analyze historical data, and make prediction.

Implement OAuth for authentication to access personal information.

use Redis cache to store common response, reducing API calls.

implement geohashing, enabling indexing to benefit searches.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

1. use Redis cache to store weather API response Data. store common response in memory, improving response efficiency. Reduce API request. Serve for NoSQL database
2. Using LRU algorithm.
3. Time based expiry, adding user-trigger overriding mechanism.
4. using hash table data structure in Redis.
5. NoSQL database like MongoDB providing flexibility, offering rapid data ingestion. Support high-volume reading and unstructured data, like weather reports.
6. introduce a connection pooling mechanism for database interaction, reusing the resources to improve response efficiency.

Database Schema Design

weather_report{

string id "primary key",

string location_id "foreign key referencing Location",

data report_data,

float temperature,

string description

}

location{

locationId,

locationName,

latitude,

longtude

}

user{

userId,

default_locationId

}

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

1. utilize a micro services architecture for scalability. Decompose the services for fetching weather data, user management, and notifications.
2. During weather events, there will be spikes of user requests. Implement load balancer to manager dynamic load. Dynamic scaling, like AWS EC2
3. use database replicas, distributing wiring and reading load across different services. NoSQL database for rapid request and changes.
4. use RabbitMQ for high volume message processing
5. monitor service load, alert warning in time. Visualize performance metrics
6. edge computing. process data closer to user location, reducing latency and reducing load for central services. (People likely requests weather where they live in) Using CDN like AWS CloudFront combined with AWS Route 53 to send user request to the nearest service.
7. implement Docker and Kubernetes for containerized application.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

1. ELB allow auto-scaling of servers. Distribute evenly. Improve fault tolerance.
2. use Redis cache to store common response, avoiding hit API Limit Rate
3. use micro service. scalable solution. Ensure downtime of the specific server will not disrupt others. (cons: introduce inter-service communication problems)
4. Utilize CDN combining AWS Route 53, routing requests to the nearest server to user, reducing latency.
5. Implement Priority Queue, prioritized the request from high traffic location, which means we could fulfill most users requirement first.(cons: low-traffic areas may face starving issue)

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

1. the application related closely to third party weather API and GPS services.
2. Integrate data from multiple can lead to discrepancies, result in provide error information for users.
3. ELB instance should set up appropriate health checks, otherwise, ELB may continue routing traffic to unhealthy instances.
4. frequent read and write operation may lead to a bottleneck of database

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

1. Migrate the application to cloud solutions. For example, migrate weather fetch API to cloud serverless functions like AWS Lambda
2. divided the GPS locations into blocks, reducing API calls, improving the chance to hit cache
3. use ALB which support WebSocket