Requirements

Functional

• Users can request weather for a chosen location (temperature, humidity, precip probability, sunny/clear, wind, etc.).
• Users receive immediate crisis notifications for hazards (storm, hurricane, high wind, flash flood).

Non-functional

• Availability & Partition Tolerance (AP): tolerate partitions; slight staleness (seconds) allowed.
• Scale target: support ~1,000,000 daily active users (steady read ~12 req/s) and large burst capacity for city-scale crises.
• Other constraints: low-latency read for GET /v1/weather, reliable prioritized delivery for crisis alerts, and cost-effective autoscaling.

API design

All endpoints JSON over TLS. Standard error envelope:

POST /v1/subscribe/{city} — subscribe/update

• Path: city (canonical id/slug)
• Body:

• Responses:
  • 201 Created subscription created (or 200 OK if updated)
  • 400 Bad Request, 404 Not Found (bad city), 409 Conflict (semantic)
• Idempotent on user_id + city.

DELETE /v1/unsubscribe/{city} — unsubscribe

• Body: { "user_id": "string" }
• 200 OK or 404 Not Found.

GET /v1/weather/{city}/{YYYY-MM-DD} — forecast read

• Response (200):

• Errors: 400 invalid date, 404 not found, 500 server error.
• Primary read path: Redis cache (cache-aside); fallback to canonical store.

Operational notes

• Rate limiting: per-API keys and per-user throttles; backoff headers on 429.
• Retries: clients should retry idempotent operations with exponential backoff.

3. Detailed component deep dives

A. Weather Subscription Service

Purpose: persist & serve who is subscribed to which city (fast fan-out key).

How it works

• API writes subscription to DB and updates/invalidate Redis cache subscribers::city::<city_id>::shard::<n>.
• Partition pattern: PK = city#<city_id>; SK = user#<user_id> (NoSQL).
• Scaling
• Partition-by-city for fan-out; shard hot cities (city#id#shard-0..k-1).
• Redis caching of per-city subscriber shards (TTL ~1–5 min).
• Data structures / algorithms
• Redis: sets / lists of subscriber records (O(1) membership/scan).
• Sharding via consistent hashing.
• Tradeoffs
• NoSQL (DynamoDB/Cassandra) → AP, fast fan-out; relational DB would require sharding and complex scaling.

3. Detailed component deep dives

A. Weather Subscription Service

Purpose: persist & serve who is subscribed to which city (fast fan-out key).

How it works

• API writes subscription to DB and updates/invalidate Redis cache subscribers::city::<city_id>::shard::<n>.
• Partition pattern: PK = city#<city_id>; SK = user#<user_id> (NoSQL).
• Scaling
• Partition-by-city for fan-out; shard hot cities (city#id#shard-0..k-1).
• Redis caching of per-city subscriber shards (TTL ~1–5 min).
• Data structures / algorithms
• Redis: sets / lists of subscriber records (O(1) membership/scan).
• Sharding via consistent hashing.
• Tradeoffs
• NoSQL (DynamoDB/Cassandra) → AP, fast fan-out; relational DB would require sharding and complex scaling.

B. Weather Forecasting Ingestion & Normalization

Purpose: accept provider feeds, normalize schema, detect hazards, dedupe, persist canonical events, and update Redis cache used by GET API.

How it works

1. Accept raw feeds (webhooks/pull/S3).
2. Map to canonical message with event_id, city_id, date(s), metrics, source, confidence.
3. Run hazard rules (config JSON): e.g., if wind_kph >= 80 → HURRICANE; wind_kph >= 35 → HIGH_WIND.
4. Deduplicate: Redis SET or content-hash with TTL (5–15m).
5. Publish canonical message to message broker and write forecast key weather::city::<id>::date::<YYYY-MM-DD> to Redis.
6. Persist raw/canonical payload to object store for replay/audit.

Scaling

• Stateless pods, autoscaled by input rate and queue backlog.
• Algorithms / data structures
• Dedup via Redis SET; fuzzy dedupe with content-hash (SHA256) + TTL.
• Rule engine configurable via JSON for quick ops changes.
• Tradeoffs
• Stateless per-event rules are fast; but complex multi-event tracking (track storm progression) requires a small state store (time-series DB), increasing complexity.

C. Messaging & Delivery (RabbitMQ + Workers)

Purpose: prioritized, reliable fan-out of events to subscribers and final push to providers (Airship/FCM/APNs).

How it works

• Ingestion publishes canonical events to topic exchange with routing keys: weather.city.<city_id>.normal / ...urgent.
• A shard mapping: shard = hash(city_id) % N_SHARDS. Bind routing keys to deliver.normal.<shard> and deliver.urgent.<shard>.
• Workers consume urgent queues first (or separate consumers), fetch subscriber shard from Redis, filter preferences, batch tokens and call push provider APIs.
• Dedup per user-event: create Redis key delivered::<event_id>::<user_id> TTL to avoid duplicate sends.
• Retries: transient failures -> exponential backoff; permanent -> mark invalid token and DLQ.

Scaling

• Increase N_SHARDS, add more worker instances; autoscale by queue length and delivery latency.
• Algorithms / data structures
• Consistent hashing for shard mapping; batching algorithms to maximize push provider batch sizes.
• Tradeoffs
• RabbitMQ provides per-message priority; for extreme scale consider Kafka (higher throughput, durable log) but priority requires additional topics.

4. Operations, DB design, capacity & failure planning (compact)

Database choices & schema (concise)

• Subscription store: DynamoDB/Cassandra. PK=city#<id>, SK=user#<id>, attrs: push_token, platform, notify_level, channels, status.
• User profile (OLTP): Postgres or DynamoDB: user_id, tz, home_city, locale, consent_flags.
• Forecast storage: Redis for cache. Long-term: S3 Parquet + Time-series DB (TimescaleDB/ClickHouse) for historical queries and model training.

Request flow (step-by-step)

1. POST /subscribe → Subscription service writes DB → update Redis shard cache.
2. Provider → Ingestion normalizes, dedupes, tags hazards → publish to RabbitMQ and update Redis forecast key.
3. RabbitMQ routes → Delivery worker pulls → fetch subscriber shard from Redis → batch & send to push providers.
4. GET /weather → read Redis → if miss, fallback to canonical store & populate Redis.

Capacity estimation (concise, arithmetic shown)

Assumptions: Users = 1,000,000, C = 10,000 cities, updates per city per hour U = 4, avg subscribers per city S = Users / C.

Step-by-step:

• Users ÷ C = 1,000,000 ÷ 10,000 = 100.
• Events/hour = C × U = 10,000 × 4 = 40,000.
• Events/sec M_s = 40,000 ÷ 3,600.
  • Divide by 100: 40,000 ÷ 100 = 400; 3,600 ÷ 100 = 36 ⇒ 400 ÷ 36 = 11.111... events/sec.
• Deliveries/sec D_s = M_s × S ≈ 11.111... × 100 = 1,111.11 deliveries/sec.

If one worker (with batching) handles ~400 deliveries/sec, required workers = ceil(1,111.11 ÷ 400) = 3 workers initially (autoscale on queue length). For crisis spikes (e.g., 200k subscribers), design for burst strategies (sharding, staggered batches, provider scaling).

Key failure scenarios & mitigations

• Provider outage: serve cached Redis forecast (mark as stale), replay from persisted raw data, alert ops.
• Huge crisis spike: urgent-only first wave, staggered batches, multi-region push provider accounts, add worker capacity and shard count.
• Redis outage: failover to DB (slower), circuit-breaker, degrade personalization to rules.
• Invalid tokens / push provider limits: token invalidation pipeline, DLQ and cleanup jobs, rate-limiting for provider calls.

Quick roadmap (top 3 next steps to reach interview-grade)

1. Add explicit DB table/NoSQL table definitions and sample queries.
2. Prepare a short sequence diagram of one crisis flow (ingest → route → deliver) to narrate in the interview.
3. Prepare a one-page capacity calculation including a 10× spike scenario and recommended autoscaling triggers.