support different apps in mobile application
performance monitoring
message targeting: user preference, based on user behavior , user segment(location, demographics, etc)
delivery schedule: timezone, frequency, after some event
personalization notification
localization
user engagement tracking
reusability
multi-tenancy
high performance
reliability
scalability
known metrics:
100K notifications per second in peak, 115 in normal
1s e2e latency
5s of scheduled time
delivery rate 95%
notification open rate 20%~30%
error rate 2%
meesage payload length avg: 2k
QPS:
peak
100K, considering each instance send 10K, need 10 worker instance
non peak
115, need 1 worker
-- need auto scaling policy, talk later
networkband width:
peak 100K * 2K = 200M
non peak
115 * 2K = 200K
disk:
1 year of storage
115 * 24 * 3600 * 365 * 2K = 7253G = 7T storage
replica = 3, total 21T
/send_notification, title, payload, receiver_id, device_type
return OK/NOT OK
noSQL
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
sql db vs nosql db
use of message queue:add complexity for decoupling system
send to endpoint failure: add a new topic for failure, have other work retry on that topic. also log those events into influxdb/datadog metrics.
noSQL hotspot, partition by userid
kafka topic can be partitioned by user_id
auto scaling for peak/non peak
machine learning process: separate recall and ranking
separate FCM workers to APN workers
backend server send to influx
have a calibration between backend servers to monitoring any data loss/duplication