Assuming a medium size organization of 1000 people and 30% of online users that are trying to book concurrently . We have to support upto 350 tps of write traffic during the peak hours of the day .
For write traffic , assuming a 100 ms p99 response time we will have 10 tps/thread . Assuming hyperthreading machine - there can be about 10 thread per core. Thus 100 tps/core is achievable .
Considering aws ec2 instance type starting size of "large" machine ( m5.large with 2 vCPU , 8 gb RAM ) . we can get 200 tps / instance .
Thus the write instances have to be load balanced to 2 instances .
For High Availability setup - considering a aws deployment on ec2 instances , wherein ec2 instances are assumed to have 99.5% availability and a single AZ provides 99.9% availability - 4 m5.large ec2 instances across 2 AZ s ( 2 in each AZ ) . Additionally DNS should have a weighted round robin routing logic to evenly distribute the load . The should ALB at each zone to further distribute the incoming traffic.
The read traffic could be higher than this ( based on the implementation of UI ) as there could be continuous polls from open clients . We may assume a consistent traffic of about 500 tps of read traffic.
Read instances aim to serve 80% of the traffic using in memory cache . The read api with in memory cache read would be in about 5 ms . Thus it can do 200 tps / thread => 2000 tps/core => 4000 tps / ec2 instance (large size) . This can further be optimized , the limiting factor here is the memory size of the data to be cached . For the sake of HA we maintain 3 ec2 instance in each AZ and 2 AZ deployment with multi master configuration . For improving data locality implement a hash based routing algo on the loadbalancer
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
*Calendar Write Service* - This is a stateless service that can be elastically scaled out to maintain availability and scalability . This microservice has multi AZ deployment to ensure the required availability.
*DB master* - A relational db like postgresql is used in multi master configuration . The master instances are deployed in multiple AZ s .
*Calendar Read Service* - This service provides the most highly used api i.e the get Timeslots for a given user for a given time period . This information will be used by clients to show the participants timeline view in meeting scheduling screen and a user's own calendar view .
*Ingress* - Service discovery is faciliated by DNS and the Ingress Load Balancer ( from the cloud provider ) . The load balancer ensure traffic to be routed to instances in multiple AZ to guarantee high availability.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
*Resource Manager* - This microservice is responsible for managing the lifecycle of meeting resources like meeting room . It also has the logic to automatically accept/decline a meeting request . Resource manager maintains an in memory interval tree per meeting room that will enable it to quickly decide if a requested timeslot is available or not .
*Mitigation* - A local cache of the User entities can be built within the user gateway to be able to locally validate participants even if the external system is down .
*OAuth Server* - is a critical infra component , a mitigation for unavailability of this is not possible as such an approach will compromise security.
*Mitigation* - A full mitigation is not possible . However an async integration ( with retry policy ) ensures that the notifications are not lost and they are eventually delivered.