Detailed Component Design
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
So first of all we need to have a service and if we're thinking in high school we will put him in something in some orchestrator something like ETS from the AWS. And it will have an HTTP endpoint it's the simplest way to do it so and we can also use a dynamic DB when and when we I'm suggesting the animal because we could Leverage The partitioning feature so even though we have a lot of a lot of tests a lot of schedules we could also. We could we still have we would still have performance with the partitioning feature. And This current job will run like from. 5 to 5 minutes I think get a batch from the most no executed yet the older registers older tasks that were not executed yet so it will run them and. Tag them as Already executed So took to persisted I think we should have like. An ID for each task and we would use the schedule date to have a partition for each day or each month. It depends on scale if it's a high scale I think we should use. So we have a Dynamo DB partition for each day and when we run the Chrome job we could just. We could do we could just fetch the a single partition so we will probably have a lot of partitions and. But this will help on performance and will enable the. The Chrome job to finish it's test before the next run this could cause a problem so one so we would have the test name the description the schedule date so we have so we could partition it and we need to have a field of. Suggesting the state of the the task like if it's painting or if it's. Running or if it's finished because we have to manage the state at some point because at some point the current job will search for the. We will fetch. Because when the current job fetches a new batch to run we need to. We need to be sure to fit only the ones that are not. Running or they're not. Finished Think that Each each time we fetch the the badge and start to execute the tasks. I will need some atom City for. For each one that is finished to be able to do with. Feeling scenarios for example if we feel. If we get a badge of a thousand tasks and if you own 100 the other 900 are still need to be so when we. Will tag someone some of the desks as finished we need to go there and we will need something like maybe a Costco topic where every time we finish processed single task we will do some message that? Will will be consumed by the service that managed the the test and we'll tag the task as finished so this way you will not we could have white processing test? In duplicity so let's let's draw this. Okay so! We will have a client that could be in your UI or mobile app or anything that does an ATP request. To By posting on you a new schedule so this new schedule we will be written. This new schedule will be waiting on the animal. And From An age 5 minutes for instance a current job will run will get a badge of I don't know if you're thinking on High School. Maybe a thousand. Tasks And this Does that those taxes start to be? Executed so each time the Chrome job is acute at the desk if you produce a message for Kafka topic saying that that task was finished the consumer will consume this message and write and we'll write on the database a new status for it saying it was finished so we don't executed multiple times. So we just have to make sure that if the service has. Multiple replicas I will consumers to consume the same consumer group so we don't have duplicated rights when writing the. Status And I think this is it? So the batch actually need to be a foot. Because every time we get a batch we need to. We need to set the status of each desk as. Processing or something like that and. If We have an error we need to. Do all back so I think we need another cap topic Costco topic? That Would be the? We need the new capital pic that will be there right back status.