System requirements

Functional:

• Merchants are able to create online stores.
• Merchants are able to collect payments.
• Merchants are able to list different kind of products.
• Merchants are able to edit what their store looks like.
• Product Inventory.
• Managing inventory synchronization.
• Order processing.

Non-Functional:

• Isolation
• Merchant data must be isolated.
• Reliability
• The system must be reliable. It needs to be able to deal with unexpected spikes in traffic.
• Scalability
• The system must be scalable and handle hundreds of thousands of request by users.
• Potentially consider using microservices.
• Addressing data backup and recovery strategies

Capacity estimation

• Total Requests per minute = Peak User Traffic * Average Requests per User = 1,000,000 * 10 = 10,000,000 requests per minute
• Total Capacity = (10,000,000 requests/min * (0.1 CPU cores + 100 MB Memory)) * (2 Horizontal Scale + 20% Vertical Scale) = (10,000,000 * (0.1 + 100)) * (2 + 20%) = (10,000,000 * 100.1) * 2.2 = 2,210,220,000 CPU cores * MB of Memory per minute

API design

1. Create Store API
2. Allow users to create a new online store by providing basic information such as store name, URL, and description.
3. Assign a unique identifier or store ID to each newly created store for easy reference.
4. User Management API
5. Allow merchants to login and logout
6. Search and filtering API
7. Search products based on keywords, categories or filters.
8. Filter products based on price range, brand, etc.
9. Sort products based on relevance, price, popularity
10. Order processing API
11. Please new orders
12. Update order status
13. Retrieve order details
14. Process payments
15. Management shipping and fulfillment
16. Product management API
17. Create, update and delete products
18. Retrieve product details, including name, price, description and images.
19. Categorize products
20. Manage product variants and options

Database design

Metadata schema for merchants

• The MERCHANT entity stores information specific to each merchant, such as the store name and contact email.
• The METADATA entity captures key-value pairs of metadata associated with each merchant, allowing for flexible storage of various configurations and settings.
• There is a one-to-many relationship between MERCHANT and METADATA, indicating that each merchant can have multiple metadata entries.

Merchant database schema

• USER entity stores information about users who interact with the platform, including authentication details.

• STORE entity represents individual stores created by users, containing store-specific details.

• PRODUCT entity holds details of products listed on the platform, linked to the respective store they belong to.

• ORDER entity tracks orders placed by users, along with order status and associated user ID.

• ORDER_ITEM entity stores individual items within orders, including quantity, price, and references to the corresponding product and order.

• PAYMENT entity records payment transactions made for orders, including payment amount, date, and status.

High-level design

We can utilize a container orchestration platform like Kubernetes to provide isolated environments for each merchant. By leveraging Kubernetes, we can create separate pods or nodes for each merchant to ensure isolation and scalability.

The code for the request handler that determines the destination pod based on the merchant context can live in various components depending on the architectural design and requirements of the system. The merchant context can be determined by the merchant_id

There would be a service discovery mechanism to dynamically handle pod discovery and routing based on merchant_id.

We can have multiple pods per merchant, where each pod corresponds to a service.

High Level Diagram(s)

We would also need components and functionalities required for e-commerce stores.

Order Processing Service

• Order status update
• Order placement
• Payment processing integration
• We would have to choose payment gateway.
• Order history

Product Management Service

• Ability to list, upload products

Inventory Management Service

• Manages the inventory stock.

Each merchant's pods comprises a set of microservices or services dedicated to that merchant's operations. These services include product management, order processing, payment handling, etc., tailored to the specific needs of the merchant. There would also be a gateway for each merchant so that the request will be routed to the targeted service's pod.

When a new merchant signs up on the platform and requests to create a store, the system triggers an automated deployment process to spin up a new merchant pod. This process involves creating containers for the services required by the merchant's store, such as web servers, databases, and application components.

Request flows

1. Each merchant has a pod within the Kubernetes cluster that contains a dedicated Database Shard specific to that merchant (Merchant 1, Merchant 2, Merchant 3).
2. The Product Service, Inventory Service, and Order Service of each merchant interact with their corresponding Database Shard for data storage and retrieval operations.
3. The API Gateway serves as the entry point for external requests, directing them to the appropriate microservices and Database Shards based on the merchant making the request.
4. The Service Mesh manages communication, security, and observability within the Kubernetes cluster, ensuring that each merchant's microservices interact securely and efficiently with their dedicated Database Shard.

Detailed component design

Kong API Gateway

1. Kong is an API gateway with features like rate limiting, authentication, and request routing that can enhance the platform's API management.
2. Utilize Kong plugins like Request Transformer or Header Transformation to modify or extract headers containing merchant context information from incoming requests. Configure Kong to forward requests with the extracted data to Istio's ingress gateway.

Istio

1. Istio can help manage the traffic flow between services, enhance security, and provide monitoring capabilities in a microservices environment
2. Define Virtual Services in Istio to map HTTP requests to specific Kubernetes services corresponding to merchant pods. Istio Virtual Services can be configured to extract merchant-specific information from requests.
3. Set up an Istio Gateway to act as the entry point for incoming traffic. Configure Virtual Services to route requests based on the merchant context extracted by Kong, directing traffic to respective merchant pods within the Kubernetes cluster.
4. Implement Istio's Quality of Service routing capabilities to ensure that merchant-specific traffic is directed to the appropriate pods based on the extracted context. Define routing rules for different merchants to ensure effective pod targeting.
5. Maintain consistency in configuration settings and shared context data between Istio and Kong to ensure accurate extraction and routing of merchant-specific information. Update configurations as needed to reflect changes in merchant context.
6. Leverage Istio's authentication and authorization policies to enforce access control and security measures for merchant-specific traffic. Ensure that only authenticated and authorized requests are routed to designated merchant pods.
7. Enable Istio's observability features like distributed tracing and monitoring to track requests, performance metrics, and traffic patterns related to merchant-specific routing. Monitor and analyze traffic flows for optimization and troubleshooting.

Data Storage and Database Component

1. Database Sharding:
2. distribute data across multiple databases based on merchant IDs for better performance and scalability.
3. Redis for Caching:
4. Utilize Redis for caching frequently accessed data to improve application performance and reduce database load.

Isolation with Network Policies

Network Policies in Kubernetes provide fine-grained control over pod-to-pod communication, enabling administrators to define rules for traffic flow within the cluster. Here's how advanced networking features like Network Policies can be leveraged in the architecture of Shoplify:

1. Isolation between Merchant Pods:
2. Network Policies can be used to enforce isolation between pods belonging to different merchants. Policies can restrict network communication and traffic flow, ensuring data privacy and security.
3. Pod-to-Pod Communication Rules:
4. Define Network Policies to specify which pods can communicate with each other based on labels, namespaces, or other metadata. This controls the traffic allowed between components within the cluster.
5. Deny-All Default Policy:
6. Implement a default-deny policy to block all pod-to-pod traffic by default. Specific rules can then be added to permit necessary communication paths, reducing the attack surface and enhancing security.
7. Traffic Shaping and QoS:
8. Network Policies enable Quality of Service (QoS) enforcement and traffic shaping rules. This helps prioritize and control network traffic based on application requirements in the Shoplify platform.
9. Port-Level Restrictions:
10. Apply Network Policies to restrict traffic to specific ports on pods hosting sensitive services like databases or payment gateways. This adds an extra layer of security and access control.
11. Ingress and Egress Rules:
12. Craft Network Policies to manage both ingress (incoming) and egress (outgoing) network traffic from pods. Define rules for allowed sources, destinations, and protocols to enforce communication policies.

Scalability - This isolation ensures that the system can easily scale horizontally by adding more resources to support an increasing number of merchants without impacting the performance of existing merchants.

Data Isolation and Security - By assigning a separate Database Shard to each merchant, the architecture enhances data isolation and security. Each merchant's data is stored in a dedicated environment, minimizing the risk of data leakage and ensuring data privacy and compliance with regulations such as GDPR. The use of a Service Mesh also enables secure communication between microservices within the cluster.

High Availability: The architecture promotes high availability by distributing services across multiple pods within the Kubernetes cluster. In case of pod failures, Kubernetes can automatically spin up new pods to maintain service availability. This redundancy enhances the reliability of the system and reduces the risk of downtime.

Edge cases:

Large-Scale Product Uploads:

• Batch Processing: Implement a batch processing system where merchants can upload products in bulk. Use asynchronous processing to handle large volumes of product data without impacting the performance of the platform.
• Distributed File Storage: Store product images and descriptions in a distributed file storage system like Amazon S3 or Google Cloud Storage to efficiently manage and serve product content.
• Queue Mechanism: Utilize a message queue system like RabbitMQ or Kafka to manage product upload tasks asynchronously and ensure reliable processing.

Multi-Merchant Transactions:

• Isolated Database Shards: As discussed in the architecture, maintain separate database shards for each merchant to ensure data isolation and prevent any impact on one merchant's transactions affecting others.
• Transaction Management Service: Implement a centralized transaction management service that coordinates transactions across different merchants, ensuring consistency and atomicity.
• Event-Driven Architecture: Use an event-driven architecture with a message broker to enable seamless communication and coordination between different microservices handling transactions from multiple merchants.

Concurrent Order Placements:

• Optimistic Locking: Implement optimistic locking mechanisms within the database transactions to prevent conflicts during concurrent order placements by different users.
• Distributed Caching: Use distributed caching solutions like Redis to store frequently accessed data such as product information, order statuses, and user sessions to reduce the load on the database and improve response times.
• Horizontal Scalability: Ensure that the system is horizontally scalable, allowing multiple instances of services to handle concurrent orders effectively. Utilize load balancers to distribute incoming order requests evenly across instances.

Merchant Pods Creation Process

When a new merchant registers on the platform, the system initiates the process to provision resources for their store. This can include creating a new namespace or resource group dedicated to the merchant's services.

We can define a Kubernetes Deployment manifest that specifies the desired state of the application, including details like the container image, resource limits, and replication settings. This manifest will serve as the blueprint for creating new pods for the merchant.

Upon merchant registration, a controller or a custom automation script can interact with the Kubernetes API to create a new Deployment object based on the predefined configuration. This action triggers Kubernetes to schedule and launch the required pods for the merchant's services.

Create a Kubernetes Service manifest to expose the merchant's pods internally or externally, allowing other components within the system to communicate with the merchant's services. By defining a Service, Kubernetes can load balance traffic to the merchant's pods and ensure seamless connectivity.

Trade offs/Tech choices

Using Kubernetes for containerized applications introduces several trade-offs related to the level of isolation it provides. Let's delve into the trade-offs associated with using Kubernetes for isolation:

Isolation vs. Resource Overhead:

• Trade-off: Kubernetes offers robust isolation boundaries between containers using features like namespaces and cgroups. However, this level of isolation can introduce additional resource overhead due to the management of these mechanisms.

Security vs. Complexity:

• Kubernetes provides security features such as network policies, PodSecurityPolicies, and RBAC for controlling access. However, configuring and managing these security controls can increase complexity.

Having to isolate each merchant's services and database this way can be quite costly.

1. Infrastructure Cost: Hosting and managing multiple sets of services, databases, and supporting infrastructure for each merchant can lead to increased infrastructure costs. Each merchant's pod would require resources such as servers, databases, networking components, etc.
2. Scaling Cost: As the number of merchants on the platform grows, scaling each merchant's resources to accommodate increasing traffic and data can further add to the costs. This includes scaling up servers, databases, and other components within each pod.
3. Maintenance and Monitoring Cost: Operating and maintaining multiple sets of services and databases, along with ensuring high availability and performance for each merchant, would require continuous monitoring, updates, and support, leading to additional operational costs.

Failure scenarios/bottlenecks

Failure scenarios such as Container Failures, Node Failures, Networking Issues, and Storage Failures are shown affecting individual pods within the Kubernetes Cluster.

Bottlenecks like Overloaded Nodes, Inefficient Communication, Slow Deployment, and Monitoring and Logging Issues are depicted as potential challenges within the system architecture.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?