Design Order Processing Infrastructure for microservices

Question

Design a distributed order processing system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
- **Order Creation**: Users can create orders for rides through the Lyft app.
- **Order Tracking**: Users can track the status of their orders in real-time.
- **Payment Processing**: Integrate a payment service to handle payments securely.
- **Notifications**: Send notifications to users about order status changes (e.g., when a driver is on the way).

### Non-Functional Requirements
- **Scalability**: Must handle up to 10,000 orders per second (QPS) during peak hours.
- **Availability**: 99.99% uptime to ensure users can place orders at any time.
- **Latency**: Order creation should take less than 200ms under normal load.
- **Consistency**: Strong consistency for payment processing to avoid double charges.

### Back-of-Envelope Calculations
- **User Base**: Assume 10 million active users.
- **Peak Traffic**: 10% of users place an order during peak times, giving 1 million concurrent users.
- **Order Rate**: Assuming each user places an order every 10 minutes, that's 6 orders per hour per user.
- **Total Orders per Hour**: 1 million users * 6 orders = 6 million orders/hour, which translates to 1,666 orders/second.
- **Scaling Factor**: To handle spikes, we estimate a 6x scaling factor, resulting in a target of 10,000 orders/second.

### Component Diagram
- **API Gateway**: NGINX to route requests to microservices.
- **Order Service**: Handles order creation and tracking, built with Node.js for non-blocking I/O.
- **Payment Service**: Java-based microservice that interfaces with payment gateways.
- **Notification Service**: Uses AWS SNS for sending notifications.
- **Database**: PostgreSQL for transactional data; Redis for caching frequently accessed data.
- **Message Queue**: Apache Kafka for asynchronous processing of order events.
- **Load Balancer**: AWS Elastic Load Balancer to distribute incoming traffic.

![High-Level Architecture Diagram](https://example.com/architecture-diagram)

### Schema Design
- **Orders Table**:  
  - `order_id` (Primary Key)  
  - `user_id`  
  - `status` (Pending, In-Progress, Completed, Cancelled)  
  - `created_at`  
  - `updated_at`

- **Payments Table**:  
  - `payment_id` (Primary Key)  
  - `order_id` (Foreign Key)  
  - `amount`  
  - `status` (Pending, Completed, Failed)  
  - `transaction_time`

- **Caching Strategy**:  
  - Use Redis to cache order statuses for fast lookups (TTL of 5 minutes).

### Architectural Decisions
- **Consistency vs. Availability**: Chose eventual consistency for the order status updates to ensure high availability. Payment processing requires strong consistency, hence a synchronous approach is used here.
- **Caching Decisions**: Opted for Redis for fast access but balanced it with a TTL to avoid stale data. Trade-off between read speed and potential stale reads.
- **Message Queue**: Using Apache Kafka allows for decoupling services and handling spikes in traffic, but introduces complexity in message ordering and delivery guarantees. 
- **Failure Handling**: Implement retries for failed payment processes, and use circuit breakers to prevent cascading failures in the order processing flow. Monitoring via Prometheus and alerting with Grafana for operational insights.

Design Order Processing Infrastructure for microservices

Lyft

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Back-of-Envelope Calculations

Submit Your Answer

Lyft Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search