Design a Order Processing Service

Question

Design a event-driven order processing system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Order Submission**: Users can submit buy/sell orders through a RESTful API.
2. **Order Status Tracking**: Users can query the status of their orders at any time.
3. **Event Notifications**: Users receive real-time updates on order status via WebSockets.
4. **Order Modification**: Users can modify or cancel orders before execution.
5. **Audit Logs**: All actions (submissions, modifications, cancellations) should be logged for compliance.

### Non-Functional Requirements
1. **Scalability**: The system should handle millions of requests per second (QPS).
2. **Availability**: The system should have a 99.99% uptime to ensure trading continuity.
3. **Latency**: Order processing should complete within 100 milliseconds.
4. **Security**: All API endpoints must be secured using OAuth 2.0 and TLS encryption.
5. **Data Consistency**: Strong consistency for order placement and modifications to avoid race conditions.

To estimate capacity, we consider the following factors:

1. **Expected Daily Trading Volume**: Assume Morgan Stanley handles approximately 1 billion orders daily.
2. **Peak Traffic**: Assume peak hours account for 20% of the daily volume. Thus, peak order submissions would be around 200 million orders per hour or about 55,556 orders per second (QPS).
3. **Data Storage**: Each order typically requires 1 KB of storage. Thus, total storage per day is around 1 TB (1 billion orders x 1 KB). For a week, this would be approximately 7 TB, requiring scalable storage solutions.
4. **Network Bandwidth**: If we consider 500 bytes of additional metadata per order for processing, the total bandwidth required for peak load would be around 27.78 MB/s (55,556 orders/sec x 500 bytes).

### Components:
1. **API Gateway**: To handle incoming API requests, rate limiting, and authentication (AWS API Gateway).
2. **Order Service**: Responsible for processing orders, applying business rules (Spring Boot).
3. **Message Queue**: To decouple services and handle asynchronous processing (Apache Kafka).
4. **Database**: A distributed SQL database (CockroachDB) for strong consistency and horizontal scaling.
5. **Cache**: Redis for caching frequently accessed order data to improve read performance.
6. **Notification Service**: For real-time updates to clients via WebSockets (Node.js).
7. **Monitoring and Logging**: Using Prometheus for monitoring and ELK stack for logging and alerting.

### Schema Design
1. **Orders Table**: 
   - `order_id` (UUID, Primary Key)
   - `user_id` (UUID, Foreign Key)
   - `symbol` (String)
   - `quantity` (Integer)
   - `price` (Decimal)
   - `status` (Enum: Pending, Executed, Canceled)
   - `created_at` (Timestamp)
   - `updated_at` (Timestamp)

2. **Users Table**: 
   - `user_id` (UUID, Primary Key)
   - `username` (String)
   - `password_hash` (String)
   - `account_balance` (Decimal)
   - `created_at` (Timestamp)

### Access Patterns
- Orders are primarily accessed by `user_id` for fetching user order history and current statuses. The system should optimize for quick lookups by user and order status.

### Architectural Decisions
1. **Consistency vs. Availability**: We prioritize strong consistency for order placement to prevent issues like double trading. This may lead to reduced availability during peak loads.
2. **Message Queue**: Using Kafka allows for decoupling of services, but introduces eventual consistency for order processing, requiring careful handling of order state.
3. **Database Choice**: Choosing CockroachDB offers strong consistency but may involve complexity in partitioning and replication, which could impact latency.
4. **Caching**: While Redis improves read performance, it introduces potential stale data issues, necessitating strategies for cache invalidation.
5. **Monitoring**: Implementing comprehensive monitoring and alerting is crucial for operational readiness, but adds complexity in terms of managing alert fatigue.

Design a Order Processing Service

Morgan Stanley

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Morgan Stanley Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search