Architect a real-time Payment Engine

Question

Design a real-time payment system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Transaction Processing**: Support for various payment methods (credit/debit cards, PayPal, etc.) and currencies.
2. **Real-time Processing**: Transactions must be processed within seconds to enhance user experience.
3. **Fraud Detection**: Implement a system to detect and prevent fraudulent transactions in real-time.
4. **User Notifications**: Users should receive immediate notifications for transaction statuses (success, failure, pending).
5. **API Integration**: Provide a robust API for third-party integrations and internal services.

### Non-Functional Requirements
1. **Scalability**: The system should handle up to 10 million transactions per day.
2. **High Availability**: The system must be operational 99.99% of the time.
3. **Data Consistency**: Ensure eventual consistency for transaction records.
4. **Latency**: Target latency of less than 200 ms for transaction processing.
5. **Security**: Implement encryption and compliance with standards such as PCI-DSS.

To estimate the capacity:
- **Peak Transactions**: Assume peak load of 10 million transactions per day, translating to approximately 115 transactions per second (TPS).
- **Data Storage**: Assuming each transaction record is around 1 KB, the daily storage requirement would be 10 million KB or about 10 GB per day.
- **Data Retention**: If we retain transaction data for 5 years, we need approximately 18.25 TB of storage for transaction records.
- **Throughput**: The system should handle bursts of up to 500 TPS during peak times (e.g., promotions) without degrading performance.

### Component Diagram
- **Client Layer**: Mobile and web clients initiate payment requests.
- **API Gateway**: Routes requests to appropriate services and handles load balancing.
- **Payment Service**: Core service managing transaction processing using Node.js.
- **Fraud Detection Service**: Machine learning model, hosted on AWS SageMaker, scoring transactions in real-time.
- **Database**: Use PostgreSQL for ACID transactions and Redis for caching frequently accessed data.
- **Notification Service**: Implemented with AWS SNS to send real-time notifications to users.
- **Monitoring and Logging**: Use Prometheus for monitoring and ELK stack for logging.

### Schema Design
- **Transactions Table**: 
  - `transaction_id` (UUID, Primary Key)
  - `user_id` (UUID, Foreign Key)
  - `amount` (Decimal)
  - `currency` (String)
  - `status` (Enum: 'pending', 'completed', 'failed')
  - `created_at` (Timestamp)
  - `updated_at` (Timestamp)

- **Users Table**: 
  - `user_id` (UUID, Primary Key)
  - `email` (String, Unique)
  - `hashed_password` (String)
  - `created_at` (Timestamp)
  - `updated_at` (Timestamp)

- **Fraud Scores Table**: 
  - `transaction_id` (UUID, Foreign Key)
  - `fraud_score` (Integer)
  - `model_version` (String)
  - `scored_at` (Timestamp)

### Access Patterns
- Frequent reads for transaction status and fraud scores.
- Write-heavy during peak transaction times.

### Architectural Decisions
1. **Consistency vs Availability**: Opting for eventual consistency for transaction records to improve availability and user experience, especially during high load.
2. **Storage vs Performance**: Using PostgreSQL for structured transaction data ensures ACID compliance, while Redis is used for caching to reduce latency for frequently accessed data.
3. **Model Freshness vs Complexity**: Implementing a canary deployment strategy for ML models allows for gradual rollouts and monitoring of model performance without impacting the entire system.
4. **Cold Start Problem**: To address cold starts for new users, we implement a default fraud score algorithm based on historical data, gradually updating it as more user-specific data becomes available.
5. **Exploration/Exploitation**: A/B testing framework to evaluate new models or features while balancing user experience and revenue generation.

Architect a real-time Payment Engine

Reddit

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Reddit Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search