System requirements

Functional Requirements

Core Functionalities:

User Registration and Authentication:
- Support for creating accounts using email, phone, or social media.
- Enable secure user authentication via passwords, OTPs, or biometric authentication.
Wallet Management:
- Add money to the wallet using linked bank accounts, credit/debit cards, or UPI.
- View wallet balance and transaction history.
Payments:
- Make online payments to merchants using wallet balance or linked accounts.
- Enable QR code scanning for offline payments.
- Support recurring payments for subscriptions or bills.
Peer-to-Peer (P2P) Transfers:
- Send and receive money from other wallet users.
- Add personalized notes or messages for each transaction.
Integration with Financial Services:
- Link bank accounts, cards, and UPI IDs.
- Provide features like savings accounts, loans, or investments.
Merchant Services:
- Allow merchants to accept payments and track transactions.
- Generate dynamic QR codes for payments.
Notifications and Alerts:
- Notify users of successful transactions, low balance, or promotional offers.
- Provide real-time updates for transactions.
Dispute Resolution and Refunds:
- Handle failed transactions, refunds, and chargebacks efficiently.

Non-Functional Requirements

Scalability:
- Support millions of users and thousands of transactions per second during peak times.
Security:
- Ensure end-to-end encryption for all transactions.
- Comply with PCI-DSS standards for secure payment processing.
High Availability:
- Guarantee 99.99% uptime for uninterrupted service.
Performance:
- Process transactions in real time with latency under 500ms.
Data Integrity:
- Maintain consistency for balances and transaction records across distributed systems.
Extensibility:
- Allow easy integration with third-party financial services and APIs.
Monitoring and Logging:
- Implement real-time monitoring of system health and transaction flows.

Capacity estimation

Estimate the scale of the system you are going to design...

Assumptions:

Users:
- Total registered users: 100 million.
- Daily active users: 10 million (10%).
Transactions:
- Average transactions per user per day: 5.
- Total daily transactions: 10M×5=50 million10M \times 5 = 50 \, \text{million}10M×5=50million.
- Peak transactions per second (TPS): 10,000.
Storage:
- Average transaction size: 500 bytes.
- Daily transaction storage: 50M×500 bytes=25 GB/day50M \times 500 \, \text{bytes} = 25 \, \text{GB/day}50M×500bytes=25GB/day.
Growth:
- Anticipated annual user growth: 25%.
- Storage requirement for transactions over a year: 25 GB/day×365=9 TB/year25 \, \text{GB/day} \times 365 = 9 \, \text{TB/year}25GB/day×365=9TB/year.

API design

Define what APIs are expected from the system...

1. User Management APIs

POST /api/users/register: Create a new user account.
POST /api/users/login: Authenticate a user.
GET /api/users/profile: Fetch user profile and linked accounts.

2. Wallet Management APIs

POST /api/wallet/add_money: Add money to the wallet.
GET /api/wallet/balance: Check wallet balance.
GET /api/wallet/history: Retrieve transaction history.

3. Payments APIs

POST /api/payments/online: Make an online payment to a merchant.
POST /api/payments/offline: Make a QR code payment for offline transactions.
POST /api/payments/recurring: Set up recurring payments.

4. P2P Transfer APIs

POST /api/transfer/send: Transfer money to another user.
GET /api/transfer/receive: Fetch incoming transfer requests.

5. Financial Integration APIs

POST /api/bank/link_account: Link a bank account to the wallet.
GET /api/bank/details: Fetch linked bank account details.
POST /api/bank/withdraw: Withdraw wallet balance to a bank account.

6. Merchant Services APIs

POST /api/merchants/register: Register a merchant account.
GET /api/merchants/transactions: Fetch merchant transaction history.
POST /api/merchants/qrcode: Generate a dynamic QR code for payments.

7. Notifications APIs

POST /api/notifications/send: Send transaction notifications.
GET /api/notifications: Retrieve past notifications.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

1. User Database

Schema Details:
- Table Name: Users
  - user_id (Primary Key): Unique identifier for each user.
  - name: Full name of the user.
  - email: Email address.
  - phone_number: Phone number.
  - password_hash: Hashed password.
  - created_at: Account creation timestamp.
Purpose:
- Store user details and authentication credentials.
Tech Used:
- Relational Database (e.g., PostgreSQL).
Tradeoff:
- Pros: Strong consistency for user-related operations.
- Cons: Requires sharding for scalability as user base grows.

2. Wallet Database

Schema Details:
- Table Name: Wallets
  - wallet_id (Primary Key): Unique identifier for each wallet.
  - user_id (Foreign Key): Associated user ID.
  - balance: Current wallet balance.
  - last_updated: Timestamp of the last balance update.
Purpose:
- Maintain wallet balances and track updates.
Tech Used:
- Relational Database (e.g., MySQL).
Tradeoff:
- Pros: Ensures transactional consistency for balance updates.
- Cons: High contention during concurrent transactions.

3. Transaction Database

Schema Details:
- Table Name: Transactions
  - transaction_id (Primary Key): Unique identifier for each transaction.
  - user_id (Foreign Key): Associated user ID.
  - amount: Transaction amount.
  - transaction_type: Type (e.g., credit, debit).
  - status: Transaction status (e.g., success, failed).
  - timestamp: Time of the transaction.
Purpose:
- Store details of all wallet transactions.
Tech Used:
- NoSQL Database (e.g., MongoDB).
Tradeoff:
- Pros: High write throughput for frequent transactions.
- Cons: Complex querying for analytical operations.

4. Merchant Database

Schema Details:
- Table Name: Merchants
  - merchant_id (Primary Key): Unique identifier for each merchant.
  - name: Merchant name.
  - email: Merchant email.
  - phone_number: Merchant phone number.
  - qrcode: Static or dynamic QR code for payments.
Purpose:
- Manage merchant details and payment configurations.
Tech Used:
- Relational Database (e.g., PostgreSQL).
Tradeoff:
- Pros: Supports complex relationships with transaction data.
- Cons: Requires indexing for fast lookup of merchant data.

5. Notification Database

Schema Details:
- Table Name: Notifications
  - notification_id (Primary Key): Unique identifier for each notification.
  - user_id (Foreign Key): Associated user ID.
  - message: Notification content.
  - timestamp: Time of notification delivery.
Purpose:
- Store and track notifications sent to users.
Tech Used:
- NoSQL Database (e.g., DynamoDB).
Tradeoff:
- Pros: Optimized for high-frequency reads and writes.
- Cons: Limited query flexibility for complex filters.

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

1. User Management Service

Overview:

Handles user registration, authentication, and profile management. Ensures secure access to the wallet and related features.

Responsibilities:

Manage user accounts and credentials.
Authenticate users via passwords, OTPs, or biometric data.
Enable linking and managing of bank accounts and cards.

2. Wallet Service

Overview:

Manages the wallet’s balance, transactions, and fund transfers. Ensures accurate updates during concurrent transactions.

Responsibilities:

Maintain wallet balances and handle money transfers.
Provide transaction history and analytics.
Process refunds and resolve disputes.

3. Payment Gateway Integration

Overview:

Handles secure payment processing for adding funds, making payments, and settling with merchants.

Responsibilities:

Integrate with third-party payment gateways (e.g., Stripe, PayPal).
Process transactions and update statuses in real-time.
Support multiple currencies and payment methods.

4. Peer-to-Peer (P2P) Transfer Service

Overview:

Facilitates money transfers between wallet users. Ensures real-time updates and secure transactions.

Responsibilities:

Enable sending and receiving money between users.
Provide transaction verification and notifications.
Handle transfer limits and fraud detection.

5. Merchant Management Service

Overview:

Allows merchants to register, generate QR codes, and accept payments. Provides detailed transaction records for reconciliation.

Responsibilities:

Register and authenticate merchants.
Manage merchant payment configurations and QR codes.
Generate reports for transaction reconciliation.

6. Notifications Service

Overview:

Manages alerts and notifications for transactions, refunds, and promotional messages.

Responsibilities:

Notify users and merchants of transaction statuses.
Send alerts for low balance or payment failures.
Track notification delivery status.

7. Analytics Service

Overview:

Tracks and analyzes wallet activities, such as transaction volume, user engagement, and merchant performance.

Responsibilities:

Provide dashboards for users and merchants.
Generate insights on spending trends and payment patterns.
Support exportable reports for audits and marketing.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

1. User Registration Request

Objective: Register a new user account.

Steps:

API Gateway:
- Receives a POST /api/users/register request with user details.
- Validates the input and forwards it to the User Management Service.
User Management Service:
- Verifies the email/phone number for uniqueness.
- Hashes the user password for secure storage.
- Creates a new user record in the User Database.
Wallet Service:
- Initializes a wallet for the user with a zero balance.
- Links the wallet ID to the user’s account.
Response:
- Confirms account creation and provides login details.

2. Add Money to Wallet Request

Objective: Add funds to a user’s wallet.

Steps:

API Gateway:
- Receives a POST /api/wallet/add_money request with the amount and payment method.
- Authenticates the user and forwards the request to the Wallet Service.
Wallet Service:
- Validates the requested amount and checks for account limits.
- Initiates a payment request with the Payment Gateway Integration.
Payment Gateway Integration:
- Processes the payment and confirms success or failure.
- Sends the payment status back to the Wallet Service.
Wallet Service:
- Updates the wallet balance for successful payments.
- Logs the transaction in the Transaction Database.
Response:
- Returns the updated wallet balance and transaction receipt.

3. Peer-to-Peer (P2P) Transfer Request

Objective: Send money to another wallet user.

Steps:

API Gateway:
- Receives a POST /api/transfer/send request with the recipient’s details and amount.
- Authenticates the sender and forwards the request to the P2P Transfer Service.
P2P Transfer Service:
- Validates the sender’s balance and the recipient’s existence.
- Deducts the amount from the sender’s wallet and credits it to the recipient’s wallet.
- Logs the transaction in the Transaction Database.
Notifications Service:
- Notifies both sender and recipient of the transaction details.
Response:
- Confirms the transfer with a transaction receipt.

4. Make Payment to Merchant Request

Objective: Process a payment for goods or services.

Steps:

API Gateway:
- Receives a POST /api/payments/online request with the merchant ID and amount.
- Authenticates the user and forwards the request to the Wallet Service.
Wallet Service:
- Validates the user’s balance and the merchant’s ID.
- Deducts the payment amount from the wallet balance.
- Notifies the Merchant Management Service of the transaction.
Merchant Management Service:
- Credits the payment to the merchant’s account.
- Logs the transaction in the Merchant Database.
Notifications Service:
- Notifies the user and merchant of the transaction status.
Response:
- Confirms payment and provides a receipt.

5. Transaction History Request

Objective: Retrieve a user’s transaction history.

Steps:

API Gateway:
- Receives a GET /api/wallet/history request.
- Authenticates the user and forwards the request to the Wallet Service.
Wallet Service:
- Queries the Transaction Database for the user’s transaction records.
- Formats and returns the transaction history.
Response:
- Provides a paginated list of transactions to the user.

6. Notification for Low Balance

Objective: Alert users about low wallet balance.

Steps:

Wallet Service:
- Periodically checks user balances against predefined thresholds.
- Triggers a notification event for low balances.
Notifications Service:
- Sends an alert via email, SMS, or push notification.
Response:
- Confirms notification delivery status.

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

1. User Management Service

End-to-End Working:

The User Management Service is responsible for user registration, authentication, and profile management. It handles user onboarding, validates credentials during login, and manages user sessions. When a new user registers, the service validates the data (e.g., email, phone number) for uniqueness, hashes passwords for secure storage, and creates an account. Upon login, it verifies credentials, generates a session token, and updates login activity.

Communication:

Protocols Used:
- HTTP/HTTPS: For client requests, such as user registration and login.
- REST APIs: Communicates with the Notification Service to send verification emails/SMS.
- gRPC: For low-latency communication with other services, such as the Wallet Service.
Inter-Service Communication:
- Notifies the Wallet Service to initialize a wallet during registration.
- Sends analytics data to the Analytics Service for monitoring user activity.

Data Structures and Algorithms:

Hash Table for Credential Caching:
- Uses an in-memory hash table (e.g., Redis) to cache hashed credentials for faster login verification.
Password Hashing:
- Implements bcrypt or Argon2 for secure password storage, resistant to brute-force attacks.
JWT Tokens for Session Management:
- Generates JSON Web Tokens (JWT) for secure, stateless session handling.

Implementation Detail (Password Hashing):

python
Copy code
from bcrypt import hashpw, gensalt, checkpw

def hash_password(password):
    return hashpw(password.encode('utf-8'), gensalt())

def verify_password(password, hashed):
    return checkpw(password.encode('utf-8'), hashed)

Scaling for Peak Traffic:

Horizontal Scaling:
- Stateless service instances are deployed behind a load balancer to handle concurrent login/registration requests.
Caching:
- Uses Redis for frequently accessed data, such as active sessions and user profiles.
Rate Limiting:
- Implements token bucket algorithms to throttle excessive requests from the same IP.

Edge Cases:

Duplicate Accounts:
- Enforces unique constraints on email and phone fields in the database.
Forgotten Passwords:
- Generates time-limited reset tokens for secure password recovery.
Session Expiry:
- Ensures JWT tokens expire after a configurable period to enhance security.

2. Wallet Service

End-to-End Working:

The Wallet Service manages user balances, processes fund transfers, and logs transactions. When a user adds money, the service deducts the amount from the linked bank/card via the Payment Gateway Integration and credits the wallet. It also verifies balance sufficiency before transfers and ensures atomic updates to prevent data inconsistency.

Communication:

Protocols Used:
- HTTP/HTTPS: Handles requests to add money, check balances, or transfer funds.
- REST APIs: Communicates with the Payment Gateway Integration for fund debits/credits.
- Message Queues (e.g., Kafka): Publishes transaction events for analytics and notifications.
Inter-Service Communication:
- Retrieves user details from the User Management Service.
- Sends transaction logs to the Analytics Service.

Data Structures and Algorithms:

Distributed Locks for Concurrent Transactions:
- Prevents race conditions during balance updates by using Redis locks.
ACID Transactions:
- Ensures atomicity by wrapping balance updates and transaction logging in a database transaction.
Ledger Model for Transactions:
- Maintains a double-entry ledger to track debits and credits for each wallet.

Implementation Detail (Ledger Model):

python
Copy code
class Ledger:
    def __init__(self):
        self.entries = []

    def add_entry(self, transaction_id, user_id, amount, transaction_type):
        self.entries.append({
            "transaction_id": transaction_id,
            "user_id": user_id,
            "amount": amount,
            "transaction_type": transaction_type
        })

Scaling for Peak Traffic:

Sharding:
- Shards wallet data by user ID to distribute load across multiple database nodes.
Asynchronous Processing:
- Queues non-critical operations (e.g., analytics updates) for background processing.
Autoscaling:
- Dynamically scales service instances based on transaction volume.

Edge Cases:

Overdraft Attempts:
- Rejects transfers if the wallet balance is insufficient.
Duplicate Transactions:
- Implements idempotency checks using unique transaction IDs.
Reconciliation Errors:
- Periodically reconciles wallet balances with transaction logs to detect inconsistencies.

3. Payment Gateway Integration

End-to-End Working:

This service acts as a bridge between the Wallet Service and external payment gateways. It handles payments for adding funds, processing refunds, and making payments to merchants. It ensures secure communication with third-party gateways, validates transactions, and updates payment statuses.

Communication:

Protocols Used:
- HTTPS: Secure communication with external payment gateways.
- REST APIs: Receives payment requests from the Wallet Service and sends responses.
Inter-Service Communication:
- Notifies the Wallet Service of successful or failed payments.
- Sends transaction metadata to the Analytics Service for reporting.

Data Structures and Algorithms:

Retry Mechanism for Failed Payments:
- Implements exponential backoff for retrying failed transactions.
Tokenization for Sensitive Data:
- Replaces card/bank details with tokens for secure storage and reduced PCI compliance scope.
HMAC for Payment Integrity:
- Uses hash-based message authentication codes (HMAC) to validate the integrity of payment requests.

Implementation Detail (Tokenization):

python
Copy code
import hashlib

def tokenize(data):
    return hashlib.sha256(data.encode()).hexdigest()

Scaling for Peak Traffic:

Multi-Region Deployment:
- Deploys service instances across regions to reduce latency for global users.
Batch Processing:
- Queues bulk transactions (e.g., refunds) for batch processing during non-peak hours.
Horizontal Scaling:
- Scales instances based on incoming payment requests.

Edge Cases:

Gateway Downtime:
- Implements fallback to secondary gateways for high availability.
Currency Mismatch:
- Provides real-time currency conversion for cross-border transactions.
Fraudulent Activities:
- Integrates AI-based fraud detection systems to flag suspicious payments.

4. Peer-to-Peer (P2P) Transfer Service

End-to-End Working:

The P2P Transfer Service allows users to send and receive money. It validates sender and recipient accounts, ensures sufficient balance, and updates both wallets atomically. It also logs transactions and notifies users of transfer details.

Communication:

Protocols Used:
- REST APIs: Handles fund transfer requests.
- Message Queues: Publishes transaction events for analytics and notifications.
Inter-Service Communication:
- Updates wallet balances in the Wallet Service.
- Sends notifications to the Notifications Service.

Data Structures and Algorithms:

Distributed Ledger:
- Maintains a ledger to track debits from senders and credits to recipients.
Hash-Based Transaction IDs:
- Generates unique transaction IDs for idempotency.

Implementation Detail (P2P Transfer):

python
Copy code
def transfer_funds(sender_id, recipient_id, amount):
    sender_wallet = get_wallet(sender_id)
    recipient_wallet = get_wallet(recipient_id)

    if sender_wallet.balance >= amount:
        sender_wallet.balance -= amount
        recipient_wallet.balance += amount
        log_transaction(sender_id, recipient_id, amount)
    else:
        raise ValueError("Insufficient Balance")

Scaling for Peak Traffic:

Partitioning:
- Partitions transaction logs by user ID to distribute load across database nodes.
Eventual Consistency:
- Uses eventual consistency for non-critical updates, such as analytics.

Edge Cases:

Invalid Recipients:
- Validates recipient accounts before initiating transfers.
Double Spending:
- Uses distributed locks to prevent concurrent transfers from overdrawing the balance.
Delayed Transfers:
- Notifies users of delays during high traffic and provides estimated completion times.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Microservices Architecture:

Trade-off: Increased complexity in deployment and inter-service communication.
Reason: Enables independent scaling, fault isolation, and easier integration of new features.

NoSQL for Transactions:

Trade-off: Limited query flexibility compared to relational databases.
Reason: Provides high write throughput and horizontal scalability for frequent transactions.

Distributed Locks for Wallet Updates:

Trade-off: Slight increase in latency due to lock acquisition.
Reason: Ensures consistency during concurrent updates.

Tokenization for Sensitive Data:

Trade-off: Additional processing overhead for token generation and validation.
Reason: Enhances security and reduces PCI compliance scope.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Concurrent Wallet Updates:

Issue: Race conditions during concurrent balance updates.
Mitigation: Use distributed locks or optimistic concurrency control.

Payment Gateway Downtime:

Issue: External payment gateway becomes unavailable.
Mitigation: Implement fallback mechanisms to secondary gateways and queue transactions for retry.

High Transaction Volume:

Issue: System overload during peak hours.
Mitigation: Autoscale services and use partitioning for database sharding.

Fraudulent Transactions:

Issue: Unauthorized or suspicious payments.
Mitigation: Integrate AI-based fraud detection systems and implement transaction velocity checks.

Data Loss in Message Queues:

Issue: Overloaded queues may drop events.
Mitigation: Use dead-letter queues and redundant queues for critical events.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

AI-Powered Fraud Detection:

Improvement: Use machine learning models to detect unusual transaction patterns.
Mitigation: Reduce fraudulent activities and enhance user trust.

Predictive Autoscaling:

Improvement: Implement predictive scaling based on historical data and traffic patterns.
Mitigation: Prevent service degradation during unexpected traffic spikes.

Blockchain for Ledger Management:

Improvement: Use blockchain for tamper-proof transaction records.
Mitigation: Enhance transparency and traceability for wallet transactions.

Enhanced Monitoring and Alerting:

Improvement: Deploy real-time monitoring dashboards with automated anomaly detection.
Mitigation: Quickly detect and resolve issues to minimize downtime.

Cross-Border Payment Support:

Improvement: Integrate international payment gateways and real-time currency conversion.
Mitigation: Provide seamless cross-border payment experiences for global users.

Solutions

nectar4678 - 07/06/2024

1

iridescent_luminous693 - 01/07/2025

0