System requirements

Functional:

Functional requirements describe what the system should do. Here are some suggested requirements for our system:

Transaction Scheduling: Users should be able to schedule different types of transactions including one-time, recurring, and conditional payments.
User Management: The system should manage user accounts including authentication, authorization, and password management.
Notification System: Users should receive notifications regarding the status of their transactions, including successes and failures.
Admin Panel: For back-end management, including transaction oversight and user account management.
Reporting and Analytics: Generate reports for transactions, spotting trends, and tracking transaction history.
Security Features: Ensure data encryption, secure data storage, and secure transmission protocols.

Non-Functional:

Non-functional requirements describe how the system operates, underlining the quality attributes of the system:

Scalability: The system must handle a high number of transactions and users without degradation of performance.
Reliability: Achieve high availability with minimal downtime.
Usability: Interface must be user-friendly for both end-users and administrators.
Security: Implement robust security measures to protect against fraud and cyber threats.
Performance: Transactions should be processed in a timely fashion, with real-time processing capabilities as needed.
Maintainability: Code and systems should be designed for easy maintenance and future upgrades.

Capacity estimation

Assumptions

To estimate the system's capacity, we need to make a few assumptions about usage patterns and transaction volumes:

Number of Users: Estimate the number of active users who will use the system. For instance, we might start with 100,000 active users.
Transaction Frequency: Estimate how often these users will schedule transactions. Let’s assume each user schedules 3 transactions monthly.
Peak Load: Identify peak hours during which transaction volume might double.

Calculation

Based on the above assumptions:

Monthly transactions = 100,000 users × 3 transactions/user = 300,000 transactions per month.
If we estimate peak hours to see 10% of daily transactions, then,
Daily transactions = 300,000 transactions / 30 days ≈ 10,000 transactions.
Peak hour transactions = 10% of daily transactions = 1,000 transactions in the peak hour.

System Requirements

To handle these transactions smoothly:

Database Operations: Capable of handling at least 50 writes per second during peak times (as transactions might involve multiple database interactions).
Storage: Sufficient storage for user data, transaction logs, and backups. Assuming small metadata per transaction, storage should scale with user base and retention policy.
Network Bandwidth: Ensure adequate bandwidth to support data transmission needs during peak loads.

API design

The APIs will be structured to facilitate core functionalities such as user management, transaction scheduling, and system administration.

User Authentication API

Authenticate users and provide tokens for session management.

POST /api/auth

Request Body:
{
  "username": "user@example.com",
  "password": "password123"
}

Response:
{
  "token": "ey123.ey456.789xyz"
}

Schedule Transaction API

Schedule a new transaction for execution.

POST /api/transactions/schedule

Request Body:
{
  "user_id": 1,
  "amount": 100.00,
  "currency": "USD",
  "transaction_type": "one-time",
  "scheduled_date": "2024-10-20"
}

Response:
{
  "transaction_id": 12345,
  "status": "scheduled"
}

Check Transaction Status API

Retrieve the status of a scheduled transaction.

GET /api/transactions/status

Request Param:
{
  "transaction_id": 12345
}

Response:
{
  "transaction_id": 12345,
  "status": "completed",
  "execution_date": "2024-10-20",
  "details": "Payment of $100 to account XYZ"
}

Admin API to Oversee Transactions

View all transactions within the system for monitoring and management purposes.

GET /api/admin/transactions

Response:
{
  "transactions": [
    {
      "transaction_id": 12345,
      "user_id": 1,
      "status": "completed",
      "amount": 100.00,
      "currency": "USD",
      "scheduled_date": "2024-10-20"
    }
  ]
}

Database design

We will use a relational database model for our system. Here's a proposed Entity-Relationship (ER) diagram and corresponding descriptions:

Entities:

User: Stores information about system users.
Transaction: Contains all details about transactions scheduled by users.
Transaction_Log: Captures logs of all transaction activities for auditing and troubleshooting.

Attributes:

User: user_id, username, email, hashed_password, created_at
Transaction: transaction_id, user_id, amount, currency, transaction_type, status, scheduled_date, executed_date
Transaction_Log: log_id, transaction_id, status, timestamp, details

Relationships:

A User can have multiple Transactions (One-to-Many).
A Transaction generates multiple Transaction_Logs (One-to-Many).

erDiagram
    User ||--o{ Transaction : schedules
    Transaction ||--o{ Transaction_Log : logs


    User {
        int user_id PK
        string username
        string email
        string hashed_password
        datetime created_at
    }


    Transaction {
        int transaction_id PK
        int user_id FK
        float amount
        string currency
        string transaction_type
        string status
        date scheduled_date
        date executed_date
    }


    Transaction_Log {
        int log_id PK
        int transaction_id FK
        string status
        datetime timestamp
        string details
    }

High-level design

The system architecture will be based on a microservices model to ensure scalability, maintainability, and robust fault isolation. Here's a breakdown of the primary components:

Key Components:

User Service: Manages user registration, authentication, and profile management.
Transaction Service: Handles all aspects of transaction scheduling, execution, and status updates.
Notification Service: Sends out alerts and notifications to users regarding transaction statuses and other important events.
Admin Service: Provides administrative capabilities to manage the system, oversee transaction logs, and generate reports.
Database: Central storage for user data, transactions, and logs.
API Gateway: Entry point for all client requests, routing them to the appropriate services.
Load Balancer: Distributes incoming traffic and requests across servers to ensure reliable and efficient performance.

graph TD
    Client[Client] -->|Request| AG[API Gateway]
    AG --> LB[Load Balancer]
    LB -->|Distribute Request| US[User Service]
    LB -->|Distribute Request| TS[Transaction Service]
    LB -->|Distribute Request| NS[Notification Service]
    LB -->|Distribute Request| AS[Admin Service]
    US -->|Read/Write| DB[(Database)]
    TS -->|Read/Write| DB
    NS -->|Read/Write| DB
    AS -->|Read/Write| DB
    US -->|Auth| Auth[Authentication Service]
    DB -->|Log Activity| Log[Logging Service]
    DB -->|Monitor DB| Mon[Monitoring Service]
    Mon -.->|Alert| NS
    Log -.->|Store Logs| ES[External Storage]
    Auth -->|Validate| Sec[Security Service]
    Sec -->|Token| AG

Component Interaction:

Client requests (e.g., scheduling a transaction or checking transaction status) are sent to the API Gateway, which routes them to the appropriate service.
The Load Balancer efficiently distributes incoming requests across multiple instances of each service to handle high loads and increase fault tolerance.
Each service interacts with the Database to retrieve or store data, ensuring that all transaction records are consistently managed and maintained.

Request flows

We'll use a sequence diagram to represent the flow for a typical transaction scheduling request, from the client making the request through the system until the transaction is scheduled in the database.

Request Flow Description:

Client Submission: A user submits a transaction scheduling request via a client interface (web or mobile).
API Gateway: The request hits the API Gateway, which authenticates the user and forwards the request to the Load Balancer.
Load Balancer: Distributes the request to one of the available Transaction Service instances.
Transaction Service: Processes the request, checks transaction rules and parameters, and then interacts with the Database to store the transaction details.
Database: Records the transaction and confirms back to the Transaction Service.
Notification Service: Optionally, the Transaction Service might call the Notification Service to send a confirmation message to the user about the scheduled transaction.
Response to Client: Finally, the Transaction Service sends a response back to the client through the API Gateway, confirming that the transaction has been scheduled.

sequenceDiagram
    participant Client
    participant API_Gateway as API Gateway
    participant Load_Balancer as Load Balancer
    participant Transaction_Service as Transaction Service
    participant Database
    participant Notification_Service as Notification Service


    Client->>+API_Gateway: Submit transaction request
    API_Gateway->>+Load_Balancer: Forward request
    Load_Balancer->>+Transaction_Service: Route request
    Transaction_Service->>+Database: Store transaction details
    Database-->>-Transaction_Service: Confirm storage
    Transaction_Service->>+Notification_Service: Send notification (optional)
    Notification_Service-->>-Transaction_Service: Confirm send
    Transaction_Service-->>-API_Gateway: Return confirmation
    API_Gateway-->>-Client: Display result

Detailed Component Design

Transaction Service

Functionality:

Manages all aspects of transactions including creation, scheduling, and status updating.
Validates transaction conditions and schedules based on user input.

Scalability:

Designed as a stateless service to scale horizontally. New instances can be added to handle increased load.
Utilizes a distributed queue (e.g., Kafka or RabbitMQ) for handling transaction execution tasks, allowing decoupling of transaction scheduling from execution.

graph TD
    TS[Transaction Service] -->|Validate Input| Q[Queue Manager]
    TS -->|Schedule Transactions| PQ[Priority Queue]
    TS -->|Store Transaction| DB[Database]
    Q -->|Dispatch to Queue| MQ[Message Queue]
    MQ -->|Handle Execution| Executor[Transaction Executor]
    PQ -->|Manage Order| Executor
    DB -->|Retrieve/Update Status| TS

Algorithms/Data Structures:

Uses a priority queue based on scheduled time for managing when transactions are executed, ensuring that transactions are processed in the correct order.
Hash tables are used to store transaction details for quick access and status updates.

Notification Service

Functionality:

Sends notifications to users regarding transaction statuses (e.g., scheduled, completed, failed).
Handles both email and SMS notifications depending on user preferences.

Scalability:

The service can scale independently based on notification volume, separate from transaction processing.
Uses a publish-subscribe model with a message broker to handle notification requests, which allows it to efficiently distribute notification tasks across multiple workers.

graph TD
    NS[Notification Service] -->|Receive Event| MB[Message Broker]
    NS -->|Generate Notification| TE[Template Engine]
    MB -->|Distribute Events| Worker[Notification Worker]
    Worker --> TE
    TE -->|Create Message| SM[Send Message]
    SM -->|Email/SMS| Channels[Notification Channels]

Algorithms/Data Structures:

Implements event-driven architecture using message queues to trigger notifications based on transaction events.
Utilizes template engines for generating notification content dynamically, optimizing the process of message creation.

Trade offs/Tech choices

1. Choice of Microservices Architecture

Pros:
Scalability: Allows each component of the system to scale independently based on demand.
Maintainability: Changes in one service do not require redeploying the entire application, reducing downtime and risk.
Cons:
Complexity: More complex to set up and manage compared to a monolithic architecture. Requires sophisticated service discovery and management strategies.
Data consistency: Ensuring data consistency across services can be challenging and often requires implementing additional patterns such as Saga.

2. Use of Relational Database

Pros:
ACID Transactions: Ensures that all transaction data is consistently stored, which is critical for financial applications.
Well-understood Technology: Relational databases are a mature technology with extensive support and community knowledge.
Cons:
Scalability Limitations: Horizontal scaling is more complex and typically less efficient than with NoSQL databases.

3. Employment of a Distributed Message Queue (e.g., Kafka)

Pros:
Decoupling: Components such as the Transaction Service and Notification Service are decoupled, enhancing fault tolerance and scalability.
Replayability: Enables replaying of messages for scenarios like system failures or for new service integration.
Cons:
Operational Overhead: Requires managing another complex component within the infrastructure.

4. Event-Driven Architecture for Notifications

Pros:
Responsiveness: Allows the system to react in real-time to transaction events, sending notifications immediately.
Scalability: Easily scales to handle large volumes of notifications by adding more subscribers.
Cons:
Event Consistency: Requires careful handling to ensure that all events are processed in order and none are missed or duplicated.

Technology Stack Suggestion

Frontend: React or Angular for dynamic web interfaces.
Backend: Node.js or Java/Spring Boot for creating robust microservices.
Database: PostgreSQL for robust relational data management.
Message Queue: Apache Kafka for handling distributed messaging.
Load Balancer: Nginx or HAProxy to distribute incoming traffic.
Monitoring and Logging: Prometheus and ELK Stack for monitoring services and logging system activities.

Future Improvements

To enhance the Scheduled Digital Transaction System further, consider the following upgrades:

Implementing Machine Learning: Use machine learning algorithms to detect fraudulent transactions and improve security measures dynamically based on transaction patterns.
Blockchain for Transaction Integrity: Explore using blockchain technology to ensure the integrity and non-repudiation of transactions, especially in a system involving financial records.
Service Mesh Implementation: Integrate a service mesh like Istio to manage service-to-service communications, providing better load balancing, service discovery, and secure service communication.
Advanced Analytics Capabilities: Develop advanced analytics for transaction data to provide insights into user behavior, predict trends, and enable data-driven decision-making.

Solutions

nectar4678 - 04/07/2024

0