Design a real-time Image Processing System

Question

Design a real-time image processing system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Image Upload**: Users can upload images for processing.
2. **Real-time Processing**: The system must process images in real-time and return results within a few seconds.
3. **Image Manipulation**: Support for operations such as resizing, cropping, filtering, and format conversion.
4. **Batch Processing**: Allow users to submit multiple images for processing in one request.
5. **API Endpoint**: RESTful API to handle requests with rate limiting.

### Non-functional Requirements
1. **Scalability**: The system should handle millions of requests per day without degradation.
2. **Availability**: Aim for 99.99% uptime to ensure users can access image processing at all times.
3. **Latency**: Maximum response time of 2 seconds for image processing requests.
4. **Consistency**: Strong consistency for processed images but eventual consistency for metadata updates.
5. **Monitoring and Alerting**: Implement monitoring for system health and performance, with alerts for failures.

Assume the system needs to handle 10 million requests per day:
- **Requests per second (RPS)**: 10 million requests / 86400 seconds = ~115.74 RPS.
- **Concurrent Users**: If we assume peak usage leads to 20% of requests happening at once, that would be 0.2 * 10 million / 86400 = ~24 concurrent requests.
- **Image Size**: Assume an average image size of 500 KB, leading to a total daily upload of 10 million * 500 KB = ~4.77 TB.
- **Processing Latency**: Target processing time is 2 seconds, thus needing to design for handling bursts of concurrent processing.

### Component Diagram
1. **API Gateway**: NGINX or AWS API Gateway to route requests and handle rate limiting.
2. **Load Balancer**: Distribute incoming requests across multiple processing nodes to ensure no single node is overwhelmed.
3. **Image Processing Service**: Microservices architecture using Docker containers orchestrated by Kubernetes. Each service will handle specific operations (resize, filter, etc.).
4. **Message Queue**: RabbitMQ or Kafka for handling batch processing asynchronously.
5. **Storage**: Amazon S3 for storing images, with a NoSQL database (DynamoDB) for metadata.
6. **Monitoring**: Prometheus and Grafana for performance metrics and alerting.

### Schema Design
- **Image Metadata Table (DynamoDB)**
  - **image_id (PK)**: Unique identifier for each image.
  - **user_id**: Identifier for the user who uploaded the image.
  - **upload_time**: Timestamp for when the image was uploaded.
  - **status**: Current processing status (e.g., 'pending', 'processing', 'completed').
  - **processed_url**: URL for the processed image in S3.

### Access Patterns
- Primarily query by image_id for individual requests and user_id for batch processing.

1. **Consistency vs. Availability**: Opting for eventual consistency for metadata (image status) allows faster processing and availability, which is crucial for a real-time system.
2. **Synchronous vs. Asynchronous**: Using message queues enables handling spikes in requests without overwhelming the processing services, but introduces latency in processing results.
3. **Complexity vs. Performance**: A microservices architecture increases complexity but allows for better scaling and resource utilization compared to a monolithic design, especially under high load.
4. **Caching Strategies**: Implementing caching for frequently accessed images or results (using Redis) can improve performance but requires careful management of cache invalidation.

Design a real-time Image Processing System

Notion

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Notion Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search