System requirements

Functional:

User Authentication and Registration

User Profile and Preferences

Spam Detection

Sending and receiving emails

Tagging groups and contacts

Non-Functional:

Scalability

Availability

Security

Capacity estimation

Estimate the scale of the system you are going to design...

API design

1. User Management

Sign Up API

• Endpoint: /api/v1/users/signup
• Method: POST
• Request Body:
• { "username": "string", "email": "string", "password": "string"}
• Response:
• { "userId": "string", "message": "User registered successfully"}

• Endpoint: /api/v1/users/login
• Method: POST
• Request Body:
• { "email": "string", "password": "string"}
• Response:
• { "token": "jwt-token-string", "expiresIn": "integer"}

Update User Details API

• Endpoint: /api/v1/users/{userId}
• Method: PUT
• Headers: Authorization: Bearer <token>
• Request Body (Example for updating profile):
• { "username": "newUsername", "email": "[email protected]", "preferences": { "theme": "dark", "notifications": true }}
• Response:
• { "message": "User details updated successfully"}

Send Email API

• Endpoint: /api/v1/emails/send
• Method: POST
• Headers: Authorization: Bearer <token>
• Request Body:
• { "to": ["[email protected]", "[email protected]"], "cc": ["[email protected]"], "bcc": ["[email protected]"], "subject": "Subject of the email", "body": "Email body content", "attachments": ["base64_encoded_file_data"]}
• Response:
• { "messageId": "uniqueMessageId", "status": "Email sent successfully"

 Email Searching

Search Emails API

• Endpoint: /api/v1/emails/search
• Method: GET
• Headers: Authorization: Bearer <token>
• Query Parameters:
  • query: String (e.g., subject:important)
  • page: Integer (e.g., 1)
  • size: Integer (e.g., 10)
• Response:
• { "emails": [ { "messageId": "string", "from": "[email protected]", "subject": "Subject", "summary": "Preview of the email body", "receivedAt": "timestamp" } ], "total": "integer", "page": "integer", "size": "integer"}

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

1. Gateway
2. Auth Service
3. Profile service

Explanation of the Diagram

• User Access: Handles authentication and 2FA. The gateway routes requests and connects to the global cache to validate tokens.
• Profile Management: Manages user profiles with secure image uploading and storage. Images are virus scanned and stored in Amazon S3, with URLs saved in the MySQL database.
• Email Processing: Manages the flow of emails using queues for asynchronous processing. Integrates spam checking using the spam detection service informed by user preferences.
• Security & Preferences: Augments spam detection with user preferences and adapts based on user feedback, continuously refining how emails are assessed.

This architecture reflects a comprehensive email system encompassing scalability, security, enhanced search capabilities, and user preference adaptability. Each component is designed to ensure efficient processing, secure storage, and a reliable user experience.

Request flows

User Authentication

 Token-Based Authentication (e.g., OAuth 2.0, JWT)

• Description: Instead of credentials being passed with each request, a token is issued.
• Pros: Tokens make stateless server-side authentication possible.
• Cons: Requires secure token storage and management of token expiry and revocation.

Flow:

1. User logs in with credentials.
2. Server verifies credentials and issues a JSON Web Token (JWT) or similar token.
3. Client stores the token locally (e.g., in local storage, cookies).
4. With each request, the client sends the token in HTTP headers (e.g., Authorization: Bearer ).
5. Server validates the token’s signature and expiry before processing the request.
6. Tokens may have expiration, encouraging refresh for prolonged sessions.

Two-Factor Authentication (2FA)

• Description: Adds an additional security layer requiring a second confirmation (e.g., SMS code).
• Pros: Enhanced security through an additional verification step.
• Cons: Can be inconvenient without proper user guidance or fail-safes.

Flow:

1. User logs in with username and password.
2. Upon successful login, the server sends a one-time code via SMS/email.
3. User enters this code into the client interface.
4. Server verifies the code, granting access if correct.

SENDING/RECEIVING EMAILS

1. User Submission: User creates and sends an email through the user client.
2. Gateway Handling: The request is routed through a gateway, which forwards it to the email service.
3. Queue Placement: Email is placed in an outgoing message queue for asynchronous processing, allowing the system to handle large volumes efficiently.
4. Spam Check: The email is checked by the notification spam service for spam characteristics.
5. User Preferences: The preference service applies user tagging and filters defined preferences, ensuring user-defined rules are observed.
6. Delivery: Upon validation, the email is sent via the SMTP server to the recipient's mail server.

1. Email Reception: The email is first received by the recipient's mail server and is routed to your SMTP server.
2. Queue Placement: The incoming email is placed in a message queue, allowing it to be processed asynchronously.
3. Spam Check: The notification spam service assesses the email for any spam indicators.
4. User Preferences: User tagging and filter preferences are applied. Tags or folders may be adjusted according to rules set by the user.
5. Indexing: The email content is indexed by Elasticsearch, making it searchable.
6. Metadata Update: Email metadata like tags, read/unread status, and user-filtered preferences are updated in the MySQL database.

Detailed component design

1. Gateway
   • Acts as the entry point for clients, handling all incoming requests.
   • Routes requests to the appropriate services based on URL patterns and methods.
2. Global Cache
   • Stores token and service registry information for quick access to service endpoints and token validation.
   • Reduces load on the authentication service by caching commonly accessed data.
3. Auth Service
   • Manages user authentication, token issuance, and Two-Factor Authentication (2FA) processes.
   • Connects to the notification service to deliver 2FA codes via email or SMS.
4. Profile Service
   • Manages user profiles, handles updates and retrieval, and manages image uploads.
   • Interacts with a file service that stores images securely, incorporating virus scanning to ensure safe uploads.
5. File Service
   • Stores user profile images in Amazon S3 and scans files for viruses.
   • Provides URLs that are stored in the relational database associated with the user's profile.
6. Email Service (SMTP/IMAP)
   • Handles sending (SMTP) and receiving (IMAP) emails.
   • Emails are added to a message queue for further processing.
7. Message Queue
   • Ensures asynchronous processing of emails, allowing the system to handle high volume efficiently.
   • Feeds other services like spam detection, preferences, and search service.
8. Spam Detection Service
   • Analyzes emails for spam characteristics, leveraging machine learning models and user preferences.
   • Uses data from the preference service where users have tagged specific emails as spam.
9. Preference Service
   • Manages user-defined email tags and preferences, influencing how emails are organized and processed.
   • Interfaces with the email service and tag emails as per user settings.
10. Search Service
    • Utilizes Elasticsearch to index and store emails for quick and complex search queries.
    • Provides search functionality across user email data.
11. Notification Service
    • Manages sending notifications such as 2FA codes, utilizing templated messages for consistency.
12. MySQL Database
    • Stores structured data such as user profiles, image URLs, and email metadata.
    • Maintains relationships such as user-to-email associations and user preferences.
13. Elasticsearch
    • Indexes email content for efficient, scalable search capabilities.
    • Allows users to perform complex queries across their email dataset.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?