Build a multi-tenant Authentication Pipeline

Question

Design a multi-tenant authentication system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **User Registration**: Allow users to create accounts with email and password, or via OAuth (Google, Facebook).
2. **Authentication**: Support login functionality using username/password and token-based authentication (JWT).
3. **Multi-Tenancy**: Handle authentication for different applications (sub-domains) under Twitter/X, isolating user data per tenant.
4. **Password Recovery**: Implement a password reset mechanism via email.
5. **Session Management**: Allow users to view and manage their active sessions.

### Non-Functional Requirements
1. **Scalability**: Handle millions of authentication requests per second with low latency.
2. **Availability**: Ensure 99.99% uptime for the authentication service.
3. **Performance**: Authentication requests should be processed in under 100 ms.
4. **Security**: Implement measures like rate limiting, account lockout after multiple failed attempts, and encryption of sensitive data.

Assuming Twitter/X has approximately 400 million active users, with an estimated 10% logging in daily, we can expect:

- **Daily Login Requests**: 40 million
- **Peak Requests**: Let's assume peak times (e.g., during events) could see a 5x increase, resulting in 200 million requests in a day.
- **Requests per Second (RPS)**: Spread over 24 hours, this results in an average of 2,314 RPS (200 million / 86,400 seconds). To accommodate spikes, we should plan for a capacity of around 5,000 RPS.

Given these numbers, our system should be able to handle significantly higher loads, with an aim for scaling to 10,000 RPS.

### Components
1. **API Gateway**: Handles incoming requests and routes them to appropriate services (e.g., AWS API Gateway).
2. **Authentication Service**: Core logic for handling registrations, logins, and token management (built with Node.js).
3. **User Database**: A distributed NoSQL database (e.g., DynamoDB) for storing user credentials and session tokens.
4. **Cache Layer**: Redis for storing session tokens and temporary user data to speed up authentication checks.
5. **Monitoring and Logging**: Use tools like Prometheus for monitoring and ELK Stack for logging to track performance and error rates.

### Diagram
- API Gateway → Authentication Service → User Database
- Authentication Service ↔ Cache Layer

### User Schema
```sql
CREATE TABLE Users (
    user_id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL,
    email VARCHAR(255) UNIQUE NOT NULL,
    password_hash VARCHAR(255) NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    CONSTRAINT tenant_unique UNIQUE (tenant_id, email)
);
```
### Session Schema
```sql
CREATE TABLE Sessions (
    session_id UUID PRIMARY KEY,
    user_id UUID REFERENCES Users(user_id),
    token VARCHAR(255) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL
);
```
### Access Patterns
- Frequent reads for user authentication and session validation.
- Writes on user registration and password updates.

1. **Consistency vs. Availability**: Using a NoSQL database like DynamoDB provides high availability but may introduce eventual consistency. For critical operations like login, we can implement a strong consistency model at the cost of latency.
2. **Caching vs. Database Load**: Introducing Redis as a cache reduces load on the database but introduces complexity in cache invalidation. We need to balance cache duration to avoid stale data while maintaining performance.
3. **Security vs. Usability**: Implementing strict security measures (e.g., 2FA) may reduce user convenience. We could introduce optional 2FA for sensitive accounts while keeping it simple for regular users.
4. **Horizontal Scaling**: The architecture is designed for horizontal scaling, allowing us to add more instances of the Authentication Service and cache layers as load increases.

Build a multi-tenant Authentication Pipeline

Twitter/X

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Twitter/X Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search