Architect a fault-tolerant Feature Flag Engine

Question

Design a fault-tolerant feature flag system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Feature Flag Management**: Ability to create, update, delete, and retrieve feature flags with associated metadata (e.g., description, targeting rules).
2. **User Targeting**: Support for targeting specific user segments based on attributes (e.g., user ID, location, subscription level).
3. **Rollout Strategies**: Ability to configure gradual rollouts or A/B testing for features.
4. **Real-time Updates**: Flags should be updated in real-time without downtime for users.

### Non-Functional Requirements
1. **Scalability**: Must handle millions of requests per second with low latency.
2. **Fault Tolerance**: System should remain operational despite component failures.
3. **Consistency**: Eventual consistency is acceptable; however, up-to-date flags should be available to clients.
4. **Monitoring and Alerting**: System should have built-in monitoring for performance metrics and alerting for failures.

Assuming Tesla's user base and service requests:
- **User Base**: 10 million active users.
- **Feature Flag Requests**: Each user may check feature flags 10 times per day.
- **Total Requests Per Day**: 10 million users * 10 requests/user = 100 million requests.
- **Peak Requests**: Assuming peak load is 10% of daily requests in one hour: 100 million requests / 24 hours * 10% = ~416,667 requests/second.

To ensure system robustness, we should design for 1 million requests/second to accommodate growth and unforeseen spikes.

### Component Diagram
- **API Gateway**: Use **AWS API Gateway** to handle incoming requests and route them.
- **Feature Flag Service**: A microservice built with **Spring Boot** to manage feature flags, interfacing with the database.
- **Database**: Use **Cassandra** for distributed database storage, ensuring high availability and fault tolerance with eventual consistency.
- **Caching Layer**: Implement **Redis** for caching frequently accessed feature flags to improve read performance.
- **Message Queue**: Use **Kafka** to handle updates to flags and ensure eventual consistency across services.
- **Monitoring**: Utilize **Prometheus** for metrics collection and **Grafana** for visualization.

### Schema Design
- **Feature Flags Table**: 
  - `flag_id` (UUID, Primary Key)
  - `name` (String, Unique)
  - `description` (String)
  - `enabled` (Boolean)
  - `targeting_rules` (JSON)
  - `created_at` (Timestamp)
  - `updated_at` (Timestamp)

- **User Segments Table**: 
  - `segment_id` (UUID, Primary Key)
  - `user_id` (UUID)
  - `flag_id` (UUID, Foreign Key)
  - `status` (Boolean)

### Access Patterns
- Read-heavy pattern for fetching feature flags, hence the use of caching and a NoSQL database like Cassandra for fast lookups.

1. **Consistency vs. Availability**: By choosing Cassandra, we prioritize availability and partition tolerance (AP system) over strict consistency. This is acceptable for feature flags since eventual consistency suffices.
2. **Caching**: Implementing Redis introduces complexity and potential cache staleness, but improves performance significantly. We need a cache invalidation strategy upon flag updates.
3. **Real-time Updates**: Using Kafka allows us to process updates asynchronously, which may lead to slight delays in propagating changes to all users, but ensures that the system remains responsive under load.

Architect a fault-tolerant Feature Flag Engine

Tesla

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Tesla Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search