Build a event-driven Ride Matching Pipeline

Question

Design a event-driven ride matching system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **User Matching**: Match riders with drivers based on location, preferences, and ride type.
2. **Real-time Updates**: Provide real-time updates to users regarding ride status (e.g., driver arrival time).
3. **Notification System**: Notify users of ride matches and updates via push notifications.
4. **Feedback Loop**: Allow users to provide ratings and feedback to improve the matching algorithm.

### Non-functional Requirements:
1. **Scalability**: Handle millions of concurrent requests with low latency.
2. **Availability**: Achieve 99.99% uptime to ensure users can always find rides.
3. **Consistency**: Maintain data consistency during ride matching to prevent double bookings.
4. **Latency**: Ensure that ride matches are provided within 2 seconds of request.

Assuming Snapchat's user base is around 500 million users with an average of 10% using the ride-matching feature:

1. **Daily Active Users**: 50 million users.
2. **Requests Per User**: Estimating 2 requests per user per day leads to:
   - Total Requests = 50 million users * 2 requests = 100 million requests per day.
3. **Peak Load**: Assuming peak usage occurs during rush hours, we can estimate a peak of 20% of daily requests within 1 hour:
   - Peak Requests = (100 million / 24 hours) * 1 hour * 20% = ~833,333 requests per minute.
4. **Throughput Requirement**: The system should ideally handle around 14,000 requests per second at peak, factoring in some buffer for growth.

### Component Diagram:

1. **Event Bus**: Use Apache Kafka for real-time event streaming to handle user requests and updates.
2. **Microservices**: 
   - **Matching Service**: Responsible for matching drivers and riders using ML algorithms.
   - **Notification Service**: Sends push notifications to users upon ride match or update.
3. **Feature Store**: Use Feast to store features for real-time inference and model training.
4. **Database**: Use PostgreSQL for relational data storage (user profiles, ride history) and Redis for caching frequent queries.
5. **Model Serving**: Use TensorFlow Serving to deploy ML models for real-time predictions.
6. **Monitoring**: Implement Prometheus and Grafana for monitoring system health and performance.

### Schema Design:

1. **Users Table**: Stores user profiles with fields:
   - `user_id`
   - `location`
   - `preferences`
   - `rating`

2. **Rides Table**: Records all ride requests:
   - `ride_id`
   - `user_id`
   - `driver_id`
   - `pickup_location`
   - `dropoff_location`
   - `status` (pending, completed, cancelled)

3. **Drivers Table**: Contains driver profiles:
   - `driver_id`
   - `current_location`
   - `vehicle_type`
   - `availability_status`

4. **Feedback Table**: Records user feedback for rides:
   - `ride_id`
   - `user_id`
   - `rating`
   - `comments`

### Access Patterns:
- Frequent reads on Users and Drivers tables for matching. Writes to Rides and Feedback tables upon user actions.

### Architectural Decisions:
1. **Consistency vs Availability**: Opted for eventual consistency in user feedback to ensure system availability during high load. This means slight delays in updating user ratings but improved system responsiveness.
2. **Cold Start Problem**: Implemented a hybrid model that combines historical data and collaborative filtering to suggest rides to new users with no prior data.
3. **Model Freshness**: Set up a continuous training pipeline, but with periodic batch updates every hour to balance between freshness and computational costs.
4. **Exploration vs Exploitation**: Utilized a multi-armed bandit approach to dynamically adjust ride matches based on real-time user feedback and behavior, ensuring effective exploration of different match strategies.
5. **Monitoring and Debugging**: Established a robust monitoring framework that tracks key performance metrics and user feedback to identify model drift and maintain model performance over time.

Build a event-driven Ride Matching Pipeline

Snapchat

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Submit Your Answer

Snapchat Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search