Design Logging Infrastructure for real-time analytics
Last updated: September 24, 2025
Quick Overview
Design a high-throughput logging system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.
Supabase
System Design
Software Engineer
Supabase
September 24, 2025Software Engineer
System Design Round
System Design
Medium
8
8
413 solved
Design a high-throughput logging system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.
ML system design at Supabase goes beyond model selection. This System Design Round question evaluates your ability to design end-to-end ML pipelines, from data collection to model serving, while considering production constraints like latency and reliability.
What the Interviewer Expects
- Define clear ML objectives with appropriate loss functions and metrics
- Design a comprehensive feature engineering pipeline
- Discuss model selection with trade-offs (complexity vs interpretability vs latency)
- Plan online and offline evaluation strategies including A/B testing
- Address serving infrastructure: batch vs real-time, latency requirements
- Consider data quality, labeling strategy, and feedback loops
Key Topics to Cover
Feature engineering and feature stores
Training pipeline and infrastructure
Feedback loops and model retraining
Monitoring and model degradation detection
Model serving and latency optimization
A/B testing and experimentation
How to Approach This
- Start by clarifying functional and non-functional requirements with the interviewer.
- Estimate the scale: QPS, storage, bandwidth. This drives your design decisions.
- Draw a high-level architecture first, then deep dive into 1-2 critical components.
- Discuss trade-offs explicitly (e.g., consistency vs availability, SQL vs NoSQL).
- Address failure scenarios, monitoring, and how the system handles 10x traffic spikes.
Possible Follow-up Questions
- How would you run A/B tests on different model versions?
- How would you ensure fairness and reduce bias in the model?
- How would you handle the cold start problem?
Practice a Similar Problem on Codemia
Solve a related problem with our interactive workspace, get AI feedback, and view detailed solutions.
Solve on CodemiaSample Answer
Requirements
- Functional Requirements:
- Ingest logs from various sources (applications, databases, etc.) in real-time.
- Support querying and analyzing logs for insights and metrics.
- Provide a dashbo...
Capacity Estimation
- Assume each log entry averages 1 KB in size.
- Targeting 10 million log entries per day, translating to approximately 115.74 logs per second.
- This equates to approximately 10 GB of log data daily,...
Submit Your Answer
Markdown supported