Post
ML System Design Interview Prep Guide
How the machine learning system design interview works, a ten-step answer structure from problem framing to monitoring, the metrics and formulas worth knowing, and practice problems from easy to hard.
By Codemia • November 14, 2026
6 min read
The ML system design interview asks you to design a complete machine learning system, such as a video recommender, a fraud detector or a search ranker, in about 45 minutes. It is the core design round for machine learning engineer, applied scientist and many data science roles at large tech companies.
It looks like a system design interview but is graded differently. Interviewers care less about load balancers and more about whether you framed the problem correctly, chose sensible data and features, picked a model you can justify, and know how you would tell whether it works. This guide covers the structure, the material and the problems to practise.
Jumping straight to the model. Candidates who spend the first ten minutes on transformer architecture and never define what they are predicting, or how success is measured, rarely pass. Framing the problem well is worth more than any model choice.
How the round differs from system design
| System design | ML system design | |
| Starting point | Functional and non-functional requirements | Business objective and how to frame it as an ML task |
| Core of the answer | Services, data stores, scaling | Data, features, model, training, evaluation |
| Deep dives | Bottlenecks, consistency, failure | Label quality, metrics, cold start, feedback loops |
| Numbers that matter | Requests per second, storage | Dataset size, class balance, latency budget per prediction |
A ten-step answer structure
You will not have time to go deep on all ten, but touching each one, and going deep on two or three, is what a strong answer looks like.
- Objective. What business goal does this serve, and what will the model predict? Turn "improve the feed" into "rank candidate posts by predicted probability of meaningful engagement". Decide whether it is classification, regression or ranking.
- Data and labels. Where do training examples come from? Explicit labels (a user reported spam) or implicit ones (a click, a watch past 30 seconds)? How noisy, delayed or biased are they?
- Features. User, item, context and interaction features. How each is computed, and which must be fresh at serving time.
- Model. Start with a simple baseline (logistic regression, gradient boosted trees), then propose something stronger and say why. For recommendation and search, describe the two-stage pattern: candidate generation, then ranking.
- Training. How the dataset is split (by time, to avoid leakage), how class imbalance is handled, how often you retrain.
- Offline evaluation. The metrics you use before shipping, and why they fit the problem.
- Online evaluation. A/B testing, the metric you would ship on, and guardrail metrics.
- Serving. Latency budget, batch versus real-time prediction, feature stores, caching.
- Monitoring. Data drift, prediction drift, label delay, and what triggers retraining.
- Iteration and risks. Feedback loops, fairness, cold start, and what you would build next.
Metrics and formulas worth knowing
Classification. Precision is the share of positive predictions that are right; recall is the share of real positives you catch. For rare events (fraud, harmful content), accuracy is useless: a model that never flags anything is 99.9% accurate. Use precision and recall at a chosen threshold, or the area under the precision-recall curve.
Binary cross-entropy is the standard training loss for click, spam and fraud models:
Ranking. For search and recommendation, the order matters. NDCG rewards putting the most relevant items at the top:
NDCG@K divides DCG@K by the best possible DCG@K, so a perfect ranking scores 1. Mean reciprocal rank and recall@K are also common.
Multi-objective scoring. Feeds and recommenders usually predict several outcomes and combine them, for example . Be ready to explain how the weights are chosen and how they map to the business goal.
Topics interviewers probe
- Label leakage: a feature that would not be available at prediction time, such as using a refund to predict fraud.
- Class imbalance: downsampling negatives, class weights, and recalibrating probabilities afterwards.
- Cold start: new users and new items with no history. Content features, popularity priors and exploration.
- Feedback loops: a recommender trained on its own outputs narrows over time. Exploration and logging policies help.
- Position bias: items shown higher get clicked more regardless of relevance.
- Delayed labels: chargebacks arrive weeks after the transaction, so fraud models train on old data.
- Embeddings and retrieval: two-tower models and approximate nearest neighbour search for candidate generation.
- Training-serving skew: features computed differently offline and online. Feature stores reduce it.
Practice problems
Every problem below links to the Codemia ML system design catalogue, where you can write your design and get it graded against that problem's key points.
Easy: a single model with a clear label
- Sentiment analysis system: text classification and label quality.
- Harmful content detection: rare positives, precision and recall trade-offs.
- Query understanding: intent classification for search.
- Similar listings: embeddings and nearest neighbour retrieval.
Medium: retrieval, imbalance and real time
- Spam detection: adversarial users and fast feedback.
- Visual search: image embeddings and an ANN index.
- Search ranking: learning to rank and NDCG.
- ETA prediction: regression with live features.
- Autocomplete: language models under a tight latency budget.
Hard: multi-stage systems and delayed labels
- Video recommendation: candidate generation plus ranking, multi-objective scoring.
- News feed ranking: engagement prediction and integrity signals.
- Fraud detection: extreme imbalance and delayed labels.
- Ad click prediction: calibration and very sparse features.
- People you may know: graph features and candidate generation.
How to prepare
- Learn the structure above until you can run through all ten steps for any problem in five minutes.
- Do the easy problems first, focusing on framing, labels and metrics rather than the model.
- Move to recommendation and ranking problems, which dominate interviews at large consumer companies. Make sure you can explain two-stage retrieval and ranking clearly.
- Practise the trade-off questions out loud: why this metric, why this model, what happens when the labels are wrong.
- Brush up on the systems side. ML system design rounds still ask about serving, feature stores and data pipelines. System design explained: a beginner's guide covers the building blocks.
Many ML engineers also face a standard system design round and sometimes an object-oriented design round. See system design practice problems, from easy to hard and the object-oriented design interview guide.