How do you actually apply a trained model?
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
Applying a trained model means running inference on new data, not retraining the model from scratch. In practice, that only works reliably if you save the model together with the exact preprocessing steps it expects and then feed production data through the same pipeline.
Save the Whole Inference Pipeline
A trained model usually expects features in exactly the same shape and order used during training. If training included imputation, scaling, one-hot encoding, or text vectorization, those steps are part of the model application path.
With scikit-learn, the cleanest pattern is to save a pipeline instead of a bare estimator:
That file can now be used directly for inference.
Run Inference on New Data
Applying the model means loading the artifact and passing new records with the same feature names:
That is the practical meaning of “apply a trained model”: take unseen examples, transform them in the same way as training data, and run the estimator’s prediction method.
Interpret the Output Correctly
The output depends on the task:
- Classification often returns class labels and class probabilities.
- Regression returns a numeric value.
- Ranking or recommendation models may return scores rather than direct labels.
Do not assume the raw output is ready for users. Many systems add business logic after the model runs. For example, a fraud model may output a probability, but the application may only flag cases above a threshold:
That threshold is part of deployment logic, not training logic.
Batch Versus Real-Time Application
There are two common ways to apply a model:
- Batch inference, such as scoring a whole table every night.
- Online inference, such as scoring one request inside an API.
A batch job might look like this:
An online API would wrap the same call in a request handler. The important part is that the inference code stays thin. Load the artifact, validate the input, run prediction, and return structured output.
Production Concerns
Successful inference depends on more than code:
- input schema validation
- model versioning
- logging and monitoring
- drift detection
- fallback behavior when the model or features are unavailable
If training used columns age, income, and plan, then production code should reject payloads that omit one of those fields or change their meaning.
Common Pitfalls
The most common mistake is saving only the estimator and forgetting preprocessing. That leads to feature mismatch bugs when new data reaches production.
Another problem is assuming inference data looks exactly like training data. Real production inputs often contain missing fields, unseen categories, and formatting issues.
Teams also forget to version artifacts. If you overwrite the same model file repeatedly, it becomes hard to explain later predictions or roll back a bad deployment.
Finally, avoid retraining inside the request path. Applying a trained model should be fast and deterministic. Training is a separate workflow.
Summary
- Applying a trained model means running inference on new data.
- Save and load the full preprocessing-plus-model pipeline, not just the estimator.
- Keep production input schema aligned with training features.
- Interpret outputs according to the task and any decision thresholds.
- Separate training, artifact storage, and inference into distinct operational steps.
Related reading
- How do you actually apply a trained model?
- How do you add new categories and training to a pretrained Inception v3 model in TensorFlow?
- How do you alter the size of a Pytorch Dataset?
- How do you compute accuracy in a regression model, after rounding predictions to classes, in keras?
- How do you calculate the average of a set of circular data?
- How do you decode one-hot labels in Tensorflow?
- How do you cleanly list all the containers in a kubernetes pod?
- How do you delete an AWS CloudWatch metric?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.