machine learning
model deployment
AI applications
predictive modeling
data science

How do you actually apply a trained model?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

Applying a trained model means running inference on new data, not retraining the model from scratch. In practice, that only works reliably if you save the model together with the exact preprocessing steps it expects and then feed production data through the same pipeline.

Save the Whole Inference Pipeline

A trained model usually expects features in exactly the same shape and order used during training. If training included imputation, scaling, one-hot encoding, or text vectorization, those steps are part of the model application path.

With scikit-learn, the cleanest pattern is to save a pipeline instead of a bare estimator:

python
1from sklearn.compose import ColumnTransformer
2from sklearn.impute import SimpleImputer
3from sklearn.linear_model import LogisticRegression
4from sklearn.pipeline import Pipeline
5from sklearn.preprocessing import OneHotEncoder, StandardScaler
6import joblib
7import pandas as pd
8
9train_df = pd.DataFrame(
10    [
11        {"age": 25, "income": 50000, "plan": "basic", "churned": 0},
12        {"age": 42, "income": 91000, "plan": "pro", "churned": 1},
13        {"age": 31, "income": 72000, "plan": "basic", "churned": 0},
14    ]
15)
16
17X = train_df[["age", "income", "plan"]]
18y = train_df["churned"]
19
20preprocess = ColumnTransformer(
21    transformers=[
22        ("num", Pipeline([
23            ("imputer", SimpleImputer(strategy="median")),
24            ("scaler", StandardScaler()),
25        ]), ["age", "income"]),
26        ("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
27    ]
28)
29
30model = Pipeline([
31    ("preprocess", preprocess),
32    ("classifier", LogisticRegression()),
33])
34
35model.fit(X, y)
36joblib.dump(model, "churn_pipeline.joblib")

That file can now be used directly for inference.

Run Inference on New Data

Applying the model means loading the artifact and passing new records with the same feature names:

python
1import joblib
2import pandas as pd
3
4model = joblib.load("churn_pipeline.joblib")
5
6new_customers = pd.DataFrame(
7    [
8        {"age": 37, "income": 84000, "plan": "pro"},
9        {"age": 22, "income": 38000, "plan": "basic"},
10    ]
11)
12
13predicted_class = model.predict(new_customers)
14predicted_prob = model.predict_proba(new_customers)
15
16print(predicted_class)
17print(predicted_prob)

That is the practical meaning of “apply a trained model”: take unseen examples, transform them in the same way as training data, and run the estimator’s prediction method.

Interpret the Output Correctly

The output depends on the task:

  • Classification often returns class labels and class probabilities.
  • Regression returns a numeric value.
  • Ranking or recommendation models may return scores rather than direct labels.

Do not assume the raw output is ready for users. Many systems add business logic after the model runs. For example, a fraud model may output a probability, but the application may only flag cases above a threshold:

python
fraud_score = 0.82
is_flagged = fraud_score >= 0.90

That threshold is part of deployment logic, not training logic.

Batch Versus Real-Time Application

There are two common ways to apply a model:

  1. Batch inference, such as scoring a whole table every night.
  2. Online inference, such as scoring one request inside an API.

A batch job might look like this:

python
scored = new_customers.copy()
scored["churn_prediction"] = model.predict(new_customers)
scored.to_csv("scored_customers.csv", index=False)

An online API would wrap the same call in a request handler. The important part is that the inference code stays thin. Load the artifact, validate the input, run prediction, and return structured output.

Production Concerns

Successful inference depends on more than code:

  • input schema validation
  • model versioning
  • logging and monitoring
  • drift detection
  • fallback behavior when the model or features are unavailable

If training used columns age, income, and plan, then production code should reject payloads that omit one of those fields or change their meaning.

Common Pitfalls

The most common mistake is saving only the estimator and forgetting preprocessing. That leads to feature mismatch bugs when new data reaches production.

Another problem is assuming inference data looks exactly like training data. Real production inputs often contain missing fields, unseen categories, and formatting issues.

Teams also forget to version artifacts. If you overwrite the same model file repeatedly, it becomes hard to explain later predictions or roll back a bad deployment.

Finally, avoid retraining inside the request path. Applying a trained model should be fast and deterministic. Training is a separate workflow.

Summary

  • Applying a trained model means running inference on new data.
  • Save and load the full preprocessing-plus-model pipeline, not just the estimator.
  • Keep production input schema aligned with training features.
  • Interpret outputs according to the task and any decision thresholds.
  • Separate training, artifact storage, and inference into distinct operational steps.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.