Practice Now
machine learning
scikit learn
python
data modeling
feature engineering

Fit model to all variables in Python Scikit Learn

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Machine LearningPython

Introduction

Fitting a model to all variables in scikit-learn means using every column in your feature matrix X as input to the model. The basic pattern is model.fit(X, y) where X contains all feature columns and y is the target. scikit-learn expects X as a 2D array (samples x features) and y as a 1D array. The key consideration is whether all variables should be included — sometimes removing irrelevant or correlated features improves performance.

Basic Model Fitting

python
1from sklearn.linear_model import LinearRegression
2from sklearn.datasets import load_diabetes
3import pandas as pd
4
5# Load dataset
6data = load_diabetes()
7X = data.data  # All 10 features
8y = data.target
9
10# Fit model to ALL variables
11model = LinearRegression()
12model.fit(X, y)
13
14print(f"Features used: {data.feature_names}")
15print(f"Coefficients: {model.coef_}")
16print(f"R-squared: {model.score(X, y):.4f}")

model.fit(X, y) trains the model using all columns in X. model.score(X, y) returns the R-squared value on the training data.

Using a Pandas DataFrame

python
1import pandas as pd
2from sklearn.ensemble import RandomForestClassifier
3from sklearn.model_selection import train_test_split
4
5# Create DataFrame
6df = pd.DataFrame({
7    'age': [25, 30, 35, 40, 45, 50, 55, 60],
8    'income': [30000, 45000, 50000, 60000, 70000, 80000, 90000, 100000],
9    'education_years': [12, 14, 16, 16, 18, 18, 20, 20],
10    'purchased': [0, 0, 0, 1, 1, 1, 1, 1]
11})
12
13# Separate features and target
14X = df.drop('purchased', axis=1)  # All columns except target
15y = df['purchased']
16
17# Split and fit
18X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
19
20model = RandomForestClassifier(random_state=42)
21model.fit(X_train, y_train)
22
23print(f"Training accuracy: {model.score(X_train, y_train):.4f}")
24print(f"Test accuracy: {model.score(X_test, y_test):.4f}")

df.drop('target_column', axis=1) gives you all columns except the target — the simplest way to use all variables as features.

Complete Pipeline with Preprocessing

python
1from sklearn.pipeline import Pipeline
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LogisticRegression
4from sklearn.model_selection import cross_val_score
5from sklearn.datasets import load_iris
6
7X, y = load_iris(return_X_y=True)
8
9# Pipeline scales all features, then fits the model
10pipeline = Pipeline([
11    ('scaler', StandardScaler()),
12    ('model', LogisticRegression(max_iter=200))
13])
14
15# Cross-validated evaluation using all features
16scores = cross_val_score(pipeline, X, y, cv=5, scoring='accuracy')
17print(f"CV Accuracy: {scores.mean():.4f} (+/- {scores.std():.4f})")
18
19# Fit on full data
20pipeline.fit(X, y)
21predictions = pipeline.predict(X)

A Pipeline chains preprocessing and modeling. StandardScaler normalizes all features to zero mean and unit variance, which is important for distance-based models and regularized linear models.

Handling Mixed Data Types

python
1from sklearn.compose import ColumnTransformer
2from sklearn.preprocessing import StandardScaler, OneHotEncoder
3from sklearn.linear_model import LogisticRegression
4from sklearn.pipeline import Pipeline
5import pandas as pd
6
7df = pd.DataFrame({
8    'age': [25, 30, 35, 40],
9    'income': [30000, 45000, 50000, 60000],
10    'city': ['NYC', 'LA', 'NYC', 'Chicago'],
11    'purchased': [0, 0, 1, 1]
12})
13
14X = df.drop('purchased', axis=1)
15y = df['purchased']
16
17# Transform numeric and categorical columns differently
18preprocessor = ColumnTransformer([
19    ('num', StandardScaler(), ['age', 'income']),
20    ('cat', OneHotEncoder(drop='first'), ['city']),
21])
22
23pipeline = Pipeline([
24    ('preprocessor', preprocessor),
25    ('model', LogisticRegression())
26])
27
28pipeline.fit(X, y)
29print(f"Score: {pipeline.score(X, y):.4f}")

ColumnTransformer applies different transformations to different columns. Numeric columns get scaled; categorical columns get one-hot encoded. This lets you fit the model to all variables regardless of type.

Feature Importance After Fitting

python
1from sklearn.ensemble import RandomForestRegressor
2from sklearn.datasets import load_diabetes
3import numpy as np
4
5X, y = load_diabetes(return_X_y=True)
6feature_names = load_diabetes().feature_names
7
8model = RandomForestRegressor(n_estimators=100, random_state=42)
9model.fit(X, y)
10
11# Feature importances
12importances = model.feature_importances_
13sorted_idx = np.argsort(importances)[::-1]
14
15for i in sorted_idx:
16    print(f"{feature_names[i]}: {importances[i]:.4f}")

After fitting to all variables, check feature importances to identify which variables actually contribute. Features with near-zero importance can be removed for a simpler model.

Automatic Feature Selection

python
1from sklearn.feature_selection import SelectKBest, f_classif
2from sklearn.datasets import load_iris
3from sklearn.linear_model import LogisticRegression
4from sklearn.pipeline import Pipeline
5
6X, y = load_iris(return_X_y=True)
7
8# Select the k best features automatically
9pipeline = Pipeline([
10    ('select', SelectKBest(f_classif, k=2)),  # Keep top 2 features
11    ('model', LogisticRegression(max_iter=200))
12])
13
14pipeline.fit(X, y)
15print(f"Score with 2 features: {pipeline.score(X, y):.4f}")
16
17# See which features were selected
18selector = pipeline.named_steps['select']
19selected = selector.get_support()
20print(f"Selected features: {np.array(load_iris().feature_names)[selected]}")

SelectKBest automatically selects the most informative features. This fits the model to a subset of all variables, often improving generalization.

Common Pitfalls

  • Including the target variable in X: If the target column is accidentally left in X, the model achieves perfect accuracy on training data but fails completely on new data. Always verify X.columns or X.shape after separation.
  • Not encoding categorical variables: scikit-learn models require numeric input. Passing string columns directly raises ValueError. Use OneHotEncoder, LabelEncoder, or pandas get_dummies() to convert categorical columns before fitting.
  • Fitting without train/test split: model.fit(X, y) followed by model.score(X, y) evaluates on training data, giving an overly optimistic score. Always split data or use cross_val_score for honest evaluation.
  • Not scaling features for sensitive models: Linear regression, logistic regression, SVM, and k-NN are sensitive to feature scales. An income column (thousands) dominates an age column (tens). Use StandardScaler or MinMaxScaler in a Pipeline.
  • Fitting to too many variables relative to samples: With more features than samples, models overfit. If you have 50 features and 100 samples, use regularization (Ridge, Lasso) or feature selection (SelectKBest, PCA) to reduce dimensionality.

Summary

  • Use X = df.drop('target', axis=1) to select all features from a DataFrame
  • model.fit(X, y) trains the model using all columns in X
  • Use Pipeline with StandardScaler to preprocess and fit in one step
  • Use ColumnTransformer to handle mixed numeric and categorical columns
  • Check feature_importances_ after fitting to identify which variables matter
  • Always evaluate with train/test split or cross-validation, not on training data
Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Codemia
© 2026 Codemia
Resources
Legal
Social
All Rights Reserved.