Introduction
Fitting a model to all variables in scikit-learn means using every column in your feature matrix X as input to the model. The basic pattern is model.fit(X, y) where X contains all feature columns and y is the target. scikit-learn expects X as a 2D array (samples x features) and y as a 1D array. The key consideration is whether all variables should be included — sometimes removing irrelevant or correlated features improves performance.
Basic Model Fitting
1from sklearn.linear_model import LinearRegression
2from sklearn.datasets import load_diabetes
3import pandas as pd
4
5# Load dataset
6data = load_diabetes()
7X = data.data # All 10 features
8y = data.target
9
10# Fit model to ALL variables
11model = LinearRegression()
12model.fit(X, y)
13
14print(f"Features used: {data.feature_names}")
15print(f"Coefficients: {model.coef_}")
16print(f"R-squared: {model.score(X, y):.4f}")
model.fit(X, y) trains the model using all columns in X. model.score(X, y) returns the R-squared value on the training data.
Using a Pandas DataFrame
1import pandas as pd
2from sklearn.ensemble import RandomForestClassifier
3from sklearn.model_selection import train_test_split
4
5# Create DataFrame
6df = pd.DataFrame({
7 'age': [25, 30, 35, 40, 45, 50, 55, 60],
8 'income': [30000, 45000, 50000, 60000, 70000, 80000, 90000, 100000],
9 'education_years': [12, 14, 16, 16, 18, 18, 20, 20],
10 'purchased': [0, 0, 0, 1, 1, 1, 1, 1]
11})
12
13# Separate features and target
14X = df.drop('purchased', axis=1) # All columns except target
15y = df['purchased']
16
17# Split and fit
18X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
19
20model = RandomForestClassifier(random_state=42)
21model.fit(X_train, y_train)
22
23print(f"Training accuracy: {model.score(X_train, y_train):.4f}")
24print(f"Test accuracy: {model.score(X_test, y_test):.4f}")
df.drop('target_column', axis=1) gives you all columns except the target — the simplest way to use all variables as features.
Complete Pipeline with Preprocessing
1from sklearn.pipeline import Pipeline
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LogisticRegression
4from sklearn.model_selection import cross_val_score
5from sklearn.datasets import load_iris
6
7X, y = load_iris(return_X_y=True)
8
9# Pipeline scales all features, then fits the model
10pipeline = Pipeline([
11 ('scaler', StandardScaler()),
12 ('model', LogisticRegression(max_iter=200))
13])
14
15# Cross-validated evaluation using all features
16scores = cross_val_score(pipeline, X, y, cv=5, scoring='accuracy')
17print(f"CV Accuracy: {scores.mean():.4f} (+/- {scores.std():.4f})")
18
19# Fit on full data
20pipeline.fit(X, y)
21predictions = pipeline.predict(X)
A Pipeline chains preprocessing and modeling. StandardScaler normalizes all features to zero mean and unit variance, which is important for distance-based models and regularized linear models.
Handling Mixed Data Types
1from sklearn.compose import ColumnTransformer
2from sklearn.preprocessing import StandardScaler, OneHotEncoder
3from sklearn.linear_model import LogisticRegression
4from sklearn.pipeline import Pipeline
5import pandas as pd
6
7df = pd.DataFrame({
8 'age': [25, 30, 35, 40],
9 'income': [30000, 45000, 50000, 60000],
10 'city': ['NYC', 'LA', 'NYC', 'Chicago'],
11 'purchased': [0, 0, 1, 1]
12})
13
14X = df.drop('purchased', axis=1)
15y = df['purchased']
16
17# Transform numeric and categorical columns differently
18preprocessor = ColumnTransformer([
19 ('num', StandardScaler(), ['age', 'income']),
20 ('cat', OneHotEncoder(drop='first'), ['city']),
21])
22
23pipeline = Pipeline([
24 ('preprocessor', preprocessor),
25 ('model', LogisticRegression())
26])
27
28pipeline.fit(X, y)
29print(f"Score: {pipeline.score(X, y):.4f}")
ColumnTransformer applies different transformations to different columns. Numeric columns get scaled; categorical columns get one-hot encoded. This lets you fit the model to all variables regardless of type.
Feature Importance After Fitting
1from sklearn.ensemble import RandomForestRegressor
2from sklearn.datasets import load_diabetes
3import numpy as np
4
5X, y = load_diabetes(return_X_y=True)
6feature_names = load_diabetes().feature_names
7
8model = RandomForestRegressor(n_estimators=100, random_state=42)
9model.fit(X, y)
10
11# Feature importances
12importances = model.feature_importances_
13sorted_idx = np.argsort(importances)[::-1]
14
15for i in sorted_idx:
16 print(f"{feature_names[i]}: {importances[i]:.4f}")
After fitting to all variables, check feature importances to identify which variables actually contribute. Features with near-zero importance can be removed for a simpler model.
Automatic Feature Selection
1from sklearn.feature_selection import SelectKBest, f_classif
2from sklearn.datasets import load_iris
3from sklearn.linear_model import LogisticRegression
4from sklearn.pipeline import Pipeline
5
6X, y = load_iris(return_X_y=True)
7
8# Select the k best features automatically
9pipeline = Pipeline([
10 ('select', SelectKBest(f_classif, k=2)), # Keep top 2 features
11 ('model', LogisticRegression(max_iter=200))
12])
13
14pipeline.fit(X, y)
15print(f"Score with 2 features: {pipeline.score(X, y):.4f}")
16
17# See which features were selected
18selector = pipeline.named_steps['select']
19selected = selector.get_support()
20print(f"Selected features: {np.array(load_iris().feature_names)[selected]}")
SelectKBest automatically selects the most informative features. This fits the model to a subset of all variables, often improving generalization.
Common Pitfalls
Including the target variable in X: If the target column is accidentally left in X, the model achieves perfect accuracy on training data but fails completely on new data. Always verify X.columns or X.shape after separation.
Not encoding categorical variables: scikit-learn models require numeric input. Passing string columns directly raises ValueError. Use OneHotEncoder, LabelEncoder, or pandas get_dummies() to convert categorical columns before fitting.
Fitting without train/test split: model.fit(X, y) followed by model.score(X, y) evaluates on training data, giving an overly optimistic score. Always split data or use cross_val_score for honest evaluation.
Not scaling features for sensitive models: Linear regression, logistic regression, SVM, and k-NN are sensitive to feature scales. An income column (thousands) dominates an age column (tens). Use StandardScaler or MinMaxScaler in a Pipeline.
Fitting to too many variables relative to samples: With more features than samples, models overfit. If you have 50 features and 100 samples, use regularization (Ridge, Lasso) or feature selection (SelectKBest, PCA) to reduce dimensionality.
Summary
Use X = df.drop('target', axis=1) to select all features from a DataFrame
model.fit(X, y) trains the model using all columns in X
Use Pipeline with StandardScaler to preprocess and fit in one step
Use ColumnTransformer to handle mixed numeric and categorical columns
Check feature_importances_ after fitting to identify which variables matter
Always evaluate with train/test split or cross-validation, not on training data
