Introduction

SMOTE is designed for feature vectors, not for raw image pixels as a first-choice image balancing method. If you apply it blindly to flattened images, the synthetic samples often land between images in a way that is mathematically valid but visually meaningless, which is why image augmentation or embedding-space oversampling is usually the better approach.

Why Raw-Pixel SMOTE Is Usually a Bad Fit

SMOTE creates new minority samples by interpolating between nearby minority points in feature space. That works reasonably well for many tabular datasets because the features already describe the examples in a meaningful numeric space.

With images, however, the raw pixel grid is a very high-dimensional representation. Two nearby flattened images do not necessarily correspond to two semantically similar visual objects. Interpolating them can produce blurry, unnatural hybrids.

That means this kind of workflow is usually weak:

• flatten image to one huge vector
• run SMOTE on those vectors
• train on synthetic pixel blends

It may improve class counts, but it often harms image realism.

Better Default: Image Augmentation

For most image tasks, the first balancing tool should be augmentation, not SMOTE.

Typical augmentations include:

• flips
• rotations
• crops
• color jitter
• small translations
• noise or blur when appropriate

In Keras, a simple augmentation pipeline might look like this:

1import tensorflow as tf
2
3augment = tf.keras.Sequential([
4    tf.keras.layers.RandomFlip("horizontal"),
5    tf.keras.layers.RandomRotation(0.1),
6    tf.keras.layers.RandomZoom(0.1),
7])
8
9images = tf.random.uniform((4, 128, 128, 3))
10augmented = augment(images)
11print(augmented.shape)

This keeps the samples in image space and usually produces more realistic minority examples than direct interpolation on flattened pixels.

If You Still Want SMOTE, Use Feature Space

The more defensible way to use SMOTE with image data is:

1. extract image embeddings using a CNN or encoder
2. apply SMOTE to those embeddings
3. train a downstream classifier on the balanced embedding vectors

That way, SMOTE operates in a representation space where distances are more meaningful.

Here is a small example using a pretrained feature extractor and then SMOTE.

1import numpy as np
2from imblearn.over_sampling import SMOTE
3from sklearn.linear_model import LogisticRegression
4from sklearn.model_selection import train_test_split
5from tensorflow.keras.applications import MobileNetV2
6from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
7
8# Example image batch.
9X_images = np.random.rand(100, 96, 96, 3).astype("float32") * 255.0
10y = np.array([0] * 85 + [1] * 15)
11
12feature_model = MobileNetV2(include_top=False, pooling="avg", input_shape=(96, 96, 3))
13X_features = feature_model.predict(preprocess_input(X_images), verbose=0)
14
15smote = SMOTE(random_state=42)
16X_balanced, y_balanced = smote.fit_resample(X_features, y)
17
18X_train, X_test, y_train, y_test = train_test_split(
19    X_balanced, y_balanced, test_size=0.2, random_state=42, stratify=y_balanced
20)
21
22clf = LogisticRegression(max_iter=1000)
23clf.fit(X_train, y_train)
24print(clf.score(X_test, y_test))

This is much more defensible than applying SMOTE directly to raw pixel arrays.

Autoencoders Are Another Option

If you want synthetic minority examples that remain closer to the original image manifold, you can encode images into a latent space with an autoencoder, oversample the latent vectors, and decode them back.

That approach is more complex than standard augmentation, but it respects image structure better than pixel-space interpolation.

The tradeoff is operational cost. You are now training both a representation model and a classifier pipeline.

Class Weighting May Be Enough

Before adding oversampling complexity, try class weighting. Many image classifiers can improve minority recall just by weighting the loss more heavily for rare classes.

class_weight = {0: 1.0, 1: 5.0}

model.fit(train_dataset, epochs=5, class_weight=class_weight)

This is often easier to maintain than generating synthetic images or synthetic embeddings.

A Practical Decision Order

For imbalanced image classification, a sensible order is usually:

1. start with class weighting
2. add image augmentation
3. evaluate threshold tuning and metrics carefully
4. only then consider embedding-space SMOTE or more advanced synthetic methods

That sequence avoids jumping into a technique that is conceptually available but not always the best fit.

Common Pitfalls

A common mistake is flattening images and applying SMOTE directly as if images were tabular rows. The resulting synthetic examples often do not preserve meaningful visual structure.

Another issue is evaluating only accuracy after oversampling. On imbalanced problems, minority recall and precision still matter more.

Developers also sometimes perform oversampling before splitting train and test data, which leaks information and inflates evaluation results.

Finally, do not assume balanced class counts automatically mean better generalization. The quality of synthetic data matters.

Summary

• SMOTE is usually not a good first choice for raw image pixels.
• Prefer class weighting and image augmentation for most image tasks.
• If you use SMOTE, apply it to learned image embeddings rather than flattened pixels.
• Be careful to oversample only in the training workflow.
• Balanced counts are useful only when the synthetic samples remain meaningful.