Introduction

If an NLTK NaiveBayesClassifier predicts only Positive, the model is usually reflecting bias in data or features rather than failing randomly. Common causes include label imbalance, inconsistent preprocessing, and feature sparsity. The fix is systematic diagnosis of data distribution and feature pipeline, not immediate model replacement.

Why Single-Class Collapse Happens

Naive Bayes combines class priors with token likelihoods. If priors are skewed and feature evidence is weak, predictions collapse to one class.

Typical triggers:

• many more positive than negative samples,
• train and test text processed differently,
• feature dictionaries mostly empty,
• tiny dataset with noisy vocabulary.

Understanding this helps you debug quickly.

Baseline Pipeline

Use a minimal reproducible setup first.

1import random
2from nltk.classify import NaiveBayesClassifier
3
4
5def features(text: str):
6    toks = text.lower().split()
7    return {f"w={t}": True for t in toks}
8
9samples = [
10    ("great movie", "Positive"),
11    ("excellent soundtrack", "Positive"),
12    ("bad acting", "Negative"),
13    ("boring plot", "Negative"),
14]
15
16random.shuffle(samples)
17feature_set = [(features(t), y) for t, y in samples]
18train = feature_set[:3]
19test = feature_set[3:]
20
21clf = NaiveBayesClassifier.train(train)
22print(clf.classify(test[0][0]))

If this tiny example behaves correctly, issue is likely in your production preprocessing or split logic.

Check Label Distribution First

Count labels before any training.

1from collections import Counter
2
3labels = [y for _, y in samples]
4print(Counter(labels))

If one class dominates, classifier may always output that class. Rebalance data or collect more minority examples.

Keep Preprocessing Identical

Train and test text must pass the exact same preprocessing function. Any mismatch can destroy useful token overlap.

Bad pattern example:

• training data lowercased and cleaned,
• test data raw with punctuation and casing differences.

This makes test features sparse and pushes prediction toward majority prior.

Use Stratified Splits

Random split on small or imbalanced data is unstable. Use stratification to keep class proportions consistent.

1from sklearn.model_selection import train_test_split
2
3texts = [t for t, _ in samples]
4y = [label for _, label in samples]
5
6x_train, x_test, y_train, y_test = train_test_split(
7    texts, y, test_size=0.3, random_state=42, stratify=y
8)

Then compute NLTK features from both subsets with identical function.

Inspect Informative Features

NLTK can show which tokens drive classification.

clf.show_most_informative_features(15)

If top features are irrelevant tokens, you likely need better tokenization, normalization, or stopword policy.

Evaluate Beyond Accuracy

Accuracy can look high even with total minority-class failure. Use confusion matrix and per-class precision and recall.

1from nltk.metrics import ConfusionMatrix
2
3pred = [clf.classify(features(t)) for t in x_test]
4cm = ConfusionMatrix(y_test, pred)
5print(cm)

If one class has zero recall, you have class-collapse even if global accuracy appears acceptable.

Improve Feature Quality

Possible upgrades:

• add bigram or negation-aware tokens,
• filter extremely rare terms,
• preserve sentiment-bearing punctuation patterns where useful,
• normalize elongated tokens and repeated symbols consistently.

Feature quality often matters more than switching to a different classifier immediately.

Practical Debug Checklist

1. Verify label counts and stratified split.
2. Confirm train and test use identical preprocessing.
3. Inspect sample feature dictionaries for emptiness.
4. Review informative features.
5. Measure confusion matrix and class-wise recall.

Following this order usually isolates root cause quickly.

Common Pitfalls

• Training on imbalanced data without class-aware validation.
• Different tokenization rules between training and inference code.
• Over-cleaning text and removing sentiment signals.
• Trusting accuracy alone in imbalanced datasets.
• Changing model family before fixing data and feature issues.

Summary

• Constant Positive output is usually a data and feature pipeline issue.
• Start by checking class balance and preprocessing consistency.
• Use stratified splits and class-wise metrics, not only accuracy.
• Inspect informative features to validate signal quality.
• Fix data and feature design first, then revisit model choice if needed.