Introduction

Naive Bayes is one of the simplest and strongest baselines for text classification, and n-grams make it more expressive by capturing short word sequences rather than isolated words only. The combination is especially useful for spam filtering, sentiment classification, and intent detection, where phrases such as not good or limited time carry meaning that a unigram model may miss.

Why N-grams Help Naive Bayes

A unigram model treats each token independently. That is often good enough for quick classification, but it loses local context.

For example, the sentence not bad contains the token bad, which by itself may look negative. A bigram feature not bad gives the model a stronger signal that the phrase has a different meaning.

Common n-gram choices are:

• unigrams for broad vocabulary coverage
• bigrams for local phrase context
• trigrams for more specific expressions

The tradeoff is feature growth. As n increases, the vocabulary becomes much larger and the training data must support that extra sparsity.

A Practical Scikit-learn Pipeline

The easiest way to train this model in Python is with CountVectorizer and MultinomialNB.

1from sklearn.feature_extraction.text import CountVectorizer
2from sklearn.naive_bayes import MultinomialNB
3from sklearn.pipeline import Pipeline
4
5train_texts = [
6    "limited time offer claim prize now",
7    "cheap meds available today",
8    "project meeting moved to tomorrow",
9    "please review the design document",
10    "exclusive deal just for you",
11    "team lunch is scheduled at noon",
12]
13
14train_labels = [
15    "spam",
16    "spam",
17    "ham",
18    "ham",
19    "spam",
20    "ham",
21]
22
23model = Pipeline([
24    ("vectorizer", CountVectorizer(ngram_range=(1, 2), lowercase=True)),
25    ("classifier", MultinomialNB(alpha=1.0)),
26])
27
28model.fit(train_texts, train_labels)
29
30tests = [
31    "exclusive offer today",
32    "review the project plan",
33]
34
35print(model.predict(tests))
36print(model.predict_proba(tests))

This pipeline uses both unigrams and bigrams with ngram_range=(1, 2). That is a common default because it adds phrase information without exploding the feature space too aggressively.

What the Model Learns

After vectorization, each document becomes a sparse count vector. Features might include:

• 'offer'
• 'project'
• 'exclusive deal'
• 'review the'

MultinomialNB then estimates how likely each feature is under each class. During prediction, it combines:

• the class prior
• the per-feature likelihoods

Even though the independence assumption is simplistic, the model often works surprisingly well because word-count evidence is strong in many text tasks.

N-grams slightly violate the pure independence story because overlapping phrases share tokens, but in practice that is usually acceptable. Naive Bayes is valued for speed and robustness more than for perfectly realistic probability modeling.

Preprocessing Choices That Matter

The vectorizer configuration affects the model at least as much as the classifier.

Useful knobs include:

• 'ngram_range=(1, 2) or (1, 3)'
• 'min_df to drop very rare features'
• 'stop_words when common words add more noise than value'
• 'binary=True if presence matters more than repeated count'

For example:

1vectorizer = CountVectorizer(
2    ngram_range=(1, 2),
3    min_df=2,
4    lowercase=True,
5    binary=False
6)

For sentiment tasks, removing stop words is not always a good idea because words such as not are highly informative. That is one reason text preprocessing should follow the task, not a generic checklist.

You can also use TfidfVectorizer with Naive Bayes, but classic Multinomial Naive Bayes is most naturally paired with count-like features. If you use TF-IDF, test carefully rather than assuming it will be better.

Evaluating the Classifier

Text classification should be evaluated on held-out data, not only on the training set. A small train-test split example:

1from sklearn.model_selection import train_test_split
2from sklearn.metrics import classification_report
3
4X_train, X_test, y_train, y_test = train_test_split(
5    train_texts,
6    train_labels,
7    test_size=0.33,
8    random_state=42
9)
10
11model.fit(X_train, y_train)
12predictions = model.predict(X_test)
13print(classification_report(y_test, predictions))

On real datasets, compare several settings such as unigram-only versus unigram-plus-bigram. The extra features help only if they add stable signal rather than sparse noise.

Common Pitfalls

The most common mistake is jumping straight to trigrams or larger n-grams on a small dataset. That often creates too many rare features and hurts generalization.

Another issue is forgetting smoothing. In MultinomialNB, the alpha parameter prevents zero probabilities for unseen features. Setting it too low can make the model brittle.

People also leak information by fitting the vectorizer on the full dataset before the train-test split. The vectorizer must be fit only on training data, which is why a pipeline is the safest pattern.

Finally, do not assume n-grams always help. They improve many tasks, but they also increase memory use and can overfit if the corpus is small or inconsistent.

Summary

• Naive Bayes with n-grams is a fast, strong baseline for text classification.
• Unigrams capture vocabulary, while bigrams and trigrams capture local phrase context.
• In scikit-learn, CountVectorizer plus MultinomialNB is the standard setup.
• Use pipelines so vectorization and classification are trained correctly without leakage.
• Start with unigrams plus bigrams and tune feature size before adding more complexity.