Introduction

Scikit-learn is an essential machine learning library in Python, offering simple and efficient tools for data analysis and modeling. One of the popular models it provides is the Naive Bayes classifier, which is particularly suitable for high-dimensional data and text classification problems. Among the available Naive Bayes models, MultinomialNB is specifically designed for classification with discrete features such as word counts.

Naive Bayes Classification

Naive Bayes classifiers are a family of probabilistic classifiers based on Bayes' Theorem, with the "naive" assumption that features are conditionally independent given the class label. Despite this often unrealistic assumption, Naive Bayes classifiers have proven effective in various applications, particularly in text classification.

Bayes' Theorem

Bayes' Theorem provides a way to update our probability estimates for a hypothesis as more evidence is gathered. The theorem is expressed as:

$$P(C|X) = \frac{P(X|C) \cdot P(C)}{P(X)}$$

Where: • $P(C|X)$ is the posterior probability of class $C$ given feature vector $X$. • $P(X|C)$ is the likelihood of feature vector $X$ given class $C$. • $P(C)$ is the prior probability of class $C$. • $P(X)$ is the evidence probability of feature vector $X$.

Multinomial Naive Bayes

MultinomialNB is a Naive Bayes variant well-suited for discrete data like term frequencies in documents. It calculates the probability of a document being in a class using the frequency of each term.

Formula for Prediction

For a set of classes $C_1, C_2, ..., C_k$:
Given a document represented by feature counts $x = (x_1, x_2, ..., x_n)$, MultinomialNB computes:

$$P(C\_i|x) = \frac{P(C\_i) \prod\_{j=1}^{n} P(x\_j|C\_i)^{x\_j}}{P(x)}$$

Here, $P(x_j|C_i)$ is the probability of feature $x_j$ given class $C_i$, determined by the frequency of $x_j$ in documents of class $C_i$.

Smoothing in MultinomialNB

MultinomialNB employs a smoothing parameter $\alpha$ (also known as Laplace smoothing) to handle the zero probability problem:

$$P(x\_j|C\_i) = \frac{N\_{ij} + \alpha}{N\_i + \alpha n}$$

Where: • $N_{ij}$ is the count of feature $j$ in class $i$. • $N_i$ is the total count of all features in class $i$. • $n$ is the total number of features. • $\alpha$ is the smoothing parameter (default is 1).

Example Usage

Below is a simple example of using MultinomialNB for text classification with scikit-learn:

• Preparation: Text data is converted into feature vectors using CountVectorizer. • Training: The MultinomialNB() model is trained with the vectorized text data. • Prediction: The model predicts target classes for the test set. • Evaluation: The model's accuracy and performance are evaluated using metrics like accuracy score and classification report.