API Reference / algorithms / bayesian /

naive_bayes_multinomial.py

Multinomial Naive Bayes classifier implementation.

Classes

NaiveBayesMultinomialClassifier

class algorithms.bayesian.naive_bayes_multinomial.NaiveBayesMultinomialClassifier(Classifier)

Multinomial Naive Bayes classifier for text and discrete data.

The multinomial Naive Bayes classifier is suitable for classification with discrete features (e.g., word counts for text classification). It models the probability of a document belonging to a class as the product of the probabilities of each word in the document given that class.
Constructor 
__init__(
    self,
    alpha: float = 1.0,
)

Overview

The algorithm classifies documents through the following steps:

  1. Count the frequency of each feature (word) for every class
  2. Apply additive smoothing to avoid zero probabilities
  3. Compute log class priors and log feature likelihoods
  4. At prediction time, compute the joint log-likelihood and return
the class with the highest score

Theory

The multinomial model assumes features are generated from a multinomial distribution. The posterior for class c given document \mathbf{x} = (x_1, \ldots, x_m) is:

P(c \mid \mathbf{x}) \propto P(c) \prod_{i=1}^{m} P(w_i \mid c)^{x_i}

where x_i is the count of feature i in the document.

With Laplace smoothing (parameter \alpha), the feature likelihood is estimated as:

\hat{P}(w_i \mid c) = \frac{N_{ic} + \alpha}{N_c + \alpha \, m}

where N_{ic} is the total count of feature i in class c, N_c is the total count of all features in class c, and m is the vocabulary size.

Parameters

alpha
float = 1.0
Additive (Laplace/Lidstone) smoothing parameter. Set to 0 for no smoothing. Smoothing prevents zero probabilities for features not seen in the training data.

Attributes

classes_
np.ndarray of shape (n_classes,)
Unique class labels encountered during fit.
class_prior_
np.ndarray of shape (n_classes,)
Log prior probability of each class: log P(class).
feature_log_prob_
np.ndarray of shape (n_classes, n_features)
Log probability of features given class: log P(feature | class).
class_count_
np.ndarray of shape (n_classes,)
Number of samples encountered for each class during fitting.
feature_count_
np.ndarray of shape (n_classes, n_features)
Aggregate feature counts encountered for each class.

Notes

Complexity:

  • Training: O(n \cdot m) where n = samples, m = features
  • Prediction: O(m \cdot c) per sample where c = classes
When to use NaiveBayesMultinomialClassifier:
  • Text classification (e.g., spam filtering, sentiment analysis)
  • Features are word counts, term frequencies, or TF-IDF values
  • High-dimensional sparse data (large vocabularies)
  • Online / incremental learning via partial_fit
While theoretically designed for integer counts, this classifier often works well with fractional counts such as TF-IDF weighted values.

References

McCallum1998
McCallum, A. and Nigam, K. (1998). A Comparison of Event Models for Naive Bayes Text Classification. AAAI-98 Workshop on Learning for Text Categorization, pp. 41-48.
Manning2008
Manning, C.D., Raghavan, P. and Schutze, H. (2008). Introduction to Information Retrieval. Cambridge University Press, Chapter 13.

See Also

NaiveBayesClassifier

Gaussian/KDE-based Naive Bayes for continuous features.

Basic text-like classification with word counts:

python 
>>> from tuiml.algorithms.bayesian import NaiveBayesMultinomialClassifier
>>> import numpy as np
>>>
>>> # Word count features for 4 documents
>>> X = np.array([[2, 1, 0], [1, 2, 0], [0, 1, 2], [0, 2, 1]])
>>> y = np.array([0, 0, 1, 1])
>>>
>>> # Fit with Laplace smoothing
>>> clf = NaiveBayesMultinomialClassifier(alpha=1.0)
>>> clf.fit(X, y)
NaiveBayesMultinomialClassifier(alpha=1.0, classes=[0, 1])
>>> clf.predict([[3, 0, 0]])
array([0])

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return parameter schema.

get_capabilities (cls) -> List[str]

Return classifier capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray) -> 'NaiveBayesMultinomialClassifier'

Fit the Multinomial Naive Bayes classifier to the training data.

Parameters
X
array-like of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features. Features should be non-negative (frequencies, counts, or TF-IDF).
y
array-like of shape (n_samples,)
Target values (class labels).
Returns
self
NaiveBayesMultinomialClassifier
Returns the fitted estimator.
predict (self, X: np.ndarray) -> np.ndarray

Perform classification on an array of test vectors X.

Parameters
X
array-like of shape (n_samples, n_features)
The input samples.
Returns
C
np.ndarray of shape (n_samples,)
Predicted target values for X.
predict_proba (self, X: np.ndarray) -> np.ndarray

Return probability estimates for the test vectors X.

Parameters
X
array-like of shape (n_samples, n_features)
The input samples.
Returns
C
np.ndarray of shape (n_samples, n_classes)
Returns the probability of the samples for each class in the model. The columns correspond to the classes in self.classes_.
predict_log_proba (self, X: np.ndarray) -> np.ndarray

Predict log class probabilities for samples.

Parameters
X
array-like of shape (n_samples, n_features)
Test features.
Returns
log_proba
np.ndarray of shape (n_samples, n_classes)
Log class probabilities. Columns correspond to classes in self.classes_.
partial_fit (self, X: np.ndarray, y: np.ndarray, classes: Optional[np.ndarray]=None) -> 'NaiveBayesMultinomialClassifier'

Incremental fit on a batch of samples.

Parameters
X
array-like of shape (n_samples, n_features)
Training vector, where n_samples is the number of samples and n_features is the number of features.
y
array-like of shape (n_samples,)
Target values (class labels).
classes
np.ndarray or None = None
List of all the classes that can possibly appear in the y vector. Must be provided at the first call to partial_fit, can be omitted in subsequent calls.
Returns
self
NaiveBayesMultinomialClassifier
Returns the fitted estimator.
__repr__ (self) -> str

String representation.