API Reference / algorithms / bayesian /

categorical_nb.py

Categorical Naive Bayes classifier implementation.

Classes

CategoricalNBClassifier

class algorithms.bayesian.categorical_nb.CategoricalNBClassifier(Classifier)

Categorical Naive Bayes classifier for discrete / nominal features.

Suitable for data whose features are categorical, each feature takes one of a finite set of integer-coded values. A separate categorical distribution is estimated per feature and class, making it the natural Naive Bayes variant for nominal data (unlike the Gaussian variant, which assumes continuous features).
Constructor 
__init__(
    self,
    alpha: float = 1.0,
    min_categories: Any = None,
)

Overview

The algorithm classifies samples through the following steps:

  1. For each class, estimate the prior P(c) from label frequencies.
  2. For each (class, feature) pair, build a category probability table
by counting how often each category value occurs, with additive (Laplace) smoothing.
  1. At prediction time, sum the log probabilities of the observed
category in every feature plus the log prior.
  1. Return the class with the highest posterior.

Theory

Assuming conditional independence between features, the posterior for class c given a sample \mathbf{x} = (x_1, \ldots, x_m) is:

P(c \mid \mathbf{x}) \propto P(c) \prod_{j=1}^{m} P(x_j \mid c)

With Laplace smoothing (parameter \alpha), each categorical likelihood is estimated as:

\hat{P}(x_j = v \mid c) = \frac{N_{cjv} + \alpha}{N_c + \alpha \, K_j}

where N_{cjv} is the count of value v for feature j in class c, N_c is the number of samples in class c, and K_j is the number of categories of feature j.

Parameters

alpha
float = 1.0
Additive (Laplace/Lidstone) smoothing parameter. 0 disables smoothing.
min_categories
int, list of int or None = None
Minimum number of categories per feature. When None, the number of categories of feature :math:`j` is inferred as max(X[:, j]) + 1.

Attributes

classes_
np.ndarray of shape (n_classes,)
Unique class labels encountered during fit.
class_log_prior_
np.ndarray of shape (n_classes,)
Smoothed log prior probability of each class.
category_log_prob_
list of np.ndarray
Per-feature arrays of shape (n_classes, n_categories_j) holding log P(x_j = v | c).
n_categories_
np.ndarray of shape (n_features,)
Number of categories assumed for each feature.

Notes

Complexity:

  • Training: O(n \cdot m) where n = samples, m = features
  • Prediction: O(n \cdot m \cdot c) where c = classes
When to use CategoricalNBClassifier:
  • Features are nominal / categorical (integer-coded)
  • Continuous features have been discretised into bins
  • A fast, interpretable probabilistic baseline is desired
Category values must be non-negative integers 0 .. K_j - 1. Values outside the range seen during fitting fall back to the smoothed zero-count probability.

References

Manning2008
Manning, C.D., Raghavan, P. and Schutze, H. (2008). Introduction to Information Retrieval. Cambridge University Press, Chapter 13.

See Also

NaiveBayesClassifier

Gaussian/KDE Naive Bayes for continuous features.
NaiveBayesMultinomialClassifier

Multinomial Naive Bayes for count features.

Classification with integer-coded categorical features:

python 
>>> from tuiml.algorithms.bayesian import CategoricalNBClassifier
>>> import numpy as np
>>>
>>> # Two categorical features (values 0..2 and 0..1)
>>> X = np.array([[0, 1], [1, 0], [2, 1], [1, 1], [0, 0]])
>>> y = np.array([0, 1, 1, 1, 0])
>>>
>>> clf = CategoricalNBClassifier(alpha=1.0)
>>> clf.fit(X, y)
CategoricalNBClassifier(alpha=1.0, classes=[0, 1])
>>> clf.predict([[0, 1]])
array([0])

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return classifier capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray) -> 'CategoricalNBClassifier'

Fit the Categorical Naive Bayes classifier.

Parameters
X
array-like of shape (n_samples, n_features)
Integer-coded categorical features (non-negative).
y
array-like of shape (n_samples,)
Target class labels.
Returns
self
CategoricalNBClassifier
The fitted estimator.
predict (self, X: np.ndarray) -> np.ndarray

Predict class labels for samples.

Parameters
X
array-like of shape (n_samples, n_features)
Integer-coded categorical features.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted class labels.
predict_proba (self, X: np.ndarray) -> np.ndarray

Predict class probabilities for samples.

Parameters
X
array-like of shape (n_samples, n_features)
Integer-coded categorical features.
Returns
proba
np.ndarray of shape (n_samples, n_classes)
Normalized class probabilities; columns follow classes_.
__repr__ (self) -> str

String representation.