API Reference / algorithms / bayesian /

naive_bayes.py

Naive Bayes classifier implementation.

Classes

NaiveBayesClassifier

class algorithms.bayesian.naive_bayes.NaiveBayesClassifier(Classifier)

Naive Bayes classifier using pluggable probability estimators.

Naive Bayes classifiers are a family of probabilistic classifiers based on applying Bayes' theorem with strong independence assumptions between the features. This implementation supports both Gaussian and kernel density estimation for modeling numeric feature distributions.
Constructor 
__init__(
    self,
    use_kernel_estimator: bool = False,
    use_laplace: bool = True,
    var_smoothing: float = 1e-09,
)

Overview

The algorithm classifies instances through the following steps:

  1. Compute the prior probability of each class from training labels
  2. For each feature, fit a probability estimator (Gaussian or KDE) per class
  3. At prediction time, compute the posterior for each class as the
product of the prior and the per-feature likelihoods
  1. Return the class with the highest posterior probability

Theory

Classification is based on Bayes' theorem. The posterior probability of class c given a feature vector \mathbf{x} is:

P(c \mid \mathbf{x}) = \frac{P(c) \, P(\mathbf{x} \mid c)}{P(\mathbf{x})}

Under the naive independence assumption, the likelihood factorises:

P(\mathbf{x} \mid c) = \prod_{i=1}^{m} P(x_i \mid c)

so the decision rule becomes:

\hat{c} = \arg\max_{c} \; P(c) \prod_{i=1}^{m} P(x_i \mid c)

Each factor P(x_i \mid c) is estimated by a pluggable density estimator -- either a Gaussian (normal) distribution or a kernel density estimator (KDE).

Parameters

use_kernel_estimator
bool = False
If True, use kernel density estimation for numeric attributes instead of assuming a Gaussian distribution. Kernel estimation is more flexible but computationally more expensive.
use_laplace
bool = True
If True, apply Laplace (add-one) smoothing to class priors. This prevents zero probabilities for classes with few samples.

Attributes

classes_
np.ndarray of shape (n_classes,)
Unique class labels discovered during fit.
class_prior_
np.ndarray of shape (n_classes,)
Prior probability of each class, i.e., P(class). Computed from training data, optionally with Laplace smoothing.
estimators_
list of list
2D list of probability estimators indexed as estimators_[class_idx][feature_idx]. Each estimator models the conditional distribution P(feature|class).

Notes

Complexity:

  • Training: O(n \cdot m \cdot c) where n = samples, m = features, c = classes
  • Prediction: O(m \cdot c) per sample (Gaussian); O(n \cdot m \cdot c) per sample (KDE)
When to use NaiveBayesClassifier:
  • When you need a fast, simple baseline classifier
  • Text classification and spam filtering
  • When features are approximately independent given the class
  • Datasets with missing values (NaN values are handled gracefully)
  • When interpretable class probabilities are desired
The classifier uses log-probabilities internally for numerical stability. For numeric features, the default is to use a Gaussian distribution. Set use_kernel_estimator=True for non-Gaussian or multimodal data.

References

John1995
John, G.H. and Langley, P. (1995). Estimating Continuous Distributions in Bayesian Classifiers. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, pp. 338-345.
Domingos1997
Domingos, P. and Pazzani, M. (1997). On the Optimality of the Simple Bayesian Classifier under Zero-One Loss. Machine Learning, 29(2-3), 103-130. DOI: 10.1023/A:1007413511361

See Also

NaiveBayesMultinomialClassifier

Multinomial variant for discrete/count data.
NormalEstimator

Gaussian probability estimator used by default.
KernelEstimator

Kernel density estimator for flexible distributions.

Basic classification with default settings:

python 
>>> from tuiml.algorithms.bayesian import NaiveBayesClassifier
>>> import numpy as np
>>>
>>> # Create sample training data
>>> X_train = np.array([[1, 2], [2, 3], [3, 4], [6, 7], [7, 8], [8, 9]])
>>> y_train = np.array([0, 0, 0, 1, 1, 1])
>>>
>>> # Fit and predict with Gaussian estimators
>>> clf = NaiveBayesClassifier()
>>> clf.fit(X_train, y_train)
NaiveBayesClassifier(classes=[0, 1], estimator=Normal)
>>> clf.predict([[2, 3]])
array([0])
>>> clf.predict_proba([[2, 3]])  # doctest: +SKIP
array([[0.99, 0.01]])

Using kernel density estimation for non-Gaussian data:

python 
>>> clf_kde = NaiveBayesClassifier(use_kernel_estimator=True)
>>> clf_kde.fit(X_train, y_train)
NaiveBayesClassifier(classes=[0, 1], estimator=Kernel)

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return the JSON schema for classifier parameters.

Returns
dict of str to dict
A dictionary mapping parameter names to their schema definitions. Each schema includes type, default, and description.
get_capabilities (cls) -> List[str]

Return the data capabilities supported by this classifier.

Returns
list of str

List of capability identifiers:

  • •"numeric": Handles numeric/continuous features
  • •"missing_values": Handles missing values (NaN)
  • •"binary_class": Supports binary classification
  • •"multiclass": Supports multi-class classification
get_complexity (cls) -> str

Return the computational complexity of the algorithm.

Returns
str

A string describing time complexity for training and prediction, where:

  • •n = number of training samples
  • •m = number of features
  • •c = number of classes
get_references (cls) -> List[str]

Return academic references for the algorithm.

Returns
list of str
List of citation strings in standard academic format.
fit (self, X: np.ndarray, y: np.ndarray) -> 'NaiveBayesClassifier'

Fit the Naive Bayes classifier to the training data.

Parameters
X
array-like of shape (n_samples, n_features)
Training feature matrix. Can be a NumPy array or any array-like that can be converted to a NumPy array. Missing values (NaN) are allowed and handled gracefully.
y
array-like of shape (n_samples,)
Target class labels for the training samples.
Returns
self
NaiveBayesClassifier
Returns the fitted classifier instance for method chaining.
partial_fit (self, X: np.ndarray, y: np.ndarray, classes: Optional[np.ndarray]=None) -> 'NaiveBayesClassifier'

Incrementally fit the Naive Bayes classifier on a batch of samples.

Parameters
X
array-like of shape (n_samples, n_features)
Incremental training features.
y
array-like of shape (n_samples,)
Incremental target labels.
classes
array-like of shape (n_classes,) = None
List of all classes expected. Must be provided at the first call, can be omitted afterwards.
Returns
self
NaiveBayesClassifier
Returns the instance itself.
predict (self, X: np.ndarray) -> np.ndarray

Predict class labels for input samples.

Parameters
X
array-like of shape (n_samples, n_features)
Feature matrix of samples to classify.
Returns
numpy.ndarray of shape (n_samples,)
Predicted class label for each sample.
Raises
RuntimeError
If the classifier has not been fitted yet.
predict_proba (self, X: np.ndarray) -> np.ndarray

Predict class membership probabilities for input samples.

Parameters
X
array-like of shape (n_samples, n_features)
Feature matrix of samples to classify.
Returns
numpy.ndarray of shape (n_samples, n_classes)
Probability of each class for each sample. Columns correspond to classes in self.classes_. Each row sums to 1.0.
Raises
RuntimeError
If the classifier has not been fitted yet.
predict_log_proba (self, X: np.ndarray) -> np.ndarray

Predict log-probabilities of class membership for input samples.

Parameters
X
array-like of shape (n_samples, n_features)
Feature matrix of samples to classify.
Returns
numpy.ndarray of shape (n_samples, n_classes)
Log-probability of each class for each sample. Columns correspond to classes in self.classes_.
Raises
RuntimeError
If the classifier has not been fitted yet.
__repr__ (self) -> str

Return a string representation of the classifier.

Returns
str
A concise string showing the classifier state. If fitted, shows the discovered classes and estimator type. If not fitted, shows the configuration parameters.