API Reference / algorithms / bayesian / estimators /

discrete.py

Discrete probability estimator.

Classes

DiscreteEstimator

class algorithms.bayesian.estimators.discrete.DiscreteEstimator(ProbabilityEstimator)

Discrete probability estimator for categorical data.

Estimates a discrete probability distribution (PMF) by counting the frequency of occurrences. Supports additive (Laplace) smoothing to handle new or rare symbols.
Constructor 
__init__(
    self,
    num_symbols: int,
    laplace: bool = True,
)

Overview

The estimator works as follows:

  1. Maintain a count array of size num_symbols
  2. On each call to add_value, increment the count for the
given symbol index by the specified weight
  1. On each call to get_probability, return the (optionally
smoothed) relative frequency of the queried symbol

Theory

Without smoothing, the probability of symbol k is the maximum likelihood estimate:

\hat{P}(k) = \frac{n_k}{N}

With Laplace smoothing, the estimate becomes:

\hat{P}(k) = \frac{n_k + 1}{N + K}

where n_k is the count for symbol k, N is the total count, and K is the number of possible symbols.

Parameters

num_symbols
int
The number of possible symbols or discrete states in the distribution.
laplace
bool = True
Whether to use Laplace smoothing (add-one smoothing). If True, probabilities are computed as (count + 1) / (total + num_symbols).

Attributes

counts
np.ndarray of shape (num_symbols,)
The raw counts (frequency) for each symbol index.
total_count
float
The aggregate count (sum of weights) of all samples added.

Notes

Complexity:

  • add_value: O(1) per observation
  • get_probability: O(1) per query
When to use DiscreteEstimator:
  • Features are categorical or have been discretised into bins
  • Estimating conditional probability tables in Bayesian networks
  • A simple, fast frequency-based estimator is sufficient

References

Cestnik1990
Cestnik, B. (1990). Estimating Probabilities: A Crucial Task in Machine Learning. Proceedings of the 9th European Conference on Artificial Intelligence, pp. 147-149.

See Also

NormalEstimator

Parametric Gaussian estimator for continuous data.
KernelEstimator

Non-parametric kernel density estimator.

Frequency-based probability estimation with Laplace smoothing:

python 
>>> from tuiml.algorithms.bayesian.estimators import DiscreteEstimator
>>>
>>> # ProbabilityEstimator for 3 possible symbols
>>> est = DiscreteEstimator(num_symbols=3, laplace=True)
>>> est.add_value(0)
>>> est.add_value(0)
>>> est.add_value(1)
>>>
>>> # Query probabilities (smoothed)
>>> est.get_probability(0)  # doctest: +SKIP
0.5

Methods

add_value (self, value: float, weight: float=1.0) -> None

Add a new value observation to the distribution.

Parameters
value
float
The index of the symbol to add (must be between 0 and num_symbols - 1). Non-integer values will be truncated.
weight
float = 1.0
The weight or frequency count of the observation.
get_probability (self, value: float) -> float

Estimate the probability of a specific symbol indicator.

Parameters
value
float
The index of the symbol to query.
Returns
prob
float
The estimated probability mass of the symbol. Returns 0.0 if the index is out of bounds.
get_count (self, value: int) -> float

Get the raw count for a symbol.