Normal (Gaussian) probability estimator.
Classes
Gaussian (Normal) distribution probability density estimator.
Estimates a normal distribution for numeric data by maintaining running sums and sums of squares. This allows for incremental updates and efficient calculation of the mean and variance.
Constructor
__init__( self, precision: Optional[float] = None, )
Overview
The estimator works as follows:
- Accumulate running sum, sum of squares, and count as
add_value
-
On each call to
get_probability, recompute the mean and
- Evaluate the Gaussian PDF at the query point
Theory
The probability density at value x is given by the normal PDF:
p(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\!\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)
where the mean \mu and variance \sigma^2 are estimated from the running statistics:
\mu = \frac{\sum w_i x_i}{\sum w_i}, \qquad \sigma^2 = \frac{\sum w_i x_i^2}{\sum w_i} - \mu^2
Parameters
precision
float or None
= None
The precision constraint for the variance. If provided, the variance will be floored at this value to avoid division by zero or negative probabilities. Defaults to
1e-6.
Attributes
sum
float
Sum of all values added to the estimator.
sum_sq
float
Sum of squares of all values added to the estimator.
count
float
Total number (or weight) of samples added.
Notes
Complexity:
-
add_value: O(1) per observation -
get_probability: O(1) per query
- Features are approximately normally distributed
- A fast, lightweight density estimator is needed
- Incremental / online estimation is required
References
John1995
John, G.H. and Langley, P. (1995).
Estimating Continuous Distributions in Bayesian Classifiers.
Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence,
pp. 338-345.
See Also
Incremental density estimation:
python
>>> from tuiml.algorithms.bayesian.estimators import NormalEstimator
>>>
>>> # Build estimator from observations
>>> est = NormalEstimator()
>>> for v in [1.0, 2.0, 3.0, 4.0, 5.0]:
... est.add_value(v)
>>>
>>> # Query density at the mean
>>> est.get_probability(3.0) # doctest: +SKIP
0.3989...