Kernel density estimator.
Classes
Gaussian Kernel Density ProbabilityEstimator (KDE) for non-parametric density estimation.
Provides a non-parametric estimation of the probability density function using a sum of Gaussian kernels centred at each data point. This is advantageous for multimodal or non-normal distributions where a simple parametric model (like Gaussian) would fail.
Constructor
__init__( self, precision: Optional[float] = None, )
Overview
The estimator works as follows:
-
Store every observed value along with its weight via
add_value - Compute the bandwidth (kernel width) using Silverman's rule of thumb
- At query time, sum the Gaussian kernel contributions from all stored
Theory
The kernel density estimate at point x is:
\hat{f}(x) = \frac{1}{n\,h} \sum_{i=1}^{n} w_i \, K\!\left(\frac{x - x_i}{h}\right)
where K is the Gaussian kernel:
K(u) = \frac{1}{\sqrt{2\pi}} \exp\!\left(-\tfrac{1}{2} u^2\right)
The bandwidth h is selected using Silverman's rule of thumb:
h = \hat{\sigma} \cdot n^{-1/5}
where \hat{\sigma} is the sample standard deviation and n is the number of observations.
Parameters
precision
float or None
= None
The precision constraint for bandwidth calculation and rounding. Defaults to
1e-6.
Attributes
values
list of float
The list of values added to the estimator.
weights
list of float
The weights corresponding to each value.
total_weight
float
Sum of all weights added.
standard_deviation
float
The calculated bandwidth (h) for the kernels. A value of
-1.0 indicates it needs to be recalculated.
Notes
Complexity:
-
add_value: O(1) per observation -
get_probability: O(n) per query where n is the number of stored values
- Feature distributions are multimodal or strongly non-Gaussian
- More flexible density estimation is needed at the cost of speed
- The number of training samples per class is relatively small
References
Silverman1986
Silverman, B.W. (1986).
Density Estimation for Statistics and Data Analysis.
Chapman and Hall, London.
John1995
John, G.H. and Langley, P. (1995).
Estimating Continuous Distributions in Bayesian Classifiers.
Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence,
pp. 338-345.
See Also
Non-parametric density estimation:
python
>>> from tuiml.algorithms.bayesian.estimators import KernelEstimator
>>>
>>> # Build estimator from bimodal data
>>> est = KernelEstimator()
>>> for v in [1.0, 1.1, 1.2, 5.0, 5.1, 5.2]:
... est.add_value(v)
>>>
>>> # Query density at each mode
>>> est.get_probability(1.1) # doctest: +SKIP
0.25...