API Reference / features / extraction /

random_projection.py

Random Projection for dimensionality reduction.

This module provides Random Projection, which reduces dimensionality using a random matrix while preserving distances (Johnson-Lindenstrauss lemma).

Classes

RandomProjectionExtractor

class features.extraction.random_projection.RandomProjectionExtractor(FeatureExtractor)

Random Projection for dimensionality reduction.

Reduces the dimensionality of the data by projecting it onto a lower dimensional subspace using a random matrix.
Constructor 
__init__(
    self,
    n_components: Union[int, float, str] = 10,
    distribution: str = 'gaussian',
    random_state: Optional[int] = None,
)

Overview

Random Projection is a computationally efficient way to reduce dimensionality. It is based on the Johnson-Lindenstrauss lemma, which states that if points in a high-dimensional space are projected onto a randomly chosen subspace of suitable dimension, then the distances between the points are approximately preserved.

Theory

For a data matrix X_{n \times p}, the projected matrix is:

X_{new} = X_{n \times p} \cdot R_{p \times k}^T

where R is a random matrix. The minimum dimension k to preserve distances within a factor of 1 \pm \epsilon is:

k \ge \frac{4 \ln(n)}{\epsilon^2 / 2 - \epsilon^3 / 3}

Parameters

n_components
int, float, or "auto" = 10

Target dimensionality:

  • •int >= 1: absolute number of components.
  • •float < 1: percentage of original features.
  • •"auto": Use the Johnson-Lindenstrauss formula to determine :math:`k`.
distribution
{"gaussian", "sparse", "rademacher"} = "gaussian"

Distribution used for the random matrix:

  • •"gaussian": Normal distribution :math:`N(0, 1/k)`.
  • •"sparse": Very-sparse projection (Achlioptas): :math:`\sqrt{3} \times \{-1, 0, 1\}` with probabilities :math:`\{1/6, 2/3, 1/6\}`.
  • •"rademacher": Rademacher :math:`\{-1, 1\}` with probabilities :math:`\{1/2, 1/2\}`.
random_state
int
Random seed for reproducibility.

Attributes

n_components_
int
Actual number of components used for projection.
components_
np.ndarray of shape (n_components, n_features)
The generated random projection matrix.

Notes

Complexity:
  • O(n \cdot p \cdot k) for projection.
  • Much faster than PCA as it doesn't require eigenvalue decomposition.
When to use:
  • Very high-dimensional datasets where PCA is computationally prohibitive.
  • When preserving pairwise distances is more important than maximizing variance.
  • As a fast preprocessing step for distance-based algorithms (k-NN, clustering).
Limitations:
  • Only approximately preserves distances.
  • Projection matrix is not optimized for the specific data (data-independent).

References

Fradkin2003
Fradkin, D. and Madigan, D. (2003). Experiments with random projections for machine learning. KDD '03, pp. 517-522.
JL1984
Johnson, W. B. and Lindenstrauss, J. (1984). Extensions of Lipschitz mappings into a Hilbert space.

See Also

PCAExtractor

Variance-maximizing projection.

Methods

fit (self, X: np.ndarray, y: Optional[np.ndarray]=None) -> 'RandomProjectionExtractor'

Fit the random projection matrix.

Parameters
X
ndarray of shape (n_samples, n_features)
Training data.
y
Ignored
Not used, present for API consistency.
Returns
self
RandomProjectionExtractor
The fitted transformer.
transform (self, X: np.ndarray) -> np.ndarray

Apply random projection to X.

Parameters
X
ndarray of shape (n_samples, n_features)
Data to transform.
Returns
X_new
ndarray of shape (n_samples, n_components)
Projected data.
inverse_transform (self, X: np.ndarray) -> np.ndarray

Approximate inverse transform using pseudo-inverse.

Parameters
X
ndarray of shape (n_samples, n_components)
Projected data.
Returns
X_original
ndarray of shape (n_samples, n_features)
Approximately reconstructed data.
get_feature_names_out (self, input_features: Optional[List[str]]=None) -> np.ndarray

Get output feature names for transformation.

Parameters
input_features
list of str
Ignored, output names are always rp0, rp1, etc.
Returns
feature_names_out
np.ndarray of str
Names of the output features.
get_parameter_schema (cls) -> Dict[str, Any]

Return JSON Schema for constructor parameters.

SparseRandomProjectionExtractor

class features.extraction.random_projection.SparseRandomProjectionExtractor(RandomProjectionExtractor)

Sparse Random Projection for dimensionality reduction.

A convenience class that uses a sparse random matrix by default, which is more computationally efficient than Gaussian projection for high-dimensional data.
Constructor 
__init__(
    self,
    n_components: Union[int, float, str] = 10,
    density: float = ...,
    random_state: Optional[int] = None,
)

Overview

Sparse random projection reduces memory requirements and speeds up the transformation by using a matrix where most entries are zero. It still preserves distances according to the Johnson-Lindenstrauss lemma.

This is equivalent to RandomProjectionExtractor with distribution="sparse".

Parameters

n_components
int, float, or "auto" = 10
Target dimensionality.
density
float = 1/3
Ratio of non-zero elements in the random matrix. The default value 1/3 corresponds to the "sparse" distribution.
random_state
int
Random seed for reproducibility.

See Also

RandomProjectionExtractor

Base random projection class.

Project high-dimensional data using sparse matrix:

python 
>>> from tuiml.features.extraction import SparseRandomProjectionExtractor
>>> import numpy as np
>>> X = np.random.randn(100, 1000)
>>> srp = SparseRandomProjectionExtractor(n_components="auto")
>>> X_projected = srp.fit_transform(X)
>>> print(f"Projected to {srp.n_components_} dimensions")

Methods

get_parameter_schema (cls) -> Dict[str, Any]

Return JSON Schema for constructor parameters.