PerceptronClassifier classifier implementation.
Classes
PerceptronClassifier classifier.
A single-layer neural network that learns a linear decision boundary for classification. Supports binary and multiclass classification using a one-vs-all strategy.
Constructor
__init__( self, learning_rate: float = 1.0, max_iter: int = 1000, tol: float = 0.001, shuffle: bool = True, random_state: Optional[int] = None, early_stopping: bool = True, )
Overview
The Perceptron trains by iterating over samples and adjusting weights when a misclassification occurs:
- Initialize weight vectors and bias terms to zero for each class
- For each training sample, compute scores across all classes
- Predict the class with the highest score
- If the prediction is incorrect, update weights: increase weights for the
- Repeat for multiple epochs until convergence or early stopping
Theory
The Perceptron uses a linear activation with the following update rule. Given input \mathbf{x} with true label y and predicted label \hat{y}:
\mathbf{w}_y \leftarrow \mathbf{w}_y + \eta \, \mathbf{x}
\mathbf{w}_{\hat{y}} \leftarrow \mathbf{w}_{\hat{y}} - \eta \, \mathbf{x}
where \eta is the learning rate. The decision function for class k is:
f_k(\mathbf{x}) = \mathbf{w}_k \cdot \mathbf{x} + b_k
The predicted label is \hat{y} = \arg\max_k f_k(\mathbf{x}).
Parameters
learning_rate
float
= 1.0
Learning rate for weight updates.
max_iter
int
= 1000
Maximum number of passes over the training data (epochs).
tol
float
= 1e-3
Tolerance for stopping criterion based on error rate.
shuffle
bool
= True
Whether to shuffle training data after each epoch.
random_state
int or None
= None
Seed used by the random number generator if
shuffle is True.
early_stopping
bool
= True
Whether to stop training if zero mistakes are made in an epoch.
Attributes
weights_
np.ndarray of shape (n_classes, n_features)
Weight vectors for each class.
bias_
np.ndarray of shape (n_classes,)
Bias terms for each class.
classes_
np.ndarray of shape (n_classes,)
Unique class labels discovered during
fit.
n_iter_
int
Number of iterations run during training.
Notes
Complexity:
- Training: O(n \cdot d \cdot T) where n = number of
- Prediction: O(n \cdot d \cdot K) where K = number of
When to use PerceptronClassifier:
- When the data is linearly separable or nearly so
- When a fast, simple baseline classifier is needed
- When interpretability of the weight vector is important
- As a building block before moving to more complex neural models
References
Rosenblatt1958
Rosenblatt, F. (1958).
The Perceptron: A Probabilistic Model for Information Storage and
Organization in the Brain.
Psychological Review, 65(6), 386-408.
Novikoff1963
Novikoff, A.B. (1963).
On Convergence Proofs for Perceptrons.
Symposium on the Mathematical Theory of Automata, 12, 615-622.
Train a Perceptron on a simple binary classification task:
python
>>> from tuiml.algorithms.neural import PerceptronClassifier
>>> import numpy as np
>>> X = np.array([[1, 2], [2, 3], [4, 5], [5, 6]])
>>> y = np.array([0, 0, 1, 1])
>>> clf = PerceptronClassifier(learning_rate=0.1, max_iter=100)
>>> clf.fit(X, y)
PerceptronClassifier(n_iter=..., n_classes=2)
>>> clf.predict([[3, 4]])
array([0])Methods
get_parameter_schema
(cls) -> Dict[str, Dict[str, Any]]
get_capabilities
(cls) -> List[str]
get_complexity
(cls) -> str
get_references
(cls) -> List[str]
partial_fit
(self, X: np.ndarray, y: np.ndarray, classes: Optional[np.ndarray]=None) -> 'PerceptronClassifier'
partial_fit
(self, X: np.ndarray, y: np.ndarray, classes: Optional[np.ndarray]=None) -> 'PerceptronClassifier'
Incrementally fit the PerceptronClassifier classifier on a batch of samples.
Parameters
X
np.ndarray of shape (n_samples, n_features)
Incremental training features.
y
np.ndarray of shape (n_samples,)
Incremental target labels.
classes
np.ndarray of shape (n_classes,)
= None
List of all classes expected. Must be provided at the first call, can be omitted afterwards.
Returns
self
PerceptronClassifier
Returns the updated instance.
__repr__
(self) -> str