API Reference / serving /

model_manager.py

Model Manager - In-memory model loading and caching for serving.

Handles loading trained models into memory, caching for performance, and provides a unified interface for predictions.

Classes

ModelManager

class serving.model_manager.ModelManager

Manage loaded models for serving predictions.

Provides thread-safe model loading, caching, and prediction interface with automatic LRU eviction when cache is full.
Constructor 
__init__(
    self,
    max_models: int = 10,
)

Parameters

max_models
int = 10
Maximum number of models to keep in memory. Older models are evicted when limit is reached.
python 
>>> from tuiml.serving import ModelManager
>>> manager = ModelManager(max_models=5)
>>> manager.load("classifier", "model.pkl")
>>> predictions = manager.predict("classifier", X)
>>> manager.list_models()
['classifier']

Methods

load (self, model_id: str, path: Union[str, Path], metadata: Optional[Dict]=None) -> Dict[str, Any]

Load a model into memory.

Parameters
model_id
str
Unique identifier for the model.
path
str or Path
Path to the saved model file.
metadata
dict
Additional metadata to associate with the model.
Returns
dict
Model info including class name, parameters, and load time.
Raises
FileNotFoundError
If the model file does not exist.
unload (self, model_id: str) -> bool

Unload a model from memory.

Parameters
model_id
str
Identifier of the model to unload.
Returns
bool
True if model was unloaded, False if not found.
get_model (self, model_id: str) -> Any

Get a loaded model instance.

Parameters
model_id
str
Identifier of the model.
Returns
object
The loaded model instance.
Raises
KeyError
If model is not loaded.
get_model_info (self, model_id: str) -> Dict[str, Any]

Get metadata for a loaded model.

Parameters
model_id
str
Identifier of the model.
Returns
dict
Model information including class, parameters, and stats.
list_models (self) -> List[str]

List all loaded model IDs.

Returns
list of str
List of model identifiers.
is_loaded (self, model_id: str) -> bool

Check if a model is loaded.

predict (self, model_id: str, X: Union[np.ndarray, List]) -> np.ndarray

Make predictions using a loaded model.

Parameters
model_id
str
Identifier of the model to use.
X
array-like
Input features for prediction.
Returns
ndarray
Model predictions.
Raises
KeyError
If model is not loaded.
ValueError
If input format is invalid.
predict_proba (self, model_id: str, X: Union[np.ndarray, List]) -> np.ndarray

Get prediction probabilities using a loaded model.

Parameters
model_id
str
Identifier of the model to use.
X
array-like
Input features for prediction.
Returns
ndarray
Class probabilities with shape (n_samples, n_classes).
Raises
KeyError
If model is not loaded.
NotImplementedError
If model doesn't support probability predictions.
get_stats (self) -> Dict[str, Any]

Get statistics about loaded models.

Returns
dict
Statistics including model count and total predictions.
clear (self)

Unload all models from memory.