API Reference / serving /

server.py

Model Serving Server - FastAPI REST API for serving trained models.

Provides HTTP endpoints for loading models, making predictions, and managing the model serving lifecycle.

Usage: # Serve a single model from tuiml.serving import serve serve("model.pkl", port=8000)

# Or create a custom app from tuiml.serving import ModelServer server = ModelServer() server.load_model("my_model", "model.pkl") app = server.create_app() # Run with uvicorn: uvicorn app:app

Classes

ModelServer

class serving.server.ModelServer

REST API server for serving trained TuiML models.

Wraps FastAPI to provide endpoints for model management and prediction serving.

Loading a model deserialises a pickle, which executes whatever the file says. The endpoints are therefore authenticated by default and POST /models -- the one that takes a path from the caller -- is disabled unless models_dir bounds where it may read from.

Constructor 
__init__(
    self,
    max_models: int = 10,
    title: str = 'TuiML Model Server',
    version: str = '1.0.0',
    auth_token: Union[str, bool] = True,
    models_dir: Optional[Union[str, Path]] = None,
    allow_origins: Optional[List[str]] = None,
)

Parameters

max_models
int = 10
Maximum number of models to keep in memory.
title
str = "TuiML Model Server"
API title for documentation.
version
str = "1.0.0"
API version.
auth_token
str or bool = True
Bearer token required on every endpoint except / and /health. True generates one; a string uses it as given; False disables authentication entirely, which is only appropriate behind a proxy that provides its own.
models_dir
str or Path
Directory that POST /models may load from. Paths are resolved and must stay inside it. When unset that endpoint is refused, because without a bound it will unpickle any file the caller names.
allow_origins
list of str
Browser origins permitted to call the API. Empty by default: a model server is normally reached by a script, not a web page, and a wildcard here lets any site the operator visits drive their local server.

Attributes

auth_token
str or None
The token clients must present, or None when authentication is off.
python 
>>> from tuiml.serving import ModelServer
>>> server = ModelServer()
>>> server.load_model("clf", "classifier.pkl")
>>> app = server.create_app()
>>> # Run with: uvicorn app:app --port 8000
>>> # Then: curl -H "Authorization: Bearer $TOKEN" localhost:8000/models

Methods

load_model (self, model_id: str, path: Union[str, Path], metadata: Optional[Dict]=None) -> Dict[str, Any]

Load a model into the server.

Parameters
model_id
str
Unique identifier for the model.
path
str or Path
Path to the saved model file.
metadata
dict
Additional metadata.
Returns
dict
Model information.
unload_model (self, model_id: str) -> bool

Unload a model from memory.

create_app (self) -> 'FastAPI'

Create the FastAPI application with all endpoints.

Returns
FastAPI
Configured FastAPI application instance.
app (self) -> 'FastAPI'

Get or create the FastAPI application.

Functions

Func

create_app

Line 533
create_app(model_path: Optional[Union[str, Path]]=None, model_id: str='default', max_models: int=10, auth_token: Union[str, bool]=True, models_dir: Optional[Union[str, Path]]=None, allow_origins: Optional[List[str]]=None) -> 'FastAPI'

Create a FastAPI application for model serving.

Parameters

model_path
str or Path
Path to a model to load at startup.
model_id
str = "default"
Identifier for the startup model.
max_models
int = 10
Maximum models to cache.
auth_token
str or bool = True
Bearer token required by the API. True generates one, which is readable afterwards as app.state.auth_token; False disables authentication.
models_dir
str or Path
Directory POST /models may load from. Unset disables that endpoint.
allow_origins
list of str
Browser origins permitted to call the API. None permits none.

Returns

FastAPI
Configured FastAPI application.
python 
>>> from tuiml.serving import create_app
>>> app = create_app("classifier.pkl")
>>> token = app.state.auth_token          # send as: Authorization: Bearer <token>
>>> # Run with: uvicorn module:app --port 8000
Func

serve

Line 594
serve(model_or_path, host: str='127.0.0.1', port: int=8000, model_id: str='default', background: bool=True, reload: bool=False, workers: int=1, auth_token: Union[str, bool]=True, models_dir: Optional[Union[str, Path]]=None, allow_origins: Optional[List[str]]=None)

Serve a trained model via REST API.

Accepts a file path, a fitted Workflow, or a model object and starts a uvicorn server exposing prediction endpoints.

Parameters

model_or_path
str, Workflow, or model object

The model to serve:

  • •str: Path to a saved model file
  • •Workflow: a fitted pipeline from train() or Workflow.fit()
  • •model object, Any object with a predict() method
host
str = "127.0.0.1"
Host to bind the server to.
port
int = 8000
Port to listen on.
model_id
str = "default"
Identifier for the model in the server.
background
bool = True
If True, run the server in a daemon thread and return immediately. If False, block until the server is stopped.
reload
bool = False
Enable auto-reload for development. Only applies when background=False.
workers
int = 1
Number of worker processes. Only applies when background=False.

Returns

dict or None
If background=True, returns a dict with server_id, url, and endpoints. If background=False, blocks and returns None.

Serve the pipeline that train() returned:

python 
>>> import tuiml
>>> model = tuiml.train({
...     "model": {"name": "NaiveBayesClassifier"},
...     "data": {"source": "iris"},
... })                                                # doctest: +SKIP
>>> info = tuiml.serve(model, port=8000)              # doctest: +SKIP
>>> info["url"]                                       # doctest: +SKIP
'http://127.0.0.1:8000'

Serve any fitted model. Every algorithm works, with or without a

pipeline around it (models also carry their own .serve() method):

python 
>>> from tuiml.algorithms.trees import RandomForestClassifier
>>> clf = RandomForestClassifier(n_estimators=100)    # doctest: +SKIP
>>> clf = clf.fit(X_train, y_train)                   # doctest: +SKIP
>>> info = tuiml.serve(clf, port=8001)                # doctest: +SKIP

Serve a fitted pipeline built from objects; the fitted preprocessing

travels with it, so the API accepts raw rows:

python 
>>> from tuiml import Workflow
>>> from tuiml.preprocessing import StandardScaler    # doctest: +SKIP
>>> wf = Workflow([StandardScaler(), RandomForestClassifier()])
...                                                   # doctest: +SKIP
>>> wf = wf.fit(X_train, y_train)                     # doctest: +SKIP
>>> info = tuiml.serve(wf, port=8002)                 # doctest: +SKIP

Serve a model saved earlier with model.save(path):

python 
>>> info = tuiml.serve("model.pkl", port=8003)        # doctest: +SKIP

Then manage the running servers:

bash 
$ tuiml.server_status()                             # doctest: +SKIP
$ tuiml.stop_server("127.0.0.1:8001")               # doctest: +SKIP
$ tuiml.stop_server()                               # doctest: +SKIP
Func

stop_server

Line 846
stop_server(server_id: Optional[str]=None) -> None

Stop running model server(s).

Parameters

server_id
str
The server ID ("host:port") to stop. If None, stops all running servers.

Stop a specific server:

python 
>>> import tuiml
>>> tuiml.stop_server("127.0.0.1:9999")

Stop all servers:

bash 
$ tuiml.stop_server()
Func

server_status

Line 877
server_status() -> List[Dict]

Get status of running model servers.

Returns

list of dict
List of server info dicts, each containing server_id, host, port, model_id, and url.
python 
>>> import tuiml
>>> tuiml.server_status()
[{'server_id': '127.0.0.1:9999', 'url': 'http://127.0.0.1:9999', ...}]