9. Specs: ML as Declarative Config

This chapter has no new modelling in it. It is about a format — and about why that format is the hinge the second half of this book turns on.

Everything so far has been Python: import a class, configure it, call fit. That works because you are a programmer sitting at a keyboard. The rest of this book is about a different situation, where the thing deciding what to train is a language model, and it cannot import anything.

You will learn:

• the full grammar of a spec, key by key
• presets, and when to stop using them
• how to move between objects and configuration in both directions
• what reproducibility actually requires
• why a dictionary is the right interface for an agent

Prerequisites: chapters 4 and 7.

import json
import tempfile
from pathlib import Path

import numpy as np
import tuiml

9.1 The grammar

A spec is a dictionary with at most six keys. Two are required.

spec = {
    "model": {"name": "LogisticRegression"},
    "data": "diabetes",
    "evaluation": {"cv": 10},
    "random_seed": 42,
}

model = tuiml.train(spec)
model.metrics_

{'cv_accuracy_score_mean': 0.7707621326042379,
 'cv_accuracy_score_std': 0.06102327014484286,
 'cv_f1_score_mean': 0.6367443231395333,
 'cv_f1_score_std': 0.08613072754441539}

model (required) — what to train, as {"name": ..., "params": {...}}. params may be omitted for defaults.

data (required) — a built-in dataset name, a file path, or in-memory arrays:

"data": "diabetes"                                    # built-in
"data": {"source": "patients.csv", "target": "dx"}    # file
"data": {"X": X_array, "y": y_array}                  # arrays

pipeline (optional) — the ordered preprocessing steps from chapter 4, each in the same {"name", "params"} shape, or a preset name.

evaluation (optional) — {"cv": 10} for k-fold, or {"test_size": 0.2, "stratify": True} for holdout, plus "metrics".

features (optional) — a column subset to keep.

random_seed (optional) — seeds splits, folds, and every component that accepts one.

The uniformity is the point: a model, a scaler, a sampler and a selector are all written {"name": ..., "params": {...}}. One rule to learn, and one rule for a program to generate.

full = {
    "model": {"name": "RandomForestClassifier", "params": {"n_estimators": 200}},
    "data": "diabetes",
    "pipeline": [
        {"name": "SimpleImputer", "params": {"strategy": "median"}},
        {"name": "StandardScaler"},
        {"name": "SMOTESampler", "params": {"random_state": 42}},
    ],
    "evaluation": {"cv": 10, "metrics": ["accuracy_score", "f1_score", "recall_score"]},
    "random_seed": 42,
}

result = tuiml.train(full)
for name, value in result.metrics_.items():
    print(f"  {name:32s} {value:.4f}")

  cv_accuracy_score_mean           0.7603
  cv_accuracy_score_std            0.0489
  cv_f1_score_mean                 0.6617
  cv_f1_score_std                  0.0836
  cv_recall_score_mean             0.6872
  cv_recall_score_std              0.1105

9.2 Specs are validated

A spec is checked before anything runs, and unknown keys are rejected rather than ignored.

try:
    tuiml.train({
        "model": {"name": "LogisticRegression"},
        "data": "diabetes",
        "n_estimators": 500,          # not a spec key — belongs in model.params
    })
except ValueError as error:
    print("ValueError:", error)

ValueError: Unknown spec keys ['n_estimators']. Allowed: ['data', 'evaluation', 'features', 'model', 'pipeline', 'random_seed', 'target'].

This matters more than it looks. A typo'd key that is silently ignored produces a run that succeeds while doing something other than what you asked — the worst possible failure mode, and one that a generated spec is especially prone to. Rejecting loose keys turns that into an immediate error with a list of what was allowed.

9.3 Presets

Common pipelines have names.

for name, steps in tuiml.PRESETS.items():
    described = ", ".join(step["name"] for step in steps) or "(no steps)"
    print(f"  {name:12s} {described}")

  minimal      (no steps)
  fast         SimpleImputer
  standard     SimpleImputer, MinMaxScaler, OneHotEncoder
  full         SimpleImputer, StandardScaler, OneHotEncoder, SelectKBestSelector
  imbalanced   SimpleImputer, MinMaxScaler, SMOTESampler

preset_model = tuiml.train({
    "model": {"name": "RandomForestClassifier"},
    "data": "diabetes",
    "pipeline": "imbalanced",
    "evaluation": {"cv": 10},
    "random_seed": 42,
})

print(f"imbalanced preset: {preset_model.metrics_['cv_accuracy_score_mean']:.4f}")

imbalanced preset: 0.7591

Presets are a starting point, not a recommendation. After chapters 3 through 6 you know that "standard" scales data that a tree does not need scaled, and that "imbalanced" makes a trade you should be making deliberately. Use a preset to get moving; replace it with an explicit list once you know what your data needs.

9.4 Objects and configuration, both directions

Chapter 4 introduced to_config(). Here is the full round trip.

from tuiml.workflow import Workflow
from tuiml.preprocessing import SimpleImputer, StandardScaler
from tuiml.algorithms.linear import LogisticRegression

# Built by hand, out of objects.
built = Workflow([
    SimpleImputer(strategy="median"),
    StandardScaler(),
    LogisticRegression(),
])

config = built.to_config()
print(json.dumps(config, indent=2))

{
  "model": {
    "name": "LogisticRegression"
  },
  "pipeline": [
    {
      "name": "SimpleImputer",
      "params": {
        "strategy": "median"
      }
    },
    {
      "name": "StandardScaler"
    }
  ]
}

# That configuration, handed straight to train().
from_config = tuiml.train({
    **config,
    "data": "diabetes",
    "evaluation": {"cv": 10},
    "random_seed": 42,
})

print(f"score: {from_config.metrics_['cv_accuracy_score_mean']:.4f}")

score: 0.7708

Objects when you are exploring interactively, configuration when you want to store, compare or transmit. Neither is primary.

9.5 An experiment in a file

Because a spec is JSON, an experiment is a file.

work_dir = Path(tempfile.mkdtemp())
experiment_path = work_dir / "experiment.json"
experiment_path.write_text(json.dumps(full, indent=2))

reloaded = tuiml.train(str(experiment_path))
print(f"ran from {experiment_path.name}: "
      f"{reloaded.metrics_['cv_accuracy_score_mean']:.4f}")

ran from experiment.json: 0.7603

Which gives you things that are awkward when an experiment is a script.

It diffs. Two runs differ by exactly the keys that differ:

variant = json.loads(json.dumps(full))
variant["model"]["params"]["n_estimators"] = 500
variant["pipeline"][0]["params"]["strategy"] = "mean"


def diff(a, b, path=""):
    """Report every leaf where two nested structures disagree."""
    if isinstance(a, dict) and isinstance(b, dict):
        for key in a.keys() | b.keys():
            diff(a.get(key), b.get(key), f"{path}.{key}")
    elif isinstance(a, list) and isinstance(b, list) and len(a) == len(b):
        for i, (x, z) in enumerate(zip(a, b)):
            diff(x, z, f"{path}[{i}]")
    elif a != b:
        print(f"  {path.lstrip('.'):48s} {a!r} -> {b!r}")


diff(full, variant)

  model.params.n_estimators                        200 -> 500
  pipeline[0].params.strategy                      'median' -> 'mean'

It is reproducible. The same spec produces the same numbers:

first = tuiml.train(full).metrics_
second = tuiml.train(full).metrics_

print("identical metrics on a re-run:", first == second)

identical metrics on a re-run: True

Remark — reproducibility needs the seed to be in the spec. random_seed propagates to the split, the folds, and every component that takes a seed. Drop it and you get a different answer every run, which is how two people comparing "the same" configuration end up disagreeing by a point and a half. If a result is going in a report, the seed belongs in the file next to it.

It is a record. A spec plus the version of TuiML that ran it is a complete description of the experiment. That is the piece missing when your experiment is a notebook cell someone has since edited.

9.6 Why this is the agent interface

Now the reason this chapter exists.

Consider what a language model would have to do to drive an ordinary ML library. It would need to write Python: import the right names from the right modules, hold the API surface in its head, produce syntactically valid code, and hand that code to something willing to execute arbitrary text. Every one of those steps is a place to go wrong, and the last one is a security problem.

Now consider what it takes to drive TuiML. It emits this:

agent_output = {
    "model": {"name": "RandomForestClassifier", "params": {"n_estimators": 200}},
    "data": "diabetes",
    "pipeline": [{"name": "SimpleImputer", "params": {"strategy": "median"}}],
    "evaluation": {"cv": 10},
    "random_seed": 42,
}

print(json.dumps(agent_output))

{"model": {"name": "RandomForestClassifier", "params": {"n_estimators": 200}}, "data": "diabetes", "pipeline": [{"name": "SimpleImputer", "params": {"strategy": "median"}}], "evaluation": {"cv": 10}, "random_seed": 42}

That is JSON. Language models are extremely good at producing JSON that matches a schema — it is what tool-calling APIs are built on — and JSON cannot execute anything. The spec is validated before it runs, so a hallucinated key is a ValueError rather than a silent wrong answer, and a hallucinated component name fails against the registry rather than becoming an AttributeError three steps later.

And the component names are discoverable, so the model does not have to have memorised them:

info = tuiml.describe_algorithm("RandomForestClassifier")

print("the agent can look up what it is allowed to write:")
for param, meta in list(info["parameters"].items())[:5]:
    print(f"  {param:20s} default={meta.get('default')!r}")

the agent can look up what it is allowed to write:
  n_estimators         default=100
  max_features         default='sqrt'
  max_depth            default=None
  min_samples_split    default=2
  min_samples_leaf     default=1

Put those together and the shape of the second half of the book follows:

1. Components are named, so they can be discovered rather than imported (chapter 0).
2. An experiment is one validated dictionary, so it can be generated rather than written (this chapter).
3. Therefore an agent needs only two capabilities: look things up, and submit a spec. That is the tuiml_list / tuiml_describe / tuiml_train triple from chapter 0.

None of this was designed for agents after the fact. The declarative API is useful on its own terms — reproducibility, diffing, config-driven experiments — and it happens to be exactly the interface a language model can use safely.

Remark — the ML does not get easier. Everything chapters 2 through 8 said about leakage, baselines, imbalance and significance applies identically to a spec an agent wrote. A model that emits {"evaluation": {"cv": 10}} is running the same cross-validation you would, and can be fooled by the same things. Chapter 11 is about checking its work.

Recap

• A spec has six keys: model and data required; pipeline, evaluation, features, random_seed optional.
• Every component is written the same way: {"name": ..., "params": {...}}.
• Unknown keys are rejected, not ignored — a generated spec fails loudly instead of quietly doing the wrong thing.
• PRESETS name common pipelines. Use them to get started, then be explicit.
• to_config() goes object → dict; tuiml.train(config) goes dict → fitted model. Both directions, no primary form.
• A spec is JSON, so an experiment is a file: diffable, storable, reviewable.
• Reproducibility requires random_seed in the spec. Without it, "the same configuration" is not the same run.
• The spec is what makes agent-driven ML tractable: JSON instead of code, validated instead of executed, discoverable instead of memorised.

Next: chapter 10 connects a real agent — Claude Desktop, ChatGPT or Cursor — to the MCP server, and watches it emit its first spec.