12. Agents in Your Own Code

Chapters 10 and 11 put an agent in a chat window. This one puts it in a Python file — for the cases where the ML has to happen inside a service, a scheduled job, or an application you are building, with no human typing prompts.

You will learn:

• how to get the TuiML tools in the shape your framework expects
• a raw tool loop, written out in full, with no framework at all
• the LangChain, Pydantic-AI and CrewAI adapters
• what the bundled system prompt costs you, and when to trim it
• the safety considerations that change once nobody is watching

Prerequisites: chapters 10 and 11.

import json
from tuiml.agent import get_workflow_tools

print(len(get_workflow_tools()), "tools available to any of the paths below")

30 tools available to any of the paths below

12.1 Two ways in

MCP (chapter 10) is for configuring an agent — you point an existing application at the server and it discovers the tools. Nothing to write.

Adapters are for building one. They hand you the same tools already shaped for your framework, with no MCP server, no subprocess, and no protocol in the middle. Everything runs in your process.

tuiml.agent.adapters.openai        OpenAI function-calling schemas
tuiml.agent.adapters.anthropic     Anthropic tool definitions
tuiml.agent.adapters.langchain     LangChain BaseTool objects
tuiml.agent.adapters.pydantic_ai   Pydantic-AI tools
tuiml.agent.adapters.crewai        CrewAI tools

Each is an optional extra and imports lazily:

pip install 'tuiml[anthropic]'     # or openai, langchain, pydantic-ai, crewai
pip install 'tuiml[frameworks]'    # all five

Every adapter exposes the same two functions: get_tools() and system_prompt().

from tuiml.agent.adapters import anthropic, openai

print("anthropic:", len(anthropic.get_tools()), "tools")
print("openai   :", len(openai.get_tools()), "tools")

anthropic: 30 tools
openai   : 30 tools

The shapes differ, because the APIs differ:

print("Anthropic — tools are flat:")
print(json.dumps(anthropic.get_tools()[0], default=str)[:180], "...")
print()
print("OpenAI — tools are wrapped in a function envelope:")
print(json.dumps(openai.get_tools()[0], default=str)[:180], "...")

Anthropic — tools are flat:
{"name": "tuiml_train", "description": "Train a machine learning model with evaluation. Two evaluation modes:\n1. Holdout (default): splits data into train/test sets using test_siz ...

OpenAI — tools are wrapped in a function envelope:
{"type": "function", "function": {"name": "tuiml_train", "description": "Train a machine learning model with evaluation. Two evaluation modes:\n1. Holdout (default): splits data in ...

Remark — use the adapters, not get_tools_for_llm(format=...). That function takes a format argument but currently returns MCP-shaped schemas whatever you pass it, so format="openai" will hand you something the OpenAI API rejects. The per-framework conversion lives in the adapter modules above.

12.2 A raw tool loop

Before reaching for a framework, it is worth seeing that there is not much to abstract. An agent loop is: send the tools, get back a request to call one, call it, send the result, repeat until the model stops asking.

import anthropic as anthropic_sdk
from tuiml.agent.adapters.anthropic import get_tools, dispatch_tool_use, system_prompt

client = anthropic_sdk.Anthropic()
messages = [{
    "role": "user",
    "content": (
        "Profile the diabetes dataset, then benchmark logistic regression "
        "against a random forest and ZeroRuleClassifier with 10-fold CV, "
        "seed 42. Tell me which differences are statistically real."
    ),
}]

while True:
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=4096,
        system=system_prompt(),
        tools=get_tools(),
        messages=messages,
    )

    messages.append({"role": "assistant", "content": response.content})

    if response.stop_reason != "tool_use":
        print(response.content[0].text)
        break

    results = []
    for block in response.content:
        if block.type == "tool_use":
            print(f"  -> {block.name}({json.dumps(block.input)[:80]})")
            results.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": json.dumps(dispatch_tool_use(block), default=str),
            })

    messages.append({"role": "user", "content": results})

dispatch_tool_use is the only TuiML-specific piece: it takes the model's tool-use block, routes it to execute_tool, and returns the result. The equivalent for OpenAI is dispatch_tool_call.

That loop is roughly thirty lines and has no dependencies beyond the provider SDK. If your needs are "call TuiML tools from a script", you are already done — the frameworks below buy you memory, multi-agent orchestration and retries, none of which you may need.

12.3 Pydantic-AI, and the one-liner

The shortest path to a working agent is tuiml.agent.agent(), which returns a Pydantic-AI agent pre-loaded with every tool and the system prompt.

from tuiml.agent import agent

result = agent().run_sync(
    "Train a random forest on the diabetes dataset with 10-fold CV, "
    "seed 42, and include a ZeroRuleClassifier baseline."
)
print(result.output)

# Or assemble it yourself, to control the model and settings.
from pydantic_ai import Agent
from tuiml.agent.adapters.pydantic_ai import get_tools, system_prompt

my_agent = Agent(
    "anthropic:claude-sonnet-5",
    tools=get_tools(),
    system_prompt=system_prompt(),
)

12.4 LangChain and CrewAI

LangChain gets BaseTool objects, ready for any agent executor:

from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate

from tuiml.agent.adapters.langchain import get_tools, system_prompt

prompt = ChatPromptTemplate.from_messages([
    ("system", system_prompt()),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

llm = ChatAnthropic(model="claude-sonnet-5")
executor = AgentExecutor(
    agent=create_tool_calling_agent(llm, get_tools(), prompt),
    tools=get_tools(),
)

executor.invoke({"input": "Benchmark three classifiers on diabetes, 10-fold CV, seed 42."})

CrewAI takes the same tools and gives them to a role:

from crewai import Agent, Task, Crew
from tuiml.agent.adapters.crewai import get_tools

analyst = Agent(
    role="ML Engineer",
    goal="Build and validate a model the team can defend",
    backstory="Rigorous about baselines, cross-validation and significance testing.",
    tools=get_tools(),
)

Crew(agents=[analyst], tasks=[Task(
    description=(
        "Profile the diabetes dataset, benchmark four classifiers including "
        "ZeroRuleClassifier with 10-fold CV and seed 42, and report which "
        "differences are statistically significant."
    ),
    expected_output="A table of scores with standard deviations and a significance verdict.",
    agent=analyst,
)]).kickoff()

12.5 The system prompt is not free

Every adapter ships a system_prompt() — a full guide to TuiML's components, conventions and idioms. It is what makes an agent competent without a fine-tune. It is also large.

prompt_text = anthropic.system_prompt()

print(f"{len(prompt_text):,} characters")
print(f"roughly {len(prompt_text) // 4:,} tokens, sent on every request")
print()
print(prompt_text[:280], "...")

51,992 characters
roughly 12,998 tokens, sent on every request

---
name: TuiML ML
description: Machine learning toolkit - train, evaluate, and compare models using 200+ algorithms, preprocessors, and datasets
version: 0.2.0
mcp_server: tuiml-mcp
---

# TuiML Framework Guide

TuiML is a Python ML framework with 200+ components across algorith ...

That is a meaningful fixed cost per call — and on a long agent loop it is paid on every turn.

Three ways to handle it:

Use prompt caching. Both Anthropic and OpenAI cache a stable prefix, which is exactly what a fixed system prompt is. This is the right answer for most applications and usually reduces the cost of the prefix by an order of magnitude.

Trim it. If your agent only ever trains and benchmarks, it does not need the sections on writing new algorithms. Slice the string, or write your own short prompt — nothing requires you to use the bundled one.

Drop it and rely on discovery. The tool descriptions and tuiml_list / tuiml_describe already tell a model what exists. The system prompt makes it better at using them, not able to.

Whichever you choose, measure it: the failure mode of a trimmed prompt is an agent that uses the tools less well, which — per chapter 11 — will not announce itself.

12.6 When nobody is watching

An agent in a chat window has a human reading every step. An agent in a cron job does not, and three things change.

Chapter 11's checklist has to become code. The metadata that told you what really ran is a dict, so assert on it:

import numpy as np

from tuiml.agent import execute_tool


def train_and_validate(**kwargs):
    """Run a training tool call and refuse results that are not defensible."""
    result = execute_tool("tuiml_train", **kwargs)

    metadata = result["metadata"]
    if metadata["evaluation_method"] != "cross_validate":
        raise ValueError(f"expected cross-validation, got {metadata['evaluation_method']}")
    if result["random_seed"] != kwargs.get("random_seed"):
        raise ValueError("seed was not honoured; run is not reproducible")

    folds = result["cv_results"]["scores"]["accuracy_score"]
    return {"mean": float(np.mean(folds)), "std": float(np.std(folds)), "folds": len(folds)}


print(train_and_validate(
    algorithm="LogisticRegression", data="diabetes",
    preprocessing=["SimpleImputer", "StandardScaler"], cv=10, random_seed=42,
))

{'mean': 0.7707621326042379, 'std': 0.06102327014484286, 'folds': 10}

# And it rejects the underspecified call from chapter 11.
try:
    train_and_validate(algorithm="LogisticRegression", data="diabetes")
except ValueError as error:
    print("rejected:", error)

rejected: expected cross-validation, got holdout

The authoring tools write code to disk. tuiml_create_algorithm accepts Python source, validates it against a denylist, and imports it. In an interactive session you see that happen. In an unattended loop you do not — so if your agent has no business writing new algorithms, do not give it those tools. get_workflow_tools() returns a dict; filter it.

SAFE = {
    "tuiml_list", "tuiml_describe", "tuiml_profile_data",
    "tuiml_train", "tuiml_benchmark", "tuiml_evaluate", "tuiml_test_statistics",
}

restricted = {name: schema for name, schema in get_workflow_tools().items()
              if name in SAFE}

print(f"{len(restricted)} tools instead of {len(get_workflow_tools())}")
print("excluded:", ", ".join(sorted(set(get_workflow_tools()) - SAFE)[:6]), "...")

7 tools instead of 30
excluded: tuiml_create_algorithm, tuiml_delete_algorithm, tuiml_edit_algorithm, tuiml_export_notebook, tuiml_generate_data, tuiml_get_skeleton ...

Loops need a budget. A model that misreads a tool result can retry indefinitely. Cap the turns, cap the wall-clock, and log every tool call with its arguments — that log is what you will need when a result looks wrong three weeks from now.

Remark — the data question from chapter 10 gets sharper here. In a chat you can see which tools get called. In a service, tuiml_read_data sending rows to a model provider is a decision made by a language model at runtime. If that matters for your data, restrict the tool list rather than trusting the prompt.

Recap

• MCP configures an existing agent; adapters build one in your own process.
• Five adapters — openai, anthropic, langchain, pydantic_ai, crewai — each with get_tools() and system_prompt().
• Use the adapters for framework-shaped tools. get_tools_for_llm(format=...) ignores its format argument.
• A raw tool loop is about thirty lines; dispatch_tool_use / dispatch_tool_call route a model's request to execute_tool.
• tuiml.agent.agent() returns a ready Pydantic-AI agent for the shortest path.
• The bundled system prompt is ~52,000 characters. Cache it, trim it, or drop it — but measure the effect.
• Unattended, turn chapter 11's checklist into assertions, restrict the tool list to what the job needs, and budget the loop.

Next: chapter 13 takes the model that survived all this scrutiny and puts it behind an HTTP endpoint.