tuning/

• GridSearchCV: Every combination in the grid. Exhaustive and

reproducible, but the cost multiplies with each parameter added.

• RandomSearchCV: Samples a fixed number of combinations. Usually finds a

comparable result far sooner when only a few parameters actually matter, since it spends its budget on distinct values rather than a full lattice.

• BayesianSearchCV: Models the score surface with a Gaussian process and

picks each next trial from it. Fewest evaluations, best when a single fit is expensive.

• SuccessiveHalvingSearchCV: Runs a large pool on a small slice of the

data, discards the worst, and repeats with more data. Most candidates die cheaply.

• HyperbandSearchCV: Several successive-halving schedules at different

aggression levels, so the schedule need not be guessed.

The first three choose which configurations to evaluate; the last two choose how much budget each one gets. They are complementary, not alternatives — the resource-allocation pair samples candidates at random, exactly as RandomSearchCV does.

• ParameterGrid / ParameterDistribution: The spaces to search.
• TuningResult: Best parameters, best score and the per-trial record.
• BaseTuner: The base class to subclass for a new strategy.

Tuning and evaluating on the same split reports an optimistic score: the parameters were chosen using that data. Benchmark avoids this by running tuning inside each outer fold's training half, so the reported score stays honest.