Experiment Manager

Design and run experiments to train and compare machine learning models

Since R2023a

Description

The Experiment Manager app enables you to create machine learning experiments to train models under multiple initial conditions and compare the results. For example, you can use machine learning experiments to:

• Try a range of hyperparameter values using Bayesian optimization.

• Compare the results of using different data sets, preprocessing steps, or metrics.

To set up your experiment quickly, start by training a model in Classification Learner or Regression Learner. Then, export the model to Experiment Manager.

Experiment Manager provides visualization tools such as confusion matrices for classification and predicted vs. actual plots for regression, filters to refine your experiment results, and annotations to record your observations. To improve reproducibility, Experiment Manager stores a copy of the experiment definition every time that you run an experiment. You can access past experiment definitions to keep track of the hyperparameter combinations that produce each of your results.

More

Experiment Manager organizes your experiments and results in projects.

• You can store several experiments in the same project.

• Each experiment contains a set of results for each time that you run the experiment.

• Each set of results consists of one or more trials corresponding to a different combination of hyperparameters.

By default, Experiment Manager runs one trial at a time. If you have Parallel Computing Toolbox™, you can configure your experiment to run multiple trials at the same time.

The Experiment Browser pane displays the hierarchy of experiments and results in the project. The green beaker indicates a custom training experiment. To open the configuration for an experiment and view its results, double-click the name of the experiment or a set of results.

Required Products

• MATLAB^®

• Statistics and Machine Learning Toolbox™

×

Open the Experiment Manager App

• MATLAB Toolstrip: On the Apps tab, under Machine Learning and Deep Learning, click the app icon.

• MATLAB command prompt: Enter experimentManager.

Examples

• Export Model from Classification Learner to Experiment Manager
• Tune Classification Model Using Experiment Manager
• Export Model from Regression Learner to Experiment Manager
• Tune Regression Model Using Experiment Manager

Version History

Introduced in R2023a

R2023a: App available in Statistics and Machine Learning Toolbox

You can now use Experiment Manager when you have a Statistics and Machine Learning Toolbox license. In previous releases, you needed a Deep Learning Toolbox™ license to use the Experiment Manager app. For more information on how to use the app for deep learning workflows, see Experiment Manager App (Deep Learning Toolbox).