Main Content

Experiment Manager

Design and run experiments to train and compare machine learning models

Since R2023a

Description

The Experiment Manager app enables you to create machine learning experiments to train models under multiple initial conditions and compare the results. For example, you can use machine learning experiments to:

  • Try a range of hyperparameter values using Bayesian optimization.

  • Compare the results of using different data sets, preprocessing steps, or metrics.

To set up your experiment quickly, start by training a model in Classification Learner or Regression Learner. Then, export the model to Experiment Manager.

Experiment Manager provides visualization tools such as confusion matrices for classification and predicted vs. actual plots for regression, filters to refine your experiment results, and annotations to record your observations. To improve reproducibility, Experiment Manager stores a copy of the experiment definition every time that you run an experiment. You can access past experiment definitions to keep track of the hyperparameter combinations that produce each of your results.

Experiment Manager organizes your experiments and results in projects.

  • You can store several experiments in the same project.

  • Each experiment contains a set of results for each time that you run the experiment.

  • Each set of results consists of one or more trials corresponding to a different combination of hyperparameters.

By default, Experiment Manager runs one trial at a time. If you have Parallel Computing Toolbox™, you can configure your experiment to run multiple trials at the same time.

The Experiment Browser pane displays the hierarchy of experiments and results in the project. The green beaker indicates a custom training experiment. To open the configuration for an experiment and view its results, double-click the name of the experiment or a set of results.

Required Products

  • MATLAB®

  • Statistics and Machine Learning Toolbox™

Experiment Manager app

Open the Experiment Manager App

  • MATLAB Toolstrip: On the Apps tab, under Machine Learning and Deep Learning, click the app icon.

  • MATLAB command prompt: Enter experimentManager.

Version History

Introduced in R2023a

expand all