Test Impact Analysis Using MATLAB Build Tool

Since R2025a

With MATLAB^® Test™, you can accelerate testing in your automated builds by integrating test impact analysis using the MATLAB build tool. With test impact analysis enabled, the build tool detects changes to tests and exercised source code, finds the tests impacted by those changes, and runs only the impacted tests rather than running the entire test suite. Incremental validation through test impact analysis reduces testing overhead, allowing for faster regression testing and more frequent integrations.

This topic shows how to enable test impact analysis and run only the impacted tests using the MATLAB build tool. You must use a built-in task created from the matlab.buildtool.tasks.TestTask class for impact-based testing. Tasks that are not created from the matlab.buildtool.tasks.TestTask class do not support test impact analysis.

What Are Impacted Tests?

Impacted tests are the tests associated with a TestTask instance that are affected by changes to specific file-based task inputs (test, source, and supporting files) since the last successful task run or a specific source control revision. Finding impacted tests involves detecting changes to these task inputs (change detection) followed by analyzing the impact of the detected changes on the tests (impact analysis).

Change Detection

If you enable test impact analysis, then the build tool detects changes to the following file-based inputs of the task (TestTask instance), which represent code and data, in a change detection window:

• Tests — Test files to run, specified in the Tests property of the task. In addition to the test files, the build tool also tracks test class folders and test superclasses.

• Source files — Source files under test, specified in the SourceFiles property of the task.

• Supporting files — Supporting files used by the tests, such as test data and helper files, specified in the SupportingFiles property of the task.

The build tool detects changes to an input when you add a new file to or modify an existing file in that input. File deletions are not taken into account.

By default, the change detection window—that is, the window in which the task detects changes to code and data for test impact analysis—is the period since the last successful task run. You can also specify a base source control revision for test impact analysis. If you specify a revision, the task considers code and data changes since the specified revision and runs the impacted tests in the repository accordingly. (since R2026a)

Note

You can specify a revision for test impact analysis only when using Git™ source control. For more information on how to use Git source control in MATLAB, see Set Up Git Source Control.

Specifying a revision for test impact analysis requires the plan root folder to be inside a Git repository with a revision history. When you specify a revision, the task detects changes to tests, source files, and supporting files since the specified revision by taking into account the following files in the Git repository:

• Committed files

• Modified files

• Untracked files

During development, with multiple iterative builds before a commit and multiple incremental commits before a push, the build tool lets you run the tests impacted by changes since the last successful task run, commit, or push. For example, each time you prepare to commit new changes, you can run only the tests impacted by changes since your previous commit using the relevant change detection window. For an example of impact-based testing using various change detection windows, see Run Impacted Tests Using TestTask Instance.

Detecting changes to task inputs depends on the task trace. A task trace is a record of the inputs, outputs, actions, and arguments of a task from its last successful run. If a TestTask instance does not have a trace, then the build tool considers all of its inputs to be changed for the purpose of test impact analysis. For example, the build tool considers all the test, source, and supporting files to be changed when:

• The task runs for the first time after creation.

• The task runs after its trace is deleted.

In either of these scenarios, the task finds and runs the tests that are impacted by changes that occurred in the change detection window. If you specify a source control revision, the task runs only the tests impacted by task input changes since that revision. Otherwise, the task runs all the tests in the test suite.

Note

If a TestTask instance does not support incremental builds, test impact analysis is not possible, and the task runs all the tests when it reruns. For a TestTask instance to support incremental builds, its SourceFiles property must be nonempty and its DisableIncremental property must be false. For more information about incremental builds, see Improve Performance with Incremental Builds.

Impact Analysis

To find the tests that are impacted by changes to code and data in the change detection window, the task runs a dependency analysis similar to that of the Dependency Analyzer app. Based on this analysis, the task runs only these tests:

• Modified or new tests

• Tests with a modified test class folder or test superclass

• Tests that depend on modified or new source files (specified in the SourceFiles property of the task)

• Tests that depend on modified or new supporting files (specified in the SupportingFiles property of the task)

For more information about dependency analysis and its limitations, see Dependency Analyzer Scope and Limitations.

Enable Test Impact Analysis

By default, a TestTask instance executes all the tests in the test suite when it runs. However, if you have a MATLAB Test license, you can perform test impact analysis by using the RunOnlyImpactedTests and BaseRevision properties and their corresponding task arguments:

• To enable test impact analysis, specify the RunOnlyImpactedTests property or task argument as true.

• To control the change detection window, specify the base source control revision for test impact analysis using the BaseRevision property or task argument. If you specify a revision, the task considers code and data changes since the specified revision and runs the impacted tests in the repository accordingly. By default, the task considers changes since its last successful run for test impact analysis.

Tip

When you use the BaseRevision task argument, you do not need to specify the RunOnlyImpactedTests property or task argument as true. The task automatically performs impact-based testing given the BaseRevision task argument.

For example, this build file includes a TestTask instance named "test" with its RunOnlyImpactedTests property set to true. The task runs only the impacted tests in the tests folder and its subfolders and fails the build if any of those tests fail.

function plan = buildfile
import matlab.buildtool.tasks.CleanTask
import matlab.buildtool.tasks.TestTask

% Create a plan with no tasks
plan = buildplan;

% Add the source code to the path
addpath("source")

% Add a task to delete outputs and traces
plan("clean") = CleanTask;

% Add a task to run only the impacted tests
plan("test") = TestTask("tests", ...
    SourceFiles="source", ...
    RunOnlyImpactedTests=true);
end

When you run the "test" task, the task runs the tests impacted by changes since the last successful run of the task.

buildtool test

To control the change detection window, specify the BaseRevision task argument as a short, full, or relative commit ID. For example, this code runs the tests impacted by changes since the last commit.

buildtool test(BaseRevision="HEAD")

This code runs the tests impacted by changes on the current branch since it diverged from the target main branch.

buildtool test(BaseRevision="origin/main")

You can use the RunOnlyImpactedTests task argument to control the task behavior at run time. For example, if you specify the task argument as false, the "test" task runs all the tests in the test suite without performing a test impact analysis.

buildtool test(RunOnlyImpactedTests=false)

Task Configurations for Test Impact Analysis

There are different ways to perform test impact analysis using the RunOnlyImpactedTests and BaseRevision properties and task arguments. This section shows common configuration examples for test impact analysis, both at task creation time and at run time. You can use various combinations of properties in the build file and task arguments at run time to achieve your goals. For example, you can specify only the RunOnlyImpactedTests property in the build file and only the BaseRevision task argument at run time to run tests impacted by changes since a specific commit.

Task Creation Time Configurations

Suppose you want to create a TestTask instance named "test" that runs only the impacted tests. This table shows how to configure the "test" task in the build file by using its RunOnlyImpactedTests and BaseRevision properties. To achieve the goal in each scenario, run the task by executing buildtool test.

Goal	Task Creation Time Configuration
Run the tests impacted by changes since the last successful task run.	plan("test") = TestTask("tests", ... SourceFiles="source", ... RunOnlyImpactedTests=true);
Run the tests impacted by changes since the last commit.	plan("test") = TestTask("tests", ... SourceFiles="source", ... RunOnlyImpactedTests=true, ... BaseRevision="HEAD");
Run the tests impacted by changes on the current branch since it diverged from the target main branch.	plan("test") = TestTask("tests", ... SourceFiles="source", ... RunOnlyImpactedTests=true, ... BaseRevision="origin/main");
Run the tests impacted by changes since a specific commit.	plan("test") = TestTask("tests", ... SourceFiles="source", ... RunOnlyImpactedTests=true, ... BaseRevision="08a4c49");

Task Run Time Configurations

Suppose your build file contains a TestTask instance with the RunOnlyImpactedTests and BaseRevision properties unset.

function plan = buildfile
import matlab.buildtool.tasks.CleanTask
import matlab.buildtool.tasks.TestTask

plan = buildplan;

addpath("source")

plan("clean") = CleanTask;

plan("test") = TestTask("tests",SourceFiles="source");
end

Because the "test" task in the build file is not configured for test impact analysis, use the RunOnlyImpactedTests and BaseRevision task arguments to perform impact-based testing instead. This table shows how to configure the "test" task at run time to achieve different goals.

Goal	Task Run Time Configuration
Run the tests impacted by changes since the last successful task run.	buildtool test(RunOnlyImpactedTests=true)
Run the tests impacted by changes since the last commit.	buildtool test(BaseRevision="HEAD")
Run the tests impacted by changes on the current branch since it diverged from the target main branch.	buildtool test(BaseRevision="origin/main")
Run the tests impacted by changes since a specific commit.	buildtool test(BaseRevision="08a4c49")