Define Parameters at Suite Creation Time

Parameterized tests let you run the same test procedure repeatedly, using different data values each time. In a parameterized test, these data values are called parameters and are represented by parameterization properties of the test class. MATLAB® uses parameterization properties to generate the parameter names and values for each test run.

In most cases, MATLAB can determine the value of a parameterization property when it loads the test class definition. Therefore, you can initialize the property using a default value. When you initialize a parameterization property with a default value, the parameters associated with the property remain fixed for different test runs. Each time you create a suite from the parameterized test class, the testing framework uses the same parameter names and values to run the tests.

In some cases, MATLAB cannot determine the value of a parameterization property when it loads the test class definition. For example, sometimes a parameterization property depends on another property defined at a higher parameterization level. Or you might not want the parameters to be determined at class load time. For instance, if parameters represent files in a folder, you might want to refresh the parameters each time you create a suite to test the files. When you cannot or do not want to initialize a parameterization property at class load time, initialize it at suite creation time using a static method with the TestParameterDefinition attribute. When you initialize a parameterization property using a TestParameterDefinition method, the parameters associated with the property can vary for different test runs. In other words, each time you create a test suite from the parameterized test class, the framework generates fresh parameter names and values to run the tests.

This example shows how to use parameterization properties with default and nondefault values to verify that the public properties of a group of classes in your current folder are nonempty. In it, you define a parameterized test class named PropertiesTest in the test subfolder of your current folder. You define three classes to test, named ClassA, ClassB, and ClassC, in the source subfolder of your current folder. For a summary of these three classes, see Classes in source Subfolder.

Create PropertiesTest Class

To test the public properties of classes defined in the source subfolder, create the PropertiesTest class in the test subfolder. This class takes three specified classes, retrieves all properties of each class, and verifies that they are nonempty. To iterate over the classes to test, parameterize PropertiesTest at the class-setup level. To iterate over the properties of each class specified by a given class-setup-level parameterization, parameterize PropertiesTest at the test level.

Define the properties used for parameterized testing:

• List the classes for the framework to iterate over in a property named classToTest. Because this example assumes that the classes in the source subfolder are fixed and known at the time MATLAB loads the test class definition, initialize the property using a default value. In order to specify the class to test before running any Test methods, make classToTest a ClassSetupParameter property.

• Define a TestParameter property named propertyToTest that you can use to iterate over the properties of whatever class the framework is currently testing. Because its value depends on the class being tested, do not assign it a default value. Instead, initialize it at suite creation time using a TestParameterDefinition method.

• To store the value of different properties on an instance of the class being tested, define a property named ObjectToTest.

classdef PropertiesTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        classToTest = {'ClassA','ClassB','ClassC'};
    end

    properties (TestParameter)
        propertyToTest
    end
    
    properties
        ObjectToTest
    end
end

Define Method to Initialize Test-Level Parameterization Property

In the PropertiesTest class, propertyToTest has a different value for each class being tested. Therefore, you cannot assign a default value to it. Instead, you must initialize it at suite creation time. To implement this requirement, add a TestParameterDefinition method named initializeProperty. Because a TestParameterDefinition method must be static, use the combined method attributes TestParameterDefinition,Static to define the method.

The initializeProperty method accepts the class-setup-level parameterization property as an input and uses it in a call to the properties function to return property names in a cell array of character vectors.

classdef PropertiesTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        classToTest = {'ClassA','ClassB','ClassC'};
    end

    properties (TestParameter)
        propertyToTest
    end
    
    properties
        ObjectToTest
    end

    methods (TestParameterDefinition,Static)
        function propertyToTest = initializeProperty(classToTest)
            propertyToTest = properties(classToTest);
        end
    end
end

In the initializeProperty method, both the input argument classToTest and the output argument propertyToTest are parameterization properties defined in the PropertiesTest class. Any time you define a TestParameterDefinition method, all inputs to the method must match parameterization properties defined in the same class or one of its superclasses. Also, all outputs of the method must match parameterization properties defined in the same class.

In the initializeProperty method, the input argument classToTest is defined at the highest parameterization level. This puts it higher than the output argument propertyToTest, which is defined at the lowest parameterization level. Any time you define a TestParameterDefinition method that accepts inputs, the inputs must be at a higher parameterization level relative to the outputs of the method. For more information about parameterization levels, see Use Parameters in Class-Based Tests.

Define Test Class Setup Method

To test for nonempty property values, you must first create an object of the class being tested so that you can retrieve the property values. To implement this requirement, add the parameterized classSetup method to the PropertiesTest class. In order to have the object ready before running any Test methods, make classSetup a TestClassSetup method.

The classSetup method creates an instance of the class being tested and stores it in the ObjectToTest property. Tests can later retrieve the property values from ObjectToTest. In this example, the framework runs the tests by calling the classSetup method three times—one time for each of ClassA, ClassB, and ClassC.

classdef PropertiesTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        classToTest = {'ClassA','ClassB','ClassC'};
    end

    properties (TestParameter)
        propertyToTest
    end
    
    properties
        ObjectToTest
    end

    methods (TestParameterDefinition,Static)
        function propertyToTest = initializeProperty(classToTest)
            propertyToTest = properties(classToTest);
        end
    end

    methods (TestClassSetup)
        function classSetup(testCase,classToTest)
            constructor = str2func(classToTest);
            testCase.ObjectToTest = constructor();
        end
    end
end

Test for Nonempty Property Values

To test that the properties on ObjectToTest are nonempty, add a Test method named testProperty. In order for the method to iterate over the properties of ObjectToTest, make the method parameterized, and pass it propertyToTest.

During each test, the testProperty method retrieves the value of a property on ObjectToTest. Then, it uses a call to the verifyNotEmpty qualification method to verify that the value is not empty. For a given class-setup-level parameterization, the framework calls testProperty once for each property on the class being tested.

classdef PropertiesTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        classToTest = {'ClassA','ClassB','ClassC'};
    end

    properties (TestParameter)
        propertyToTest
    end
    
    properties
        ObjectToTest
    end

    methods (TestParameterDefinition,Static)
        function propertyToTest = initializeProperty(classToTest)
            propertyToTest = properties(classToTest);
        end
    end

    methods (TestClassSetup)
        function classSetup(testCase,classToTest)
            constructor = str2func(classToTest);
            testCase.ObjectToTest = constructor();
        end
    end

    methods (Test)
        function testProperty(testCase,propertyToTest)
            value = testCase.ObjectToTest.(propertyToTest);
            testCase.verifyNotEmpty(value)
        end
    end
end

Create Parameterized Test Suite and Run Tests

Now that the PropertiesTest class definition is complete, you can create a parameterized test suite and run the tests. To do this, make sure that the source and test subfolders are on the path.

addpath("source","test")

Create a suite from the PropertiesTest class.

suite = testsuite("PropertiesTest");

The test suite includes eight elements. Each element corresponds to a property defined within the source subfolder. Return the name of the first suite element.

suite(1).Name

ans = 
'PropertiesTest[classToTest=ClassA]/testProperty(propertyToTest=PropA1)'

The name of the first element is composed of these parts:

• PropertiesTest — Test class name

• [classToTest=ClassA] — Class-setup-level property and parameter name

• testProperty — Test method name

• (propertyToTest=PropA1) — Test-level property and parameter name

Run the tests. Because two properties in the source subfolder are empty, two of the tests fail.

suite.run

Running PropertiesTest
..
================================================================================
Verification failed in PropertiesTest[classToTest=ClassA]/testProperty(propertyToTest=PropA3).
    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyNotEmpty failed.
    --> The value must not be empty.
    --> The value has a size of [0  0].
    
    Actual Value:
         []
    ------------------
    Stack Information:
    ------------------
    In C:\TEMP\Examples\matlab-ex41465327\test\PropertiesTest.m (PropertiesTest.testProperty) at 30
================================================================================
....
================================================================================
Verification failed in PropertiesTest[classToTest=ClassC]/testProperty(propertyToTest=PropC1).
    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyNotEmpty failed.
    --> The value must not be empty.
    --> The value has a size of [0  0].
    
    Actual Value:
         []
    ------------------
    Stack Information:
    ------------------
    In C:\TEMP\Examples\matlab-ex41465327\test\PropertiesTest.m (PropertiesTest.testProperty) at 30
================================================================================
..
Done PropertiesTest
__________

Failure Summary:

     Name                                                                    Failed  Incomplete  Reason(s)
    =====================================================================================================================
     PropertiesTest[classToTest=ClassA]/testProperty(propertyToTest=PropA3)    X                 Failed by verification.
    ---------------------------------------------------------------------------------------------------------------------
     PropertiesTest[classToTest=ClassC]/testProperty(propertyToTest=PropC1)    X                 Failed by verification.

ans = 
  1×8 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   6 Passed, 2 Failed (rerun), 0 Incomplete.
   0.2348 seconds testing time.

Run Tests for Specific Class

Run only the tests for ClassB. To do this, use the selectIf method of the matlab.unittest.TestSuite class to select test suite elements that use a particular parameterization. The resulting test suite is a filtered suite and has only three elements.

suite2 = suite.selectIf("ParameterName","PropB*");
{suite2.Name}'

ans = 3×1 cell
    {'PropertiesTest[classToTest=ClassB]/testProperty(propertyToTest=PropB1)'}
    {'PropertiesTest[classToTest=ClassB]/testProperty(propertyToTest=PropB2)'}
    {'PropertiesTest[classToTest=ClassB]/testProperty(propertyToTest=PropB3)'}

Run the filtered suite.

suite2.run;

Running PropertiesTest
...
Done PropertiesTest
__________

Alternatively, you can run the same tests by creating a selector that filters the test suite by parameterization.

import matlab.unittest.selectors.HasParameter
import matlab.unittest.constraints.StartsWithSubstring
suite3 = matlab.unittest.TestSuite.fromClass(?PropertiesTest, ...
    HasParameter("Name",StartsWithSubstring("PropB")));
suite3.run;

Running PropertiesTest
...
Done PropertiesTest
__________

Classes in source Subfolder

This section provides the contents of the classes in the source subfolder.

ClassA has three properties. Two of its properties have nonempty values.

classdef ClassA
    properties
        PropA1 = 1;
        PropA2 = 2;
        PropA3
    end
end

ClassB has three properties. All of its properties have nonempty values.

classdef ClassB
    properties
        PropB1 = 1;
        PropB2 = 2;
        PropB3 = 'a';
    end
end

ClassC has two properties. One of its properties has a nonempty value.

classdef ClassC
    properties
        PropC1
        PropC2 = [1 2 3];
    end
end