Main Content

Batch Equivalence Testing of Multiple Components

This example shows how to create equivalence test cases and test harnesses for multiple components in batch mode using sltest.testmanager.createTestForComponent. Coverage collection and report generation are also included in the example.

The sltestCruiseControl model used in this example has Atomic Subsystem blocks, Virtual Subsystems blocks & Model Reference Blocks .

Note that this example is configured only for Windows machines.

Open the Model

topModel = "sltestCruiseControl";
load_system(topModel);

Generate the Code

The equivalence tests in this example compare normal mode code and generated code. Embedded Coder is required to generate the code for the model. The model has been configured with appropriate C code generation and coder mapping settings for Windows 64-bit systems. You can change these settings, if desired, before generating the code.

slbuild(topModel);
### Starting serial model reference code generation build
### Checking the status of model reference code generation target for model 'sltestCruiseControlMode' used in 'sltestCruiseControl'
### Model reference code generation target (sltestCruiseControlMode.c) for model sltestCruiseControlMode is out of date because sltestCruiseControlMode.c does not exist.
### Starting build procedure for: sltestCruiseControlMode
### Generating code and artifacts to 'Model specific' folder structure
### Generating code into build folder: C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\slprj\ert\sltestCruiseControlMode
### Invoking Target Language Compiler on sltestCruiseControlMode.rtw
### Using System Target File: S:\matlab\rtw\c\ert\ert.tlc
        ### Loading TLC function libraries
.......
### Initial pass through model to cache user defined code
..
### Caching model source code
...............................................................................
......
### Writing header file sltestCruiseControlMode_types.h
### Writing header file sltestCruiseControlMode.h
### Writing header file rtwtypes.h
.
### Writing source file sltestCruiseControlMode.c
### Writing header file sltestCruiseControlMode_private.h
### TLC code generation complete.
..### Creating HTML report file index.html
### Saving binary information cache.
### Using toolchain: Microsoft Visual C++ 2017 v15.0 | nmake (64-bit Windows)
### Creating 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\slprj\ert\sltestCruiseControlMode\sltestCruiseControlMode.mk' ...
### Successful completion of code generation for: sltestCruiseControlMode
### Checking the status of model reference code generation target for model 'sltestDriverSwRequest' used in 'sltestCruiseControl'
### Model reference code generation target (sltestDriverSwRequest.c) for model sltestDriverSwRequest is out of date because sltestDriverSwRequest.c does not exist.
### Starting build procedure for: sltestDriverSwRequest
### Generating code and artifacts to 'Model specific' folder structure
### Generating code into build folder: C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\slprj\ert\sltestDriverSwRequest
### Invoking Target Language Compiler on sltestDriverSwRequest.rtw
### Using System Target File: S:\matlab\rtw\c\ert\ert.tlc
        ### Loading TLC function libraries
.......
### Initial pass through model to cache user defined code
..
### Caching model source code
...................................................
### Writing header file sltestDriverSwRequest_types.h
### Writing source file sltestDriverSwRequest.c
### Writing header file sltestDriverSwRequest_private.h
### Writing header file sltestDriverSwRequest.h
.
### TLC code generation complete.
### Creating HTML report file index.html
### Saving binary information cache.
### Using toolchain: Microsoft Visual C++ 2017 v15.0 | nmake (64-bit Windows)
### Creating 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\slprj\ert\sltestDriverSwRequest\sltestDriverSwRequest.mk' ...
### Successful completion of code generation for: sltestDriverSwRequest
### Simulink cache artifacts for 'sltestCruiseControlMode' were created in 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\sltestCruiseControlMode.slxc'.
### Simulink cache artifacts for 'sltestDriverSwRequest' were created in 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\sltestDriverSwRequest.slxc'.
### Starting build procedure for: sltestCruiseControl
### Generating code and artifacts to 'Model specific' folder structure
### Generating code into build folder: C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\sltestCruiseControl_ert_rtw
### Invoking Target Language Compiler on sltestCruiseControl.rtw
### Using System Target File: S:\matlab\rtw\c\ert\ert.tlc
        ### Loading TLC function libraries
.......
### Generating TLC interface API for custom data
.
### Initial pass through model to cache user defined code
..
### Caching model source code
...............................................................................
....................
### Writing header file sltestCruiseControl_types.h
### Writing header file sltestCruiseControl.h
.
### Writing source file sltestCruiseControl.c
### Writing header file sltestCruiseControl_private.h
### Writing source file ert_main.c
### TLC code generation complete.
.### Saving binary information cache.
### Using toolchain: Microsoft Visual C++ 2017 v15.0 | nmake (64-bit Windows)
### Creating 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\sltestCruiseControl_ert_rtw\sltestCruiseControl.mk' ...
### Successful completion of code generation for: sltestCruiseControl
### Simulink cache artifacts for 'sltestCruiseControl' were created in 'C:\Users\dschwart\Documents\MATLAB\ExampleManager\dschwart.Bdoc22a.j1841572\simulinktest-ex89766869\sltestCruiseControl.slxc'.

Build Summary

Code generation targets built:

Model                    Action          Rebuild Reason                             
====================================================================================
sltestCruiseControlMode  Code generated  sltestCruiseControlMode.c does not exist.  
sltestDriverSwRequest    Code generated  sltestDriverSwRequest.c does not exist.    

Top model targets built:

Model                Action          Rebuild Reason                                    
=======================================================================================
sltestCruiseControl  Code generated  Code generation information file does not exist.  

3 of 3 models built (0 models already up to date)
Build duration: 0h 1m 8.244s

Specify the Components to Test

The example tests all of the atomic subsystems that have nonreusable function packaging in the generated code.

componentsToTest = find_system(topModel,...
    "BlockType","SubSystem",...
    "TreatAsAtomicUnit","on",...
    "RTWSystemCode","Nonreusable function");

To improve traceability of testing artifacts, such as test case and test harness names, customize the default names of the created harnesses. The names will use the component name instead of the owning model name.

sltest.harness.setHarnessCreateDefaults("Name","$Component$_Harness");

Create the Test Cases and Test Harnesses in Batch Mode

Use the sltest.testmanager.createTestForComponent API to create multiple test cases and test harnesses at the same time. Using createTestForComponent, you specify to create a test file and the test file name. You also specify the top model, components to test, and the test type and simulations settings. For the test inputs, you specify to use Simulink Design Verifier to automatically generate the inputs in Microsoft Excel format. If you have other existing inputs, you can add them to the test cases and test both those inputs and the generated inputs.

[tc, status] = ...
    sltest.testmanager.createTestForComponent(...
        "CreateTestFile",true,...
        "TestFile","myB2BTestsDemoEx.mldatx",...
        "TopModel",topModel,...
        "Component",componentsToTest,...
        "TestType","equivalence",...
        "Simulation1Mode","Normal",...
        "Simulation2Mode","Software-in-the-Loop (SIL)",...
        "SLDVTestGeneration","on",...
        "CreateExcelFile",true);

Test Execution

The createTestForComponent API generated the test file, test harnesses, and test input signals in the current working directory. Save the test file.

tf = sltest.testmanager.TestFile("myB2BTestsDemoEx.mldatx");
tf.saveToFile;

Enable coverage collection for test file and run the tests.

cov = getCoverageSettings(tf);
cov.RecordCoverage = true;
cov.MetricSettings = "dcmr";
tf.saveToFile;

resultSet = tf.run;
bdclose all;

sltest.testmanager.exportResults(resultSet,"myB2BResults.mldatx");
sltestmgr;

Generate a report with coverage and equivalence test results.

sltest.testmanager.report(...
    resultSet,"myB2BResultsReport.pdf",...
    "IncludeCoverageResult",true,...
    "IncludeSimulationSignalPlots",true,...
    "IncludeComparisonSignalPlots",true,...
    "IncludeTestResults",0,...
    "IncludeSimulationMetadata",true);

results_rpt.png

Clean Up

close_system(topModel)
sltest.testmanager.clear
sltest.testmanager.clearResults
sltest.testmanager.close

See Also