Generate Artifacts Using MATLAB Unit Test Plugins

The MATLAB® unit testing framework enables you to customize your test runner using the plugin classes in the matlab.unittest.plugins namespace. You can use some of these plugin classes to generate test reports and artifacts compatible with continuous integration (CI) platforms:

• matlab.unittest.plugins.TestReportPlugin creates a plugin that directs the test runner to produce a test result report. Using this plugin, you can produce readable and archivable test reports.

• matlab.unittest.plugins.TAPPlugin creates a plugin that produces a Test Anything Protocol (TAP) stream.

• matlab.unittest.plugins.XMLPlugin creates a plugin that produces JUnit-style XML output.

• matlab.unittest.plugins.CodeCoveragePlugin creates a plugin that produces a coverage report for MATLAB source code.

You also can generate CI-compatible artifacts when you run Simulink® Test™ test cases. For more information, see Output Results for Continuous Integration Systems (Simulink Test).

Run Tests with Customized Test Runner

This example shows how to create a test suite and customize the test runner to report on test run progress and produce CI-compatible artifacts.

In a file in your current folder, create the function quadraticSolver, which returns the roots of quadratic polynomials.

function r = quadraticSolver(a,b,c)
% quadraticSolver returns solutions to the
% quadratic equation a*x^2 + b*x + c = 0.

if ~isa(a,"numeric") || ~isa(b,"numeric") || ~isa(c,"numeric")
    error("quadraticSolver:InputMustBeNumeric", ...
        "Coefficients must be numeric.")
end

r(1) = (-b + sqrt(b^2 - 4*a*c)) / (2*a);
r(2) = (-b - sqrt(b^2 - 4*a*c)) / (2*a);

end

To test quadraticSolver, create the test class SolverTest in your current folder.

classdef SolverTest < matlab.unittest.TestCase
    methods (Test)
        function realSolution(testCase)
            actSolution = quadraticSolver(1,-3,2);
            expSolution = [2 1];
            testCase.verifyEqual(actSolution,expSolution)
        end
        function imaginarySolution(testCase)
            actSolution = quadraticSolver(1,2,10);
            expSolution = [-1+3i -1-3i];
            testCase.verifyEqual(actSolution,expSolution)
        end
        function nonnumericInput(testCase)
            testCase.verifyError(@()quadraticSolver(1,"-3",2), ...
                "quadraticSolver:InputMustBeNumeric")
        end
    end
end

At the command prompt, create a test suite from the SolverTest class.

suite = testsuite("SolverTest");

Create a TestRunner instance that produces output using the matlab.unittest.TestRunner.withTextOutput method. This method enables you to set the maximum verbosity level for logged diagnostics and the display level for test event details. In this example, the test runner displays test run progress at the matlab.automation.Verbosity.Detailed level (level 3).

import matlab.unittest.TestRunner
runner = TestRunner.withTextOutput("OutputDetail",3);

Create a TestReportPlugin instance that sends output to the file testreport.pdf and add the plugin to the test runner.

import matlab.unittest.plugins.TestReportPlugin
pdfFile = "testreport.pdf";
p1 = TestReportPlugin.producingPDF(pdfFile);
runner.addPlugin(p1)

Create an XMLPlugin instance that writes JUnit-style XML output to the file junittestresults.xml. Then, add the plugin to the test runner.

import matlab.unittest.plugins.XMLPlugin
xmlFile = "junittestresults.xml";
p2 = XMLPlugin.producingJUnitFormat(xmlFile);
runner.addPlugin(p2)

Create a plugin that outputs a Cobertura code coverage report for the source code in the file quadraticSolver.m. Instruct the plugin to write its output to the file cobertura.xml and add the plugin to the test runner.

import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoberturaFormat
sourceCodeFile = "quadraticSolver.m";
reportFile = "cobertura.xml";
reportFormat = CoberturaFormat(reportFile);
p3 = CodeCoveragePlugin.forFile(sourceCodeFile,"Producing",reportFormat);
runner.addPlugin(p3)

Run the tests.

results = runner.run(suite)

 Running SolverTest
  Setting up SolverTest
  Done setting up SolverTest in 0 seconds
   Running SolverTest/realSolution
   Done SolverTest/realSolution in 0.016834 seconds
   Running SolverTest/imaginarySolution
   Done SolverTest/imaginarySolution in 0.0043659 seconds
   Running SolverTest/nonnumericInput
   Done SolverTest/nonnumericInput in 0.0086213 seconds
  Tearing down SolverTest
  Done tearing down SolverTest in 0 seconds
 Done SolverTest in 0.029822 seconds
__________

Generating test report. Please wait.
    Preparing content for the test report.
    Adding content to the test report.
    Writing test report to file.
Test report has been saved to:
 C:\work\testreport.pdf

results = 

  1×3 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   3 Passed, 0 Failed, 0 Incomplete.
   0.029822 seconds testing time.

List the files in your current folder. The three specified artifacts are stored in your current folder.

dir

.                                                     
..                                                    
GenerateArtifactsUsingMATLABUnitTestPluginsExample.m  
SolverTest.m                                          
cobertura.xml                                         
html                                                  
junittestresults.xml                                  
metadata                                              
quadraticSolver.m                                     
testreport.pdf                                        

You can process the generated artifacts on CI platforms. You also can view the contents of the generated artifacts. For example, open the PDF test report.

open("testreport.pdf")