Data driven Unit Tests

Is there a way to have data driven unit tests using the unit test framework in MATLAB? I have a TestCase class that is set up to test let say a single file.

I would then like to repeat that TestCase with all the methods on several files. I know I can make many copies of the test and set a property in the class that points to a file. But now I have several TestCase files that have exactly the same code with the only difference is a property that points to a different file.

Ideally I would like to have a list of something like files or values for a property and re-run the tests for each item in the list.

Thanks.

Answer 1

[Andy Campbell]

Hi Stephen,

You may be able to achieve this using a common test as a base class that has an Abstract property or an abstract method. It might look like so:

In FileTest.m

classdef FileTest < matlab.unittest.TestCase

      properties(Abstract)
          File
      end

      methods(Abstract)
          data = produceRequiredData(testCase)
      end

      methods(Test)
          function testOne(testCase)
              file = testCase.File;
              success = operateOnFile(file);
              testCase.verifyTrue(success);
          end
          function testTwo(testCase)
              data = testCase.produceRequiredData();
              success = operateOnData(data);
              testCase.verifyTrue(success);
          end
      end
  end

Then you can quickly implement several subclasses for each of these files/data without duplicating the test content. Two such subclasses might look like:

In FirstFileTest.m:

classdef FirstFileTest < FileTest

      properties
          File = 'some/path/to/data/file.foo';
      end

      methods
          function data = produceRequiredData(~)
              data = generateDataOneWay;
          end
      end

end

SecondFileTest.m

classdef SecondFileTest < FileTest

      properties
          File = 'some/path/to/another/data/file.bar';
      end

      methods
          function data = produceRequiredData(~)
              data = generateDataAnotherWay;
          end
      end

end

Note that for both of these tests they do in fact each contain the test methods defined in the base class. Also, the FileTest class is not an executable test itself because of its abstract property and method, so it is not picked up to run as a test. However, all of its concrete subclasses are. Using this approach you can even add specific Test methods to individual test subclasses if these classes need some testing specific to the subclass.

Hope that helps!

Andy

[Stephen]

This worked perfectly. Actually both this solution and the other provided below worked, but this one allowed me to easily use the

TestSuite.fromFolder(pwd)

option and then utilize the remaining features of the test class. The solution on looping over the files, while worked, didn't offer me the same ability to see the progress on the running, and other features built into the unittest framework. Essentially I had an array of results that I had to manipulate.

Very appreciative of both responses.

Answer 2

[David Hruska]

Another possibility, as of release 2014a, is to write a parameterized test. Here's a basic skeleton showing how such a test might look:

classdef FileTest < matlab.unittest.TestCase
    properties(TestParameter)
        File = {'List','Of','Files','To','Be','Tested'};
    end

      methods(Test)
          function testOne(testCase, File)
              % The framework calls this test method once for each value of
              % the "File" property, each time passing in a single value to
              % be used in this test.
              disp(['The value of File is: ', File]);
          end
      end
  end

And here's the output of running the test:

>> run(FileTest)
Running FileTest
The value of File is: List
.The value of File is: Of
.The value of File is: Files
.The value of File is: To
.The value of File is: Be
.The value of File is: Tested
.
Done FileTest
__________

ans = 
  1x6 TestResult array with properties:

      Name
      Passed
      Failed
      Incomplete
      Duration
  Totals:
     6 Passed, 0 Failed, 0 Incomplete.
     0.001732 seconds testing time.

The framework also includes features for selecting various portions of your suite based on the parameterization.

Answer 3

[Sean de Wolski]

I think if I was going to do this, I would have a separate class that contains the list of files and a run() method.

This class' run() would create the TestCase object (your custom class inheriting from matlab.unittest.TestCase ) whose constructor would take the filename, assign this as a property, and then when it was run, would apply the various methods to this file. For example with the two attached files

F = Filer(pwd)
run(F)
F.Results

To see it fail delete a *.txt file from pwd after creation of the Filer object.