How to Use Equivalence Testing for Code Integrity
Create and run a back-to-back equivalency test using MATLAB Test™. Compare MATLAB® code results with the generated code results from MATLAB Coder™ or MATLAB Compiler SDK™. Equivalence testing to ensure robust code transformation is a recommended practice by standards such as IEC 62304 and ISO® 26262.
Published: 15 Feb 2023
A number of workflows take MATLAB code and transform it into standalone versions. Two common ones are generating C or C++ code using MATLAB Coder and creating a deployed application using MATLAB Compiler SDK. Verifying the behavior of the original MATLAB code is simple. You can write tests for the code in MATLAB using the MATLAB Unit Test Framework.
Verifying the behavior of the deployed code, however, is usually a more challenging prospect. Fortunately, MATLAB Test makes it just as easy to write tests that verify your deployed artifacts using equivalence testing. Equivalence testing is a testing strategy that consists of three phases. First you build the deployed artifact you want to test. Next you execute both the deployed artifact and the original MATLAB code. Finally, you compare the outputs from the original code and the deployed version and make sure that they match. Using this strategy, you can gain confidence that your deployed artifact's behavior is the same as your original MATLAB code.
Let's see an example of equivalence testing using MATLAB Test. Let's say that you wanted to generate C code from this simple function using MATLAB Coder and then verify that the generated code gives the same outputs as the original MATLAB code. This test performs the equivalence verification in a few simple lines. Using a specialized test case class, the test generates C code for the function, executes the generated code against the set of inputs, and then verifies that the results match the original MATLAB function.
The execution of the test can further be customized to collect coverage on the generated code. This can help you assess how thorough your equivalence testing is and can identify testing holes. Let's run the test and verify our deployed code.
We can see from the report that our equivalence testing isn't covering some parts of the generated code. Let's quickly fix that by adding in a test point and running the test again. Now we have full coverage of the generated code and we can be confident it's behaving the same as our MATLAB function.
To summarize, MATLAB Test provides a number of tools to help verify deployment workflows. You can easily alter tests to verify equivalence between your MATLAB code and your deployed artifacts. It includes support for code generation with MATLAB Coder and application deployment with MATLAB Compiler SDK. Additionally, for code generation workflows, MATLAB Test helps you collect coverage on your generated code to ensure you have comprehensive testing.