Main Content

Back-to-Back Test Status Distribution for Normal and PIL Mode

Since R2024a

Metric ID

slcomp.pil.B2BTestStatusDistribution

Description

The back-to-back testing metrics perform translation validation between a model and the generated code.

This metric returns the distribution of back-to-back testing statuses for each unit by comparing, at each time step, the outputs of the model simulation and the outputs of the code executed in processor-in-the-loop (PIL) mode. The metric compares the normal mode and PIL mode test runs from baseline, equivalence, and simulation tests.

Supported Artifacts

You can collect this metric for the Units in your project. To control what the dashboard classifies as a unit, see Categorize Models in Hierarchy as Components or Units.

Computation Details

For information on how the metric calculates the back-to-back testing statuses, see Back-to-Back Test Status for Normal and PIL Mode.

Collection

To collect data for this metric, execute the metric engine and use getMetrics with the metric ID slcomp.pil.B2BTestStatusDistribution.

metric_engine = metric.Engine;
execute(metric_engine,"slcomp.pil.B2BTestStatusDistribution");
results = getMetrics(metric_engine,"slcomp.pil.B2BTestStatusDistribution")

Collecting data for this metric loads the model file and test result files and requires a Simulink® Test™ license.

Results

For this metric, the function getMetrics returns a metric.Result instance for each unit in the project.

Instances of metric.Result return Value as a distribution structure that contains these fields:

  • BinCounts — The number of failed, passed, and untested tests, returned as a vector that contains these elements:

    • BinCounts(1) — Number of tests that failed back-to-back testing

    • BinCounts(2) — Number of tests that passed back-to-back testing

    • BinCounts(3) — Number of tests that are untested for back-to-back testing

    For example, if one test fails, two tests pass, and three tests are untested, BinCounts returns [1;2;3].

  • BinEdges — The possible back-to-back testing statuses, returned as a vector:

    • 0 — Test failed back-to-back testing

    • 1 — Test passed back-to-back testing

    • 2 — Test is untested for back-to-back testing

  • OverallCount — The total number of tests. The metric calculates OverallCount as the sum of the values in BinCounts.

  • Ratios — The percentage, in decimal form, of tests with a given status, returned as a vector that contains these elements:

    • Ratios(1) — Percentage of tests that failed back-to-back testing

    • Ratios(2) — Percentage of tests that passed back-to-back testing

    • Ratios(3) — Percentage of tests that are untested for back-to-back testing

    For example, if 0% of the tests failed, 10% of the tests passed, and 90% of the tests are untested, Ratios returns a vector with the percentages in decimal form: [0;0.1000;0.9000].

Compliance Thresholds

The default compliance thresholds for this metric are:

  • Compliant — 100% of tests in the unit passed

  • Non-Compliant — One or more tests failed or are untested

  • Warning — None

See Also

|

Related Topics