Optimal Subpattern Assignment Metric

Calculate Optimal Subpattern Assignment Metric

Since R2021a

• 

Libraries:
Sensor Fusion and Tracking Toolbox / Track Metrics

Description

The Optimal Subpattern Assignment Metric block computes the optimal subpattern assignment metric between a set of tracks and known truths. You can enable different types of OSPA metrics by configuring these parameters:

• Traditional OSPA — Specify the Metric parameter as OSPA and specify the Labeling error parameter, on the Properties tab, as 0. The traditional OSPA metric, which evaluates instantaneous tracking performance, contains two components:

  • Localization error component — Accounts for state estimation errors between assigned tracks and truths.

  • Cardinality error component— Accounts for the number of unassigned tracks and truths.

• Labeled OSPA — Specify the Metric parameter as OSPA and specify the Labeling error parameter, on the Properties tab, as a positive scalar. The labeled OSPA (LOSPA) metric, which evaluates instantaneous tracking performance and includes penalties for incorrect assignments, contains three components:

  • Localization error component — Accounts for state estimation errors between assigned tracks and truths.

  • Cardinality error component— Accounts for the number of unassigned tracks and truths.

  • Labelling error component — Accounts for the error of incorrect assignments.

• OSPA⁽²⁾ — Specify the Metric parameter as OSPA(2). The OSPA⁽²⁾ metric evaluates cumulative tracking performance for a duration of time.

You can output each component individually from the block. For more details on the algorithm, see Algorithm and References.

Examples

Extended Object Tracking of Highway Vehicles with Radar and Camera in Simulink

Track highway vehicles around an ego vehicle in Simulink. In this example, you use multiple extended object tracking techniques to track highway vehicles and evaluate their tracking performance. This example closely follows the Extended Object Tracking of Highway Vehicles with Radar and Camera MATLAB® example.

• Since R2021b
• Open Model

Ports

Input

Tracks — Track list

Simulink^® bus containing MATLAB^® structure

Track list, specified as a Simulink bus containing a MATLAB structure.

If you specify the Track bus parameter on the Port Setting tab to objectTrack, the structure must use this form:

Field	Description
NumTracks	Number of tracks
Tracks	Array of track structures

Each track structure must contain TrackID and State fields. Additionally, if you specify an NEES-based distance (posnees or velnees) in the Distance type parameter, each structure must contain a StateCovariance field.

Field	Definition
TrackID	Unique track identifier used to distinguish multiple tracks, specified as a nonnegative integer.
State	Value of state vector at the update time, specified as an N-element vector, where N is the dimension of the state.
StateCovariance	Uncertainty covariance matrix, specified as an N-by-N matrix, where N is the dimension of the state.

If you specify an NEES-based distance (posnees or velnees) in the Distance type parameter, then the structure must contain a StateCovariance field.

If you specify the Track bus parameter to custom, then you can use your own track bus format. In this case, you must define a track extractor function using the Track extractor function parameter. The function must use this syntax:

tracks = trackExtractorFcn(trackInputFromBus)

where trackInputFromBus is the input from the track bus and tracks must return as an array of structures with TrackID and State fields.

Truths — Truth list

Simulink bus containing MATLAB structure

Truth list, specified as a Simulink bus containing a MATLAB structure.

If you specify the Truth bus parameter on the Port Setting tab to Platform, the structure must use this form:

Field	Description
NumPlatforms	Number of truth platforms
Platforms	Array of truth platform structures

Each platform structure has these fields:

Field	Definition
PlatformID	Unique identifier used to distinguish platforms, specified as a nonnegative integer.
Position	Position of the platform, specified as an M-element vector, where M is the dimension of the position state. For example, M = 3 for 3-D position.
Velocity	Velocity of the platform, specified as an M-element vector, where M is the dimension of the velocity state. For example, M = 3 for 3-D velocity.

If you specify the Truth bus parameter as Actor, the structure must use this form:

Field	Description
NumActors	Number of truth actors
Actors	Array of truth actor structures

Each actor structure has these fields:

Field	Definition
ActorID	Unique identifier used to distinguish actors, specified as a nonnegative integer.
Position	Position of the actor, specified as an M-element vector, where M is the dimension of the position state. For example, M = 3 for 3-D position.
Velocity	Velocity of the actor, specified as an M-element vector, where M is the dimension of the velocity state. For example, M = 3 for 3-D velocity.

If you specify the Truth bus parameter to custom, then you can define your own truth bus format. In this case, you must define a truth extractor function using the Truth extractor function parameter. The function must use this syntax:

 truths = truthExtractorFcn(truthInputFromBus)

where truthInputFromBus is the input from the truth bus and truths must return as an array of structures with PlatformID, Position, and Velocity fields.

Assignments — Known assignment

K-by-2 matrix of nonnegative integers

Known assignment, specified as aK-by-2 matrix of nonnegative integers. K is the number of assignment pairs. The first column elements are track IDs, and the second column elements are truth IDs. The two IDs in a row are assigned to each other. If a track or truth is not assigned, specify 0 as the other element in the row.

Assignments between tracks and truths must be unique. Redundant or false tracks should be treated as unassigned tracks by assigning them to the "0" TruthID.

Dependencies

To enable this port, on the Port Setting tab, select Assignments.

Output

OSPA Metric — OSPA metric

nonnegative real scalar

OSPA metric, returned as a nonnegative real scalar. Depending on the values of the Metric and Labeling error parameters, the block can output traditional OSPA, labeled OSPA (LOSPA), or OSPA⁽²⁾.

Metric Parameter Value	Labeling error Parameter Value	Metric
OSPA	0	OSPA
OSPA	Positive scalar	LOSPA
OSPA(2)	Not applicable	OSPA(2)

Example: 10.1

Localization Error — Localization error component

nonnegative real scalar

Localization error component, returned as a nonnegative real scalar.

Example: 8.5

Dependencies

To enable this port, on the Port Setting tab, select Localization error.

Cardinality Error — Cardinality error component

nonnegative real scalar

Cardinality error component, returned as a nonnegative real scalar.

Example: 6

Dependencies

To enable this port, on the Port Setting tab, select Cardinality error.

Labeling Error — Labeling error component

nonnegative real scalar

Labeling error component, returned as a nonnegative real scalar.

Example: 7.5

Dependencies

To enable this port, on the Port Setting tab, select Labeling error.

Parameters

Properties

Metric — Metric option

OPSA (default) | OSPA(2)

Metric option, specified as OSPA or OSPA(2).

• OSPA — Computes the traditional OSPA metric by default, or computes the labeled OSPA metric after additionally specifying the Labeling error parameter as a positive value.

• OSPA(2) — Computes the OSPA⁽²⁾ metric, which evaluates cumulative tracking performance. Selecting this option enables these parameters for configuring the metric:

  • Window length

  • Window sum order (q)

  • Window weights

  • Window weight exponent (r)

  • Custom window weights

  Selecting this option also disables two parameters used to evaluate the labeling error component:

  • Assignments

  • Labeling error

Cutoff distance — Threshold for cutoff distance between track and truth

30 (default) | real positive scalar

Threshold for the cutoff distance between track and truth, specified as a real positive scalar. If the computed distance between a track and the assigned truth is higher than the threshold, the actual distance incorporated in the metric is reduced to the threshold.

Example: 40

Order — Order of OSPA metric

2 (default) | positive integer

Order of the OSPA metric, specified as a positive integer.

Example: 3

Distance type — Distance type

posnees (default) | velnees | posabserr | velabserr | custom

Distance type, specified as posnees, velnees, posabserr, or velabserr. The distance type specifies the physical quantity used for distance calculations:

• posnees – Normalized estimation error squared (NEES) of track position

• velnees – NEES error of track velocity

• posabserr – Absolute error of track position

• velabserr – Absolute error of track velocity

• custom – Custom distance error

If you select custom, you must also specify a distance function in the Custom distance function parameter.

Custom distance function — Custom distance function

function handle

Custom distance function, specified as a function handle. The function must support the following syntax:

d = myCustomFcn(Track,Truth)

where Track is a structure of track information, Truth is a structure of truth information, and d is the distance between the truth and the track. See objectTrack for an example on how to organize track information.

Example: @myCustomFcn

Dependencies

To enable this property, set the Distance type parameter to custom.

Motion model — Desired platform motion model

constvel (default) | constacc | constturn | singer

Desired platform motion model, specified as constvel, constacc, constturn, or singer. This property selects the motion model used by the Tracks input port.

The motion models expect the State field of the track structure to have a column vector containing these values:

• constvel — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

• constacc — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

• constturn — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

• singer — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The StateCovariance field of the track structure input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the State field of the track structure.

Labeling error — Penalty for incorrect assignment

0 (default) | nonnegative scalar

Penalty for incorrect assignment of track to truth, specified as a nonnegative scalar. The function decides if an assignment is correct based on the provided known assignment input from the Assignments input port. If the assignment is not provided as an input, the last known assignment is assumed to be correct.

Example: 5

Dependencies

To enable this parameter, set the Metric parameter to OSPA.

Window length — Sliding window length for OSPA⁽²⁾ metric

100 (default) | positive integer

Sliding window length for the OSPA⁽²⁾ metric, specified as a positive integer. The window length defines the number of time steps from a previous time to the current time used to evaluate the metric. For more details, see OSPA(2) Metric.

Dependencies

To enable this parameter, set the Metric parameter to OSPA(2).

Data Types: single | double

Window sum order (q) — Order of weighted sum for track and truth history

2 (default) | positive scalar

Order of weighted sum for the track and truth history, specified as a positive scalar. For more details, see OSPA(2) Metric.

Dependencies

To enable this parameter, set the Metric parameter to OSPA(2).

Data Types: single | double

Window weights — Options for window weights

auto (default) | custom

Options for window weights, specified as auto or custom.

• auto — Automatically generates the window weights using the algorithm given in OSPA(2) Metric.

• custom — Customizes the window weights using the Custom window weights parameter.

Dependencies

To enable this parameter, set the Metric parameter to OSPA(2).

Data Types: single | double

Window weight exponent (r) — Exponent for automatic weight calculation

1 (default) | nonnegative scalar

Exponent for automatic weight calculation, specified as a nonnegative scalar. An exponent value, r, of 0 represents equal weights in the window. A higher value of r assigns more weights on recent data. For more details, see OSPA(2) Metric.

Dependencies

To enable this parameter, set the Window weights parameter to auto.

Data Types: single | double

Custom window weights — Custom weights in time window

N-element of vector of nonnegative values

Custom weights in the time window, specified as an N-element of vector of nonnegative values, when N is the window length, specified in the Window length parameter.

Dependencies

To enable this parameter, set the Window weights parameter to custom.

Data Types: single | double

Simulate using — Type of simulation to run

Interpreted execution (default) | Code Generation

Select a simulation type from these options:

• Interpreted execution — Simulate the model using the MATLAB interpreter. This option shortens startup time. In Interpreted execution mode, you can debug the source code of the block.

• Code generation — Simulate the model using generated C code. The first time you run a simulation, Simulink generates C code for the block. The C code is reused for subsequent simulations as long as the model does not change. This option requires additional startup time.

Port Setting

Assignments — Enable assignment input

off (default) | on

Select this parameter to enable the input of known assignments through the Assignments input port.

Dependencies

To enable this parameter, set the Metric parameter to OSPA.

Localization error — Enable localization error component output

off (default) | on

Select this parameter to enable the output of the localization error component through the Localization Error output port.

Cardinality error — Enable cardinality error component output

off (default) | on

Select this parameter to enable the output of the cardinality error component through the Cardinality Error output port.

Labeling error — Enable labeling error component output

off (default) | on

Select this parameter to enable the output of the labeling error component through the Labeling Error output port.

Dependencies

To enable this parameter, set the Metric parameter to OSPA.

Track bus — Track bus selection

objectTrack (default) | custom

Track bus selection, specified as objectTrack or custom. See the description of the Tracks input port for more details about each selection.

Truth bus — Truth bus selection

Platform (default) | Actor | custom

Truth bus selection, specified as Platform, Actor, or custom. See the description of the Truths input port for more details about each selection.

Track extractor function — Track extractor function

function handle

Track extractor function, specified as a function handle. The function must support this syntax:

tracks = trackExtractorFcn(trackInputFromBus)

where trackInputFromBus is the input from the track bus and tracks must return as an array of structures with TrackID and State fields. If you specify an NEES-based distance (posnees or velnees) in the Distance type parameter, then the structure must contain a StateCovariance field.

Example: @myCustomFcn

Dependencies

To enable this property, set the Track bus parameter to custom.

Truth extractor function — Truth extractor function

function handle

Truth extractor function, specified as a function handle. The function must support this syntax:

 truths = truthExtractorFcn(truthInputFromBus)

where truthInputFromBus is the input from the track bus and truths must return as an array of structures with PlatformID, Position, and Velocity as field names.

Example: @myCustomFcn

Dependencies

To enable this property, set the Truth bus parameter to custom.

Algorithms

OSPA Metric

At time t_k, a list of truths is:

$$X=[x_1,x_2,\dots ,x_m]$$

At the same time, a tracker obtains a list of tracks:

$$Y=[y_1,y_2,\dots ,y_n]$$

The traditional OSPA metric is:

$$OSPA={(d_{loc}^p+d_{card}^p)}^{1/p}$$

Assuming m ≤ n, the two components, d_loc and d_card are calculated using these equations. The localization error component d_loc is computed as:

$$d_{loc}={\left\{\frac{1}{n}{\displaystyle \sum _{i=1}^m{d}_c^p\left(x_i,y_{\pi (i)}\right)}\right\}}^{1/p}$$

where p is the order of the OSPA metric, d_c is the cutoff-based distance, and y_π(i) represents the track assigned to truth x_i. The cutoff-based distance d_c is defined as:

$$d_c(x,y)=\min \left\{d_b(x,y),c\right\}$$

where c is the cutoff distance threshold, and d_b(x,y) is the distance between truth x and track y calculated by the distance function. The cutoff-based distance d_c takes the smaller value of d_b and c.

The cardinality error component d_card is:

$$d_{card}={\left\{\frac{n-m}{n}{c}^p\right\}}^{1/p}$$

The labeled OSPA (LOSPA) is:

$$OSPA={(d_{loc}^p+d_{card}^p+d_{lab}^p)}^{1/p}$$

Here, additionally, the labeling error component d_lab is:

$$d_{lab}={\left\{\frac{1}{n}{\displaystyle \sum _{i=1}^m{\alpha }^p\gamma \left(L(x_i),L(y_{\pi (i)})\right)}\right\}}^{1/p}$$

where α is the penalty for incorrect assignment in the labeling error component, L(x_i) represents the truth ID of x_i, and L(y_π(i)) represents the track ID of y_π(i). The function γ = 0 if the IDs of the truth and track pair agree with the known assignment given by the assignment input, or agree with the assignment in the last update if the known assignment is not given. Otherwise, γ = 1.

If m > n, exchange m and n in the formulation to obtain the OSPA metric.

OSPA⁽²⁾ Metric

Consider a time period of N time steps, from time t_k-N+1 to time t_k. During this time period, you have a list of m truth histories:

$$X=[x_1,x_2,\dots ,x_m]$$

Each truth history x_i, is composed of :

$$x_i=[x_{i(k-N+1}),\dots ,x_{i(k)}]$$

where x_i(s) is the track history for x_i at time step t_s, and x_i(s)= ∅ if x_i does not exist at time t_s. For the same time period, you have a list of n track histories:

$$Y=[y_1,y_2,\dots ,y_n]$$

Each track history y_i is composed of :

$$y_i=[y_{i(k-N+1}),\dots ,y_{i(k)}]$$

where y_i(s) is the track history at time step t_s, and y_i(s)= ∅ if y_i does not exist at time t_s.

Assuming m ≤ n, the OPSA⁽²⁾ metric is calculated as:

$$OSP{A}^{(2)}={\left[d_{loc}^p+d_{card}^p\right]}^{1/p}$$

where the cardinality error component d_card is:

$$d_{card}={\left\{\frac{n-m}{n}{c}^p\right\}}^{1/p}$$

In this equation, p is the order of the OSPA metric, and c is the cutoff distance threshold.

The localization error component d_loc is computed as:

$$d_{loc=}{\left\{\frac{1}{n}{\displaystyle \sum _{i=1}^m{d}_q(x_i,y_{\pi (i)})}\right\}}^{1/p}$$

where y_π(i) represents the track assigned to truth x_i, and d_q is the base distance between a truth and a track, accounting for cumulative tracking errors.

You can obtain d_q between a truth x_i and a track y_j as:

$$d_q(x_i,y_j)={\left[{\displaystyle \sum _{\tau =k-N+1}^{k}w(\tau )d^{*}{\left(x_i(\tau ),y_j(\tau )\right)}^q}\right]}^{1/q}$$

where N is the window length, w(τ) is the window weight at time step τ, and q is the window sum order. d* is defined as:

$$d^{*}\left(x_i(\tau ),y_j(\tau )\right)=\{\begin{array}{l}{d}_c\left(x_i(\tau ),y_j(\tau )\right)=\min \left\{d_b\left(x_i(\tau ),y_j(\tau )\right),c\right\},if{x}_i(\tau )\ne \varnothing and{y}_i(\tau )\ne \varnothing \\ c,if{x}_i(\tau )=\varnothing and{y}_i(\tau )\ne \varnothing ,or{x}_i(\tau )\ne \varnothing and{y}_i(\tau )=\varnothing \\ 0,if{x}_i(\tau )=y_i(\tau )=\varnothing \end{array}$$

From the equation, the cutoff-based distance d_c takes the smaller value of d_b and c, where d_b(x_i(τ),y_j(τ) ) is the distance between truth x_i and track y_j at time τ, calculated by the distance function.

If you do not customize the window weights, the object assigns the window weights as:

$$w(\tau )=\frac{{(N-k+\tau )}^r}{{\displaystyle \sum _{\tau =k-N+1}^k{(N-k+\tau )}^r}}$$

where r is the window weight component.

If m > n, exchange m and n in the formulation to obtain the OSPA⁽²⁾ metric.

References

[1] Schuhmacher, B., B. -T. Vo, and B. -N. Vo. "A Consistent Metric for Performance Evaluation of Multi-Object Filters." IEEE Transactions on Signal Processing, Vol, 56, No, 8, pp. 3447–3457, 2008.

[2] Ristic, B., B. -N. Vo, D. Clark, and B. -T. Vo. "A Metric for Performance Evaluation of Multi-Target Tracking Algorithms." IEEE Transactions on Signal Processing, Vol, 59, No, 7, pp. 3452–3457, 2011.

[3] M. Beard, B. -T. Vo, and B. -N. Vo. “OSPA (2) : Using the OSPA Metric to Evaluate Multi-Target Tracking Performance.” 2017 International Conference on Control, Automation and Information Sciences, IEEE, 2017, pp. 86–91.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using Simulink® Coder™.

Version History

Introduced in R2021a

R2023a: Simulink buses do not show in workspace

As of R2023a, the Simulink buses created by this block no longer show in MATLAB workspace.