Main Content

huber

Huber loss for regression tasks

    Description

    The Huber operation computes the Huber loss between network predictions and target values for regression tasks. When the 'TransitionPoint' option is 1, this is also known as smooth L1 loss.

    The huber function calculates the Huber loss using dlarray data. Using dlarray objects makes working with high dimensional data easier by allowing you to label the dimensions. For example, you can label which dimensions correspond to spatial, time, channel, and batch dimensions using the 'S', 'T', 'C', and 'B' labels, respectively. For unspecified and other dimensions, use the 'U' label. For dlarray object functions that operate over particular dimensions, you can specify the dimension labels by formatting the dlarray object directly, or by using the 'DataFormat' option.

    example

    loss = huber(dlY,targets) returns the Huber loss between the formatted dlarray object dlY containing the predictions and the target values targets for regression tasks. The input dlY is a formatted dlarray. The output loss is an unformatted dlarray scalar.

    For unformatted input data, use the 'DataFormat' option.

    loss = huber(dlY,targets,weights) applies weights to the calculated loss vales. Use this syntax to weight the contributions of classes, observations, or regions of the input to the calculated loss values.

    loss = huber(___,'DataFormat',FMT) also specifies the dimension format FMT when dlY is not a formatted dlarray.

    loss = huber(___,Name,Value) specifies options using one or more name-value pair arguments in addition to the input arguments in previous syntaxes. For example, 'NormalizationFactor','all-elements' specifies to normalize the loss by dividing the reduced loss by the number of input elements.

    Examples

    collapse all

    Create an array of predictions for 12 observations over 10 responses.

    numResponses = 10;
    numObservations = 12;
    
    Y = rand(numResponses,numObservations);
    dlY = dlarray(Y,'CB');

    View the size and format of the predictions.

    size(dlY)
    ans = 1×2
    
        10    12
    
    
    dims(dlY)
    ans = 
    'CB'
    

    Create an array of random targets.

    targets = rand(numResponses,numObservations);

    View the size of the targets.

    size(targets)
    ans = 1×2
    
        10    12
    
    

    Compute the Huber loss between the predictions and the targets.

    loss = huber(dlY,targets)
    loss = 
      1x1 dlarray
    
        0.7374
    
    

    Create arrays of predictions and targets for 12 sequences of varying lengths over 10 responses.

    numResponses = 10;
    numObservations = 12;
    maxSequenceLength = 15;
    
    sequenceLengths = randi(maxSequenceLength,[1 numObservations]);
    
    Y = cell(numObservations,1);
    targets = cell(numObservations,1);
    
    for i = 1:numObservations
        Y{i} = rand(numResponses,sequenceLengths(i));
        targets{i} = rand(numResponses,sequenceLengths(i));
    end

    View the cell arrays of predictions and targets

    Y
    Y=12×1 cell array
        {10×13 double}
        {10×14 double}
        {10×2  double}
        {10×14 double}
        {10×10 double}
        {10×2  double}
        {10×5  double}
        {10×9  double}
        {10×15 double}
        {10×15 double}
        {10×3  double}
        {10×15 double}
    
    
    targets
    targets=12×1 cell array
        {10×13 double}
        {10×14 double}
        {10×2  double}
        {10×14 double}
        {10×10 double}
        {10×2  double}
        {10×5  double}
        {10×9  double}
        {10×15 double}
        {10×15 double}
        {10×3  double}
        {10×15 double}
    
    

    Pad the prediction and target sequences in the second dimension using the padsequences function and also return the corresponding mask.

    [Y,mask] = padsequences(Y,2);
    targets = padsequences(targets,2);

    Convert the padded sequences to dlarray with format 'CTB' (channel, time, batch). Because formatted dlarray objects automatically sort the dimensions, keep the dimensions of the targets and mask consistent by also converting them to a formatted dlarray objects with the same formats.

    dlY = dlarray(Y,'CTB');
    targets = dlarray(targets,'CTB');
    mask = dlarray(mask,'CTB');

    View the sizes of the prediction scores, targets, and the mask.

    size(dlY)
    ans = 1×3
    
        10    12    15
    
    
    size(targets)
    ans = 1×3
    
        10    12    15
    
    
    size(mask)
    ans = 1×3
    
        10    12    15
    
    

    Compute the Huber loss between the predictions and the targets. To prevent the loss values calculated from padding from contributing to the loss, set the 'Mask' option to the mask returned by the padsequences function.

    loss = huber(dlY,targets,'Mask',mask)
    loss = 
      1×1 dlarray
    
        8.1834
    
    

    Input Arguments

    collapse all

    Predictions, specified as a formatted dlarray, an unformatted dlarray, or a numeric array. When dlY is not a formatted dlarray, you must specify the dimension format using the 'DataFormat' option.

    If dlY is a numeric array, targets must be a dlarray.

    Target responses, specified as a formatted or unformatted dlarray or a numeric array.

    Specify the targets as an array with the same size and format as dlY.

    If targets is a formatted dlarray, its dimension format must be the same as the format of dlY, or the same as 'DataFormat' if dlY is unformatted

    If targets is an unformatted dlarray or a numeric array, then the format of dlY or the value of 'DataFormat' is implicitly applied to targets.

    Tip

    Formatted dlarray objects automatically sorts their dimensions. To ensure that the dimensions of dlY and targets are consistent, when dlY is a formatted dlarray, also specify targets as a formatted dlarray.

    Weights, specified as a dlarray or a numeric array.

    To specify response weights, specify a vector with a 'C' (channel) dimension with size matching the 'C' (channel) dimension of the dlX. Specify the 'C' (channel) dimension of the response weights by using a formatted dlarray object or by using the 'WeightsFormat' option.

    To specify observation weights, specify a vector with a 'B' (batch) dimension with size matching the 'B' (batch) dimension of the dlY. Specify the 'B' (batch) dimension of the class weights by using a formatted dlarray object or by using the 'WeightsFormat' option.

    To specify weights for each element of the input independently, specify the weights as an array of the same size as dlY. In this case, if weights is not a formatted dlarray object, then the function uses the same format as dlY. Alternatively, specify the weights format using the 'WeightsFormat' option.

    Name-Value Pair Arguments

    Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

    Example: 'NormalizationFactor','all-elements' specifies to normalize the loss by dividing the reduced loss by the number of input elements

    Point where Huber loss transitions from a quadratic function to a linear function, specified as the comma-separated pair consisting of 'TransitionPoint' and a positive scalar.

    When 'TransitionPoint' is 1, this is also known as smooth L1 loss.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Mask indicating which elements to include for loss computation, specified as the comma-separated pair consisting of 'Mask' and a dlarray object, a logical array, or a numeric array with the same size as dlY.

    The function includes and excludes elements of the input data for loss computation when the corresponding value in the mask is 1 and 0, respectively.

    The default value is a logical array of ones with the same size as dlY.

    Tip

    Formatted dlarray objects automatically sorts their dimensions. To ensure that the dimensions of dlY and the mask are consistent, when dlY is a formatted dlarray, also specify the mask as a formatted dlarray.

    Mode for reducing array of loss values, specified as the comma-separated pair consisting of 'Reduction' and one of the following:

    • 'sum' – Sum all of the elements in the array of loss values. In this case, the output loss is scalar.

    • 'none' – Do not reduce the array of loss values. In this case, the output loss is an unformatted dlarray object with the same size as dlY.

    Divisor for normalizing reduced loss when 'Reduction' is 'sum', specified as the comma-separated pair consisting of 'NormalizationFactor' and one of the following:

    • 'batch-size' – Normalize the loss by dividing by the number of observations in dlX.

    • 'all-elements' – Normalize the loss by dividing by the number of elements of dlX.

    • 'mask-included' – Normalize the loss by dividing the loss values by the number of included elements specified by the mask for each observation independently. To use this option, you must specify a mask using the 'Mask' option.

    • 'none' – Do not normalize the loss.

    Dimension order of unformatted input data, specified as the comma-separated pair consisting of 'DataFormat' and a character vector or string scalar FMT that provides a label for each dimension of the data.

    When specifying the format of a dlarray object, each character provides a label for each dimension of the data and must be one of the following:

    • 'S' — Spatial

    • 'C' — Channel

    • 'B' — Batch (for example, samples and observations)

    • 'T' — Time (for example, time steps of sequences)

    • 'U' — Unspecified

    You can specify multiple dimensions labeled 'S' or 'U'. You can use the labels 'C', 'B', and 'T' at most once.

    You must specify 'DataFormat' when the input data is not a formatted dlarray.

    Example: 'DataFormat','SSCB'

    Data Types: char | string

    Dimension order of the class weights, specified as the comma-separated pair consisting of 'WeightsFormat' and a character vector or string scalar that provides a label for each dimension of the weights.

    When specifying the format of a dlarray object, each character provides a label for each dimension of the data and must be one of the following:

    • 'S' — Spatial

    • 'C' — Channel

    • 'B' — Batch (for example, samples and observations)

    • 'T' — Time (for example, time steps of sequences)

    • 'U' — Unspecified

    You can specify multiple dimensions labeled 'S' or 'U'. You can use the labels 'C', 'B', and 'T' at most once.

    You must specify 'WeightsFormat' when weights is a numeric vector and dlX has two or more nonsingleton dimensions.

    If weights is not a vector, or both weights and dlY are vectors, then default value of 'WeightsFormat' is the same as the format of dlY.

    Example: 'WeightsFormat','CB'

    Data Types: char | string

    Output Arguments

    collapse all

    Huber loss, returned as an unformatted dlarray. The output loss is an unformatted dlarray with the same underlying data type as the input dlY.

    The size of loss depends on the 'Reduction' option.

    Algorithms

    collapse all

    Huber Loss

    For each element Yj of the input, the huber function computes the corresponding Huber element-wise loss values using the formula

    lossj={12(YjTj)2if |YjTj|δδ|YjTj|12δ2otherwise,

    where Tj is the corresponding target value to Yj and δ is the transition point where the loss transitions from a quadratic function to a linear function.

    When the transition point is 1, this is also known as smooth L1 loss.

    To reduce the loss values to a single scalar, the function then reduces the element-wise loss values to a scalar loss using the formula

    loss=1Njmjwjlossj,

    where N is the normalization factor, mj is the mask value for element j, and wj is the weight value for element j.

    If not reducing the loss, the function applies the mask and the weights to the loss values directly:

    lossj*=mjwjlossj

    Extended Capabilities

    Introduced in R2021a