stftLayer

R2026b

Short-time Fourier transform layer

Description

An STFT layer computes the short-time Fourier transform of the input. Use of this layer requires Deep Learning Toolbox™.

Creation

Syntax

layer = stftLayer

layer = stftLayer(PropertyName=Value)

Description

layer = stftLayer creates a Short-Time Fourier Transform (STFT) layer. The input to stftLayer must be a real-valued dlarray (Deep Learning Toolbox) object in "CBT" format with a size along the time dimension greater than the length of Window. stftLayer formats the output as "SCBT". For more information, see Layer Output Format.

Note

When you initialize the learnable parameters of stftLayer, the layer weights are set to the analysis window used to compute the transform. It is not recommended to initialize the weights directly.

layer = stftLayer(PropertyName=Value) sets properties using one or more name-value arguments. You can specify the analysis window and the number of overlapped samples, among others.

Note

You cannot use this syntax to set the Weights property.

Example: stfl = stftLayer(Window=triang(64),OverlapLength=48,FFTLength=512) creates an STFT layer with a 64-sample triangular window, 48 samples of overlap between adjoining windows, and 512 DFT points.

Properties

STFT

Window — Analysis window

hann(128,'periodic') (default) | vector

This property is read-only after object creation.

Analysis window used to compute the STFT, specified as a vector with two or more elements.

Example: (1-cos(2*pi*(0:127)'/127))/2 and hann(128) both specify a Hann window of length 128.

Data Types: double | single

OverlapLength — Number of overlapped samples

96 (default) | positive integer

This property is read-only after object creation.

Number of overlapped samples, specified as a positive integer strictly smaller than the length of Window.

The stride between consecutive windows is the difference between the window length and the number of overlapped samples.

Data Types: double | single

FFTLength — Number of DFT points

128 (default) | positive integer

This property is read-only after object creation.

Number of frequency points used to compute the discrete Fourier transform, specified as a positive integer greater than or equal to the window length. If you do not specify this property, stftLayer defaults it to the length of the window.

Data Types: double | single

TransformMode — Layer transform mode

"mag" (default) | "squaremag" | "logmag" | "logsquaremag" | "realimag"

Layer transform mode, specified as one of these:

• "mag" — STFT magnitude

• "squaremag" — STFT squared magnitude

• "logmag" — Natural logarithm of the STFT magnitude

• "logsquaremag" — Natural logarithm of the STFT squared magnitude

• "realimag" — Real and imaginary parts of the STFT, concatenated along the channel dimension

Data Types: char | string

Weight Tying

OutputLearnables — Learnable parameters to output to other layers

{} (default) | "Weights"

Since R2026b

This property is read-only after object creation.

Learnable parameters to output to other layers, specified as one of these values:

• {} (empty) — Do not output weights to another layer.

  If you set this property to empty, then the layer has one output with the name "out", which corresponds to the output data.

• "Weights" — Output weights to another layer.

  If you set this property to "Weights", then the layer has two outputs with the names "out" and "Weights", which correspond to the output data and weights, respectively. In this case, the layer also outputs the weights that it uses in the layer operation.

For more information, see Neural Network Weight Tying (Deep Learning Toolbox).

Parameters and Initialization

Weights — Layer weights

[] (default) | numeric array | dlarray object

Layer weights, specified as [], a numeric array, or a dlarray object.

The layer weights are learnable parameters. You can use initialize (Deep Learning Toolbox) to initialize the learnable parameters of a dlnetwork (Deep Learning Toolbox) that includes stftLayer objects. When you initialize the layers, initialize sets Weights to the analysis window used to compute the transform. For more information, see Initialize Short-Time Fourier Transform Layer. (since R2025a)

It is not recommended to initialize the weights directly.

Data Types: double | single

Learning Rate and Regularization

WeightLearnRateFactor — Multiplier for weight learning rate

0 (default) | nonnegative scalar

Multiplier for weight learning rate, specified as a nonnegative scalar. If not specified, this property defaults to zero, resulting in weights that do not update with training. You can also set this property using the setLearnRateFactor (Deep Learning Toolbox) function.

Data Types: double | single

WeightL2Factor — L₂ regularization factor for weights

0 (default) | nonnegative scalar

Since R2026b

L₂ regularization factor for the weights, specified as a nonnegative scalar.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the weights in this layer. For example, if WeightL2Factor is 2, then the L₂ regularization for the weights in this layer is twice the global L₂ regularization factor. You can specify the global L₂ regularization factor using the trainingOptions (Deep Learning Toolbox) function.

Data Types: double | single

Layer

Name — Layer name

'' (default) | character vector

Layer name, specified as a character vector. For Layer array input, the trainnet (Deep Learning Toolbox) and dlnetwork (Deep Learning Toolbox) functions automatically assign names to unnamed layers.

Data Types: char

NumInputs — Number of inputs

Read-only: 1 (default)

This property is read-only.

Number of inputs to the layer, represented as 1. This layer has a single input only.

Data Types: double

InputNames — Input name

Read-only: {'in'} (default)

This property is read-only.

Input name, represented as {'in'}. This layer has a single input only.

NumOutputs — Number of outputs

Read-only: 1 (default)

This property is read-only.

Number of outputs from the layer, represented as 1. This layer has a single output only.

Data Types: double

OutputNames — Output name

Read-only: {'out'} (default)

This property is read-only.

Output name, represented as {'out'}. This layer has a single output only.

Examples

Short-Time Fourier Transform of Chirp

Generate a signal sampled at 600 Hz for 2 seconds. The signal consists of a chirp with sinusoidally varying frequency content. Store the signal in a deep learning array with "CTB" format.

fs = 6e2; 
x = vco(sin(2*pi*(0:1/fs:2)),[0.1 0.4]*fs,fs);

dlx = dlarray(x,"CTB");

Create a short-time Fourier transform layer with default properties. Create a dlnetwork object consisting of a sequence input layer and the short-time Fourier transform layer. Specify a minimum sequence length of 128 samples. Run the signal through the predict method of the network.

ftl = stftLayer;

dlnet = dlnetwork([sequenceInputLayer(1,MinLength=128) ftl]);
netout = predict(dlnet,dlx);

Convert the network output to a numeric array. Use the squeeze function to remove the length-1 channel and batch dimensions. Plot the magnitude of the STFT. The first dimension of the array corresponds to frequency and the second to time.

q = extractdata(netout);

waterfall(squeeze(q)')
set(gca,XDir="reverse",View=[30 45])
xlabel("Frequency")
ylabel("Time")

Short-Time Fourier Transform of Sinusoid

Generate a 3 × 160 (× 1) array containing one batch of a three-channel, 160-sample sinusoidal signal. The normalized sinusoid frequencies are π/4 rad/sample, π/2 rad/sample, and 3π/4 rad/sample. Save the signal as a dlarray, specifying the dimensions in order. dlarray permutes the array dimensions to the "CBT" shape expected by a deep learning network.

nch = 3;
N = 160;
x = dlarray(cos(pi.*(1:nch)'/4*(0:N-1)),"CTB");

Create a short-time Fourier transform layer that can be used with the sinusoid. Specify a 64-sample rectangular window, 48 samples of overlap between adjoining windows, and 1024 DFT points. By default, the layer outputs the magnitude of the STFT.

stfl = stftLayer(Window=rectwin(64),OverlapLength=48,FFTLength=1024);

Create a two-layer dlnetwork object containing a sequence input layer and the STFT layer you just created. Treat each channel of the sinusoid as a feature. Specify the signal length as the minimum sequence length for the input layer.

layers = [sequenceInputLayer(nch,MinLength=N) stfl];
dlnet = dlnetwork(layers);

Run the sinusoid through the forward method of the network.

dataout = forward(dlnet,x);

Convert the network output to a numeric array. Use the squeeze function to collapse the size-1 batch dimension. Permute the channel and time dimensions so that each array page contains a two-dimensional spectrogram. Plot the STFT magnitude separately for each channel in a waterfall plot.

q = squeeze(extractdata(dataout));
q = permute(q,[1 3 2]);

tiledlayout("vertical")
for kj = 1:nch
    nexttile
    waterfall(q(:,:,kj)')
    view(30,45)
    zlabel("Ch. " + kj)
end

ISTFT of Zero-Padded Data in Deep Learning Network

Create a deep learning network that demonstrates perfect reconstruction of the short-time Fourier transform (STFT) of a deep learning array. To minimize edge effects, the network zero-pads the data before computing the STFT.

Generate a 3-by-2000-by-5 array containing five batches of a three-channel sinusoidal signal sampled at 1 kHz for two seconds. Save the array as a dlarray, specifying the dimensions in order. dlarray permutes the array dimensions to the "CBT" shape expected by a deep learning network. Display the array dimension sizes.

Fs = 1e3;
nchan = 3;
nbtch = 5;
nsamp = 2000;
t = (0:nsamp-1)/Fs;

x = zeros(nchan,nsamp,nbtch);
for k=1:nbtch
    x(:,:,k) = sin(k*pi.*(1:nchan)'*t)+cos(k*pi.*(1:nchan)'*t);
end

xd = dlarray(x,"CTB");

Design a periodic Hann window of length 100 and set the number of overlap samples to 75. Check the window and overlap length for COLA compliance.

nwin = 100;
win = hann(nwin,"periodic");
noverlap = 75;

tf = iscola(win,noverlap)

tf = logical
   1

Create a STFT layer and an ISTFT layer using the stftLayer and istftLayer objects, respectively.

• For both layers, use the Hann window and number of overlap samples. Set the FFT length to 128.

• To concatenate the real and imaginary parts of the STFT layer output along the channel dimension, set the layer transform mode to "realimag" in the stftLayer object.

• To apply weight tying, set the OutputLearnables and InputLearnables properties to "Weights" in the stftLayer and istftLayer object, respectively.

fftlen = 128;

ftl = stftLayer(Window=win,FFTLength=fftlen, ...
    OverlapLength=noverlap,TransformMode="realimag", ...
    Name="stft",OutputLearnables="Weights");

iftl = istftLayer(Window=win,FFTLength=fftlen, ...
    OverlapLength=noverlap, ...
    Name="istft",InputLearnables="Weights");

Create a deep learning network appropriate for the data that demonstrates perfect reconstruction of the STFT. Use a function layer to zero-pad the data on both sides along the time dimension before computing the STFT. The length of the zero padding is the window length. Use a function layer after the ISTFT layer to trim both sides of the ISTFT layer output by the same amount. To complete the weight tying, connect the STFT and ISTFT layers.

layers = [
    sequenceInputLayer(nchan,MinLength=nsamp)
    functionLayer(@(X) paddata(X,nsamp+2*nwin,dimension=3,side="both"))
    ftl
    iftl
    functionLayer(@(X) trimdata(X,nsamp,dimension=3,side="both"))];
dlnet = dlnetwork(layers,Initialize=false);
dlnet = connectLayers(dlnet,"stft/Weights","istft/Weights");
dlnet = initialize(dlnet);

Analyze the network using the data. The number of channels of the STFT layer output is twice the layer input.

analyzeNetwork(dlnet,xd)

Run the data through the forward method of the network.

dataout = forward(dlnet,xd);

The output is a dlarray object in "CBT" format. Convert the network output to a numeric array. Permute the dimensions so that each page is a batch.

xrec = extractdata(dataout);
xrec = permute(xrec,[1 3 2]);

Choose a batch. Plot the original and reconstructed multichannel signal of that batch as a stacked plot.

wb = 4;
tiledlayout(2,1)
nexttile
stackedplot(x(:,:,wb)',DisplayLabels="Channel "+string(1:nchan))
title("Batch "+num2str(wb)+": Original")
nexttile
stackedplot(xrec(:,:,wb)',DisplayLabels="Channel "+string(1:nchan))
title("Batch "+num2str(wb)+": Reconstruction")

Confirm perfect reconstruction of the data.

max(abs(x(:)-xrec(:)))

ans = single

5.6394e-07

Initialize Short-Time Fourier Transform Layer

Since R2025a

Verify that the weights of a short-time Fourier transform (STFT) layer are reset to the specified window when you reinitialize the containing network.

Define an array of seven layers: a sequence input layer, an STFT layer, a 2-D convolutional layer, a batch normalization layer, a rectified linear unit (ReLU) layer, a fully connected layer, and a softmax layer. There is one feature in the sequence input. Set the minimum signal length in the sequence input layer to 512 samples. For the STFT layer, use a 256-sample Hamming window and an overlap length of 128 samples.

win = hamming(256);
layers = [
   sequenceInputLayer(1,MinLength=512)
   stftLayer(Window=win,OverlapLength=128,Name="stft")
   convolution2dLayer(4,16,Padding="same")
   batchNormalizationLayer
   reluLayer
   fullyConnectedLayer(3)
   softmaxLayer];

Create a deep learning neural network from the layer array. By default, the dlnetwork function initializes the network at creation. For reproducibility, use the default random number generator.

rng("default")
net = dlnetwork(layers);

Display the table of learnable parameters. The network weights and bias are nonempty dlarray objects.

tInit1 = net.Learnables

tInit1 = 7×3 table
       Layer       Parameter           Value        
    ___________    _________    ____________________

    "stft"         "Weights"    {256×1      dlarray}
    "conv"         "Weights"    {  4×4×1×16 dlarray}
    "conv"         "Bias"       {  1×1×16   dlarray}
    "batchnorm"    "Offset"     {  1×16     dlarray}
    "batchnorm"    "Scale"      {  1×16     dlarray}
    "fc"           "Weights"    {  3×2064   dlarray}
    "fc"           "Bias"       {  3×1      dlarray}

Compare the initialized weights of the STFT layer from the list of learnable parameters with the Window property of the STFT layer. The stftLayer weights are single precision and initialized to the specified window.

isequal(tInit1.Value{1},single(net.Layers(2).Window))

ans = logical
   1

Set the learnable parameters to empty arrays. Reinitialize the network. Display the network and the learnable parameters. The network weights and bias are nonempty dlarray objects.

net = dlupdate(@(x)[],net);
net = initialize(net);
tInit2 = net.Learnables

tInit2 = 7×3 table
       Layer       Parameter           Value        
    ___________    _________    ____________________

    "stft"         "Weights"    {256×1      dlarray}
    "conv"         "Weights"    {  4×4×1×16 dlarray}
    "conv"         "Bias"       {  1×1×16   dlarray}
    "batchnorm"    "Offset"     {  1×16     dlarray}
    "batchnorm"    "Scale"      {  1×16     dlarray}
    "fc"           "Weights"    {  3×2064   dlarray}
    "fc"           "Bias"       {  3×1      dlarray}

Compare the weights from the STFT and 2-D convolutional layers along the two initialization calls. The STFT layer sets the weights using the specified window, while the convolutional layer weights consists of a new set of random values.

tiledlayout flow
nexttile
plot(tInit1.Value{1})
hold on
plot(tInit2.Value{1},"--")
hold off
title("STFT Weights (Window)")
legend(["First" "Second"] + " Initialization")
nexttile
plot([tInit1.Value{2}(:) tInit2.Value{2}(:)])
title("2-D Convolutional Weights")
legend(["First" "Second"] + " Initialization")

More About

Short-Time Fourier Transform

The short-time Fourier transform (STFT) is used to analyze how the frequency content of a nonstationary signal changes over time. The magnitude squared of the STFT is known as the spectrogram time-frequency representation of the signal. For more information about the spectrogram and how to compute it using Signal Processing Toolbox™ functions, see Spectrogram Computation with Signal Processing Toolbox.

The STFT of a signal is computed by sliding an analysis window g(n) of length M over the signal and calculating the discrete Fourier transform (DFT) of each segment of windowed data. The window hops over the original signal at intervals of R samples, equivalent to L = M – R samples of overlap between adjoining segments. Most window functions taper off at the edges to avoid spectral ringing. The DFT of each windowed segment is added to a complex-valued matrix that contains the magnitude and phase for each point in time and frequency. The STFT matrix has

$$k=\lfloor \frac{N_x-L}{M-L}\rfloor$$

columns, where N_x is the length of the signal x(n) and the ⌊⌋ symbols denote the floor function. The number of rows in the matrix equals N_DFT, the number of DFT points, for centered and two-sided transforms and an odd number close to N_DFT/2 for one-sided transforms of real-valued signals.

The mth column of the STFT matrix $$X(f)=\left[\begin{array}{ccccc}{X}_1(f)& X_2(f)& X_3(f)& \cdots & X_k(f)\end{array}\right]$$ contains the DFT of the windowed data centered about time mR:

$$X_m(f)={\displaystyle \sum _{n=-\infty }^{\infty }x(n)g(n-mR)e^{-j2\pi fn}}.$$

Layer Output Format

stftLayer formats the output as "SCBT", a sequence of 1-D images where the image height corresponds to frequency, the second dimension corresponds to channel, the third dimension corresponds to batch, and the fourth dimension corresponds to time.

• You can feed the output of stftLayer unchanged to a 1-D convolutional layer when you want to convolve along the frequency ("S") dimension. For more information, see convolution1dLayer (Deep Learning Toolbox).

• To feed the output of stftLayer to a 1-D convolutional layer when you want to convolve along the time ("T") dimension, you must place a flatten layer after the stftLayer. For more information, see flattenLayer (Deep Learning Toolbox).

• You can feed the output of stftLayer unchanged to a 2-D convolutional layer when you want to convolve along the frequency ("S") and time ("T") dimensions. For more information, see convolution2dLayer (Deep Learning Toolbox).

• To use stftLayer as part of a recurrent neural network, you must place a flatten layer after the stftLayer. For more information, see lstmLayer (Deep Learning Toolbox) and gruLayer (Deep Learning Toolbox).

• To use the output of stftLayer with a fully connected layer as part of a classification workflow, you must reduce the time ("T") dimension of the output so that it has size 1. To reduce the time dimension of the output, place a global pooling layer before the fully connected layer. For more information, see globalAveragePooling2dLayer (Deep Learning Toolbox) and fullyConnectedLayer (Deep Learning Toolbox).

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™. (since R2025a)

Usage notes and limitations:

• You can generate generic C/C++ code that does not depend on third-party libraries and deploy the generated code to hardware platforms.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™. (since R2025a)

Usage notes and limitations:

• You can generate CUDA code that is independent of deep learning libraries and deploy the generated code to platforms that use NVIDIA^® GPU processors.

Version History

Introduced in R2021b

R2026b: Apply tying or set L₂ regularization factor to layer weights

Starting in R2026b, you can use the new OutputLearnables property to apply weight tying in stftLayer objects. To learn more about weight tying, see Neural Network Weight Tying (Deep Learning Toolbox).

Alternatively, you can use the new WeightL2Factor property to set an L₂ regularization factor to the weights.

R2025a: Initialize layer weights through dlnetwork initialization

Starting in R2025a, you can use initialize (Deep Learning Toolbox) to initialize learnable parameters for deep learning neural networks that include stftLayer objects.

R2025a: New Weights default value

Starting in R2025a, the default value of the Weights property is []. Prior to R2025a, stftLayer set the default value to the analysis window used to compute the transform.

R2025a: C/C++ and GPU Code Generation

The stftLayer object supports:

• C/C++ code generation. You must have MATLAB^® Coder™ to generate C/C++ code.

• Code generation for NVIDIA GPUs. You must have GPU Coder™ to generate GPU code.

R2023b: Weights initialized to analysis window

Starting in R2023b, stftLayer initializes the Weights learnable parameter to the analysis window used to compute the transform. Previously, the parameter was initialized to an array containing the Gabor atoms for the STFT.

R2022b: OutputMode property to be removed in a future release

The OutputMode property of stftLayer will be removed in a future release. Update your code and networks to make them compatible with stftLayer output in "SCBT" format. For more information, see Layer Output Format.