How can I interpolate across columns within a matrix where the dataset are different lengths and NaN values are at the end of the columns?

Hi everyone,

I have a data set where I am trying to normalise, interpolate and transpose the data. Each column of data is angle data from a separate trial. Each trial is a different length, leaving NaN values at the end. How can I interpolate these to be the same length, accounting for the NaN values?

I need each column to be done separately, as the data needs to be interpolated for each trial separately.I have tried using the interp1 function, but this just fills the matrix with more NaNs. I have tried using the inpaint_nans function, but I think this interpolates across each trial, meaning the end of the data jumps around! Here is where I have got to:

clc
clearvars

%% Read in data 
head_rot=readmatrix('Head_Rotation_Data');
sho_rot=readmatrix('Shoulder_Rotation_Data');

%% Unwrap so doesn't jump at 180 degrees
head_rot_unwrap=unwrap(head_rot*pi/180)*180/pi;
sho_rot_unwrap=unwrap(sho_rot*pi/180)*180/pi;
[m,n] = size(head_rot_unwrap);

%% Start each turn at 0 degrees
for normalise_loop=1:n %do separately for each time series (i.e. each participant or each trial)
    hr_norm(:,normalise_loop)= (head_rot_unwrap(:,normalise_loop) - head_rot_unwrap(1,normalise_loop));
    sr_norm(:,normalise_loop)= (sho_rot_unwrap(:,normalise_loop) - sho_rot_unwrap(1,normalise_loop));
end

%% Plot normalised data
figure(1),
subplot(211),plot(hr_norm)
title('Head Rotation Data');
subplot(212),plot(sr_norm)
title('Shoulder Rotation Data');

%% interpolate to longest dataset
hr_interp=inpaint_nans(hr_norm);
sr_interp=inpaint_nans(sr_norm);

%% Transpose
hr_transpose=transpose(hr_interp);
sr_transpose=transpose(sr_interp);

%% Plot interpolated data
figure(2),
subplot(211),plot((1:m),hr_interp)
title('Head Rotation Data');
subplot(212),plot((1:m),sr_interp)
title('Shoulder Rotation Data');

%% Plot transposed data
figure(3),
subplot(211),plot((1:m),hr_transpose)
title('Head Rotation Data');
subplot(212),plot((1:m),sr_transpose)
title('Shoulder Rotation Data');

Thanks for your help!

[KSSV]

Read about fillmissing

Answer 1

[Simon Chan]

Use extrapolate in function interp1, the folloiwing use linear as the extrapolation method, you may choose other method, such as spline.

clc
clearvars
%% Read in data 
head_rot=readmatrix('Head_Rotation_Data');
sho_rot=readmatrix('Shoulder_Rotation_Data');
%% Unwrap so doesn't jump at 180 degrees
head_rot_unwrap=unwrap(head_rot*pi/180)*180/pi;
sho_rot_unwrap=unwrap(sho_rot*pi/180)*180/pi;
[m,n] = size(head_rot_unwrap);
%% Start each turn at 0 degrees
for normalise_loop=1:n %do separately for each time series (i.e. each participant or each trial)
    hr_norm(:,normalise_loop)= (head_rot_unwrap(:,normalise_loop) - head_rot_unwrap(1,normalise_loop));
    sr_norm(:,normalise_loop)= (sho_rot_unwrap(:,normalise_loop) - sho_rot_unwrap(1,normalise_loop));
end
%% Plot normalised data
figure(1),
subplot(211),plot(hr_norm)
title('Head Rotation Data');
subplot(212),plot(sr_norm)
title('Shoulder Rotation Data');

%% interpolate to longest dataset
[Ny,Nx] = size(hr_norm);
Nz = sum(~isnan(hr_norm));                          % Determine number of non-NaN
for k = 1:Nx
    hr_interp(:,k) = interp1(1:Nz(k),hr_norm(1:Nz(k),k),1:Ny,'linear','extrap');    
end
[Ny,Nx] = size(sr_norm);
Nz = sum(~isnan(sr_norm));                          
for k = 1:Nx
    sr_interp(:,k) = interp1(1:Nz(k),sr_norm(1:Nz(k),k),1:Ny,'linear','extrap');
end
%% Transpose
hr_transpose=transpose(hr_interp);
sr_transpose=transpose(sr_interp);
%% Plot interpolated data
figure(2),
subplot(211),plot((1:m),hr_interp)
title('Head Rotation Data');
subplot(212),plot((1:m),sr_interp)
title('Shoulder Rotation Data');

%% Plot transposed data
figure(3),
subplot(211),plot((1:m),hr_transpose)
title('Head Rotation Data');
subplot(212),plot((1:m),sr_transpose)
title('Shoulder Rotation Data');

[pkll201]

You are incredible! Thank you so much for your help :)