How can i change the precision using datenum?

Im using datenum on some datasets to try to see the intersection between the data (using intersect function), the problem is that im getting different times with the same time (the last few numbers are a little bit different). I already saw some people using a different aproach but i don't know how can i do it with my data. Im posting the code im using:

    horas = datestr(horas);
    datas = datestr(datas);
    timesteps = [num2str(datas) num2str(horas(:,12:17))];
    timesteps = datenum(timesteps,'dd-mmm-yyyy HH:MM');
    [c,ia,ib]=intersect(timesteps,time_interval);
    hm0(ib) = hm0;

It should work but since it's not getting all the times, it fails and tells me "In an assignment A(:) = B, the number of elements in A and B must be the same.". But if i can solve the early problem it will work since i already used this function before. Thank you

[Stephen23]

To see the real datenum values download James Tursa's FEX submission:

https://www.mathworks.com/matlabcentral/fileexchange/22239-num2strexact-exact-version-of-num2str

Note that you can change how the value is displayed by changing the format.

Some useful discussions on floating point numbers:

https://www.mathworks.com/matlabcentral/answers/57444-faq-why-is-0-3-0-2-0-1-not-equal-to-zero

https://www.mathworks.com/matlabcentral/answers/316889-why-two-equal-numbers-are-not-equal

http://www.mathworks.com/matlabcentral/answers/102419-how-do-i-determine-if-the-error-in-my-answer-is-the-result-of-round-off-error-or-a-bug

http://www.mathworks.com/matlabcentral/answers/140656-about-numerical-difficulty-of-equality-comparison-operation-between-two-double-real-numbers-in-matlab

https://www.mathworks.com/matlabcentral/answers/320473-we-found-a-possible-bug-in-our-matlab

https://www.mathworks.com/matlabcentral/answers/376108-is-this-a-bug-in-max-x

This is worth reading as well:

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

http://faculty.tarleton.edu/agapie/documents/cs_343_arch/papers/1991_Goldberg_FloatingPoint.pdf

Answer 1

[Stephen23]

The basic problem is that intersect tests for exact equivalency of floating point numbers (which is all that datenum's are). This should be avoided, for reasons that have been discussed thousands of times before (see the link in my comment). Here are some proposals for how to deal with this:

• Round the datenum to some number of decimal places. This sounds simple, but it can introduce artifacts into the data, and requires picking an appropriate number of digits to round to. A bit fiddly.
• Download and use my FEX submission dateround, which rounds to the nearest second, minute, hour, etc.
• Convert to datevec, pick the units/column you want to use (e.g. years to minutes), and then use intersect with the 'rows' option.

Personally I prefer the last solution: neat, intuitive, and does not involve comparing floating point numbers. Here is a complete working example, which demonstrates how differences in the seconds can be ignored:

>> C1 = {'2018-10-29 12:34:56','2018-10-29 12:00:00'};
>> C2 = {'2018-10-29 12:34:55','2018-10-29 12:01:00'};
>> V1 = datevec(C1(:))
V1 =
   2018     10     29     12     34     56
   2018     10     29     12      0      0
>> V2 = datevec(C2(:))
V2 =
   2018     10     29     12     34     55
   2018     10     29     12      1      0
>> intersect(V1(:,1:5),V2(:,1:5),'rows')
ans =
   2018     10     29     12     34

Note that this is equivalent to doing a floor operation on the seconds column: to round the seconds up/down to the nearest minute you could either use dateround or do the rounding yourself.

[André Fernandes]

Thank you very much Stephen. I used the last solution and it seems to be working :) I will try the other solutions aswell!

[Stephen23]

@André Fernandes: I hope that it helps. You might also like to consider using datetime objects, which can probably be rounded to particular units and compared.

Answer 2

[André Fernandes]

Yes, i used:

time_interval = datevec(time_interval);
timesteps = datevec(timesteps);
[c,ia,ib]=intersect(timesteps(:,1:5),time_interval(:,1:5),'rows','stable');
hm0(ib) = hm0; %went to get the variable i wanted.

Im having a problem cus since this is inside a loop and the timeseries change everystep, in an interation where he just get 18 intersections then my hm0(ib) = hm0 gives me an error again "In an assignment A(:) = B, the number of elements in A and B must be the same."

[Stephen23]

@André Fernandes: there seems to be a mistake in your logic: you did not explain what hm0 is, or what size it has, but clearly this indexing and allocation

hm0(ib) = hm0

does not work. You should check the index ib and confirm that it has as many unique indices as hm0 has elements.

[André Fernandes]

Ahh i saw the problem, im getting different times of sampling... It changes from getting data at 00:24:00 to 00:25:00. Is there any solution for it instead of using :

formatSpec = '%{yyyy-MM-dd}D%{HH:mm:ss}D%f%f%f%f%f%f%f%f%f%f%f%f%f%[^\n\r]';
time_init=datenum('2017-01-01 00:24','yyyy-mm-dd HH:MM');
time_end=datenum('2017-12-31 00:00','yyyy-mm-dd HH:MM'); % hipotético
dt=1/48;
time_interval=time_init:dt:time_end;

I used this to create the time_interval since i thought my data was being measured at the same times every step of the loop, but it's not. Can you help me out? Im a little new on reasearch since i just started my internship of the master degree.

[André Fernandes]

for i=1:20%length(runs(:,1)) data = [archive filesep runs(i,:)]; fid = fopen(data,'r'); fileID = fopen(data,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'TextType', 'string', 'ReturnOnError', false); fclose(fileID); Se{i} = table(dataArray{1:end-1}, 'VariableNames', {'Data','Horas','hm0','t02', 'tp','thtp','sprtp','hs','tz','hmax','tmax','thmax','temp','perc','nt'});

    %Variáveis temporárias
    datas = (Se{i}.Data);
    horas = (Se{i}.Horas);
    hm0 = Se{i}.hm0;
    t02 = Se{i}.t02;
    tp = Se{i}.tp;
    thtp = Se{i}.thtp;
    sprtp = Se{i}.sprtp;
    hs = Se{i}.hs;
    tz = Se{i}.tz;
    hmax = Se{i}.hmax;

    horas = datestr(horas);
    datas = datestr(datas);
    timesteps = [num2str(datas) num2str(horas(:,12:17))];
    timesteps = datenum(timesteps,'dd-mmm-yyyy HH:MM');
    timesteps = datevec(timesteps);
    %     if timesteps == time_interval;
      [c,ia,ib]=intersect(timesteps(:,1:5),time_interval(:,1:5),'rows','stable');
       hm0(ib) = hm0;
%     else fillmissing()
%     end
    clearvars -except runs archive formatSpec delimiter hm0 t02 tp datas horas timesteps time_interval c ib ia Se
end

Here is the entire code so maybe it's easy for you to understand

[Stephen23]

"I used this to create the time_interval since i thought my data was being measured at the same times every step of the loop, but it's not."

Possibly you should reconsider your approach: if the data are not regular or don't behave as you expect, then you might have to broaden your matching criteria or use a different approach, e.g.:

• Round to the nearest five minutes, or ten minutes, or whatever unit/duration you want (multiply datenum of the time by datenum of that duration, round, divide again by that duration).
• Use datetime objects and tables to group the data within the required period.
• Interpolate the data at regularly spaced sample times.
• etc.

Which to choose depends on your data and how you want to process it.

[André Fernandes]

Alright, going to try it out after lunch. Thank you very much Stephen, ur helping alot!

[Peter Perkins]

This, from Stephen,

"Use datetime objects and tables to group the data within the required period."

is the correct advice.