intersect two datetime vectors

Hi,

I have 2 datetime vectors of different size, I'm trying to find the indices of identical timestamps in both vectors. Intersect and Ismember return empty or zero respectively, Datestr doesnt work, I assume something is wrong with the datetime vectors (attached)?

cheers,

Answer 1

[rakbar]

It looks like your datetime arrays have some NaT in them. This is what I did: (1) remove NaT, (2) convert to datestr, (3) convert to datenum (4) use intersect:

data_format = 'dd-mmm-yyyy HH:MM:SS';

%% find and remove NaT
idx_U = isnat(timeU);
idx_C = isnat(timeC);

timeU(idx_U) = [];
timeC(idx_C) = [];
%% Convert to datestr
date_str_U = datestr(timeU);
date_str_C = datestr(timeC);
%% convert to datenum
date_num_U = datenum(date_str_U,data_format);
date_num_C = datenum(date_str_C,data_format);

% C = A(ia) and C = B(ib).
[C,ia,ib] = intersect(date_num_U,date_num_C);

% to get the dates back:
datetime(date_num_U(ia),'ConvertFrom','datenum')

Answer 2

[Akira Agata]

The following can find identical timestamp with 0.1 sec tolerance.

% Load data
load('example1.mat');

% Find identical timestamp with 0.1 sec tolerance
dt = 0.1; %[sec]
tol = (dt/(60*60*24))/max(datenum([timeC;timeU]));
[idx,loc] = ismembertol(datenum(timeC),datenum(timeU),tol);

% Extract identical timestamp
timeC2 = timeC(idx);
timeU2 = timeU(loc(idx));

The result is:

% Check the result
timeC2.Format = 'yyyy/MM/dd HH:mm:ss.SSS';
timeU2.Format = 'yyyy/MM/dd HH:mm:ss.SSS';

disp([timeC2,timeU2])

>> disp([timeC2,timeU2])

2018/11/22 16:20:30.000 2018/11/22 16:20:29.917

2018/11/22 18:50:04.000 2018/11/22 18:50:04.051

2018/11/22 20:05:07.000 2018/11/22 20:05:07.041

2018/11/22 21:11:25.000 2018/11/22 21:11:24.912

...

[Walter Roberson]

Note that hypothetically a time can be within 0.1 tolerance of two different samples that are not within 0.1 tolerance of each other.

Answer 3

[Walter Roberson]

There are no identical timestamps between the two.

timeC.Format = 'dd-MMM-uuuu HH:mm:ss.SSSSSSSSS';
timeU.Format = 'dd-MMM-uuuu HH:mm:ss.SSSSSSSSS';
>> timeC(83957),timeU(24308)
ans = 
  datetime
   22-Nov-2018 22:27:40.000000000
ans = 
  datetime
   22-Nov-2018 22:27:39.966000000
>> dddd = timeC(83957)-timeU(24308); dddd.Format = 'hh:mm:ss.SSSSSSSSS'
dddd = 
  duration
   00:00:00.034000000

There is no smaller time difference (but there might be some other places with the same difference.)

Answer 4

[Peter Perkins]

As others have said, you don't have any matches between those two datetime vectors. The seem to be off by anything from about .5s to about 1s.

Does 22-Nov-2018 15:42:28.482 match 22-Nov-2018 15:42:28.000 in the other? I guess so. But then does 22-Nov-2018 22:27:58.965 match 22-Nov-2018 22:27:58.000 in the other? or 22-Nov-2018 22:27:59.000.

I think your problem is very ill-posed, and just making the code run without erroring is not your problem. So, some advice:

1) Do not use datenums. If you must convert datetimes to numeric in order to use ismembertol, use posixtime or something similar.

2) withtol will give you want clearly seem like wrong answers.

3) If the issue is that you have a progressive drift, use dateshift to remove that fractional part.