Datetimes don't seem to be recognised

Can anyone explain to me why this isn't working? For some reason, the datetime t is not recognised as occurring between times.t1.start and times.t1.end, even though it DOES! It is recognised as occurring after t1.start, but not before t1.end.

isbetween() also does not work so I am guessing I have overlooked something simple...

start=datetime(2021,08,11,09,58,25); 
last=datetime(2021,08,11,10,01,00); 

t=gps.DateTime(ii) %reading same datetime from my table
t=datetime(2021,08,11,09,59,25)

%if vessel position time occurs during particular transect working on
if (t>=start)&&(t<last)
    
    %append it to specific part of structure
    disp('It works!')
    
else
    disp('It does not work')
end


%Datetime from my table:
t = 

  datetime

   11-Aug-2021 09:59:25
   
%Datetime datetime(2021,08,11,09,59,25)

t = 

  datetime

   11-Aug-2021 09:59:25

Answer 1

[Star Strider]

I’m not certain what you’re doing and I can’t run the posted code. I’m also not certain what the actual problem is.

See if the isbetween function will do what you want with respect to the if block.

.

[Louise Wilson]

Sorry, I have amended the code above. Basically, it works if I manually enter the datetime, but if I pull the same datetime from a table, it doesn't work, even though the values look the same? What could be wrong?

[Star Strider]

I’m guessing here, however be certain that the ‘datetime’ from the table is actually a datetime array and not a string or character array the just looks like a datetime array.

One way to determine that is with the whos function —

a = '11 Nov 1918 11:00:00';
b = datetime('11 Nov 1918 11:00:00', 'InputFormat','dd MMM yyyy HH:mm:ss');

whos a b

  Name      Size            Bytes  Class       Attributes

  a         1x20               40  char                  
  b         1x1                 8  datetime              

They may appear to be the same however are not. Sometimes, readtable will read a character array as a character array if it does not recognise the format, so it will not be converted automatically to an actual datetime array. (That was actually the situation in this illustration, and the reason I added the 'InputFormat' name-value pair, when it failed the first time to convert automatically.)

And for the record, I still believe that isbetween would be likely more efficient and robust for the if test!

.

[Louise Wilson]

Hmm, that makes sense, but I am not sure it is the case here.

>> whos(subset.DateTime(1))
Error using whos
Must be a string scalar or character vector.

[Star Strider]

Well, they¹re definitely datetime values. So much for that theory!

I used this code —

LD = load('Louise Wilson subset.mat');
subset = LD.subset;

start=datetime(2021,08,11,09,58,25); 
last=datetime(2021,08,11,10,01,00); 

TF = isbetween(subset.DateTime, start, last)

to get this result —

TF =

5×1 logical array

0

0

0

0

0

So if it’s any consolation, none of them meet the criteria.

I still can’t follow the code so I’m not certain what you’re doing, however this may be part of the problem, if these are the dates and times being compared. Nothing is actually wrong, it’s just that the dates in ‘subset.DateTime’ aren’t in the interval being tested. That may not be true for all values of a larger datetime array, however it’s true for this set.

EDIT —

The problem is that the format is different between the two sets of datetime arrays —

start.Format = dt.Format

produces —

start =

datetime

08/11/2021 09:58:25

so the days and months are reversed between the different values. The ‘subset.DateTime’ format is 'MM/dd/uuuu HH:mm:ss' and the ‘start’ and ‘end’ format is 'dd-MMM-uuuu HH:mm:ss' that being the problem. I am not certain which is the desired format so I will not convert them. Nevertheless, they all must be the same, and then the logic will work as intended. Convert them using the approach I used here, supplying the desired format string for both the comparison and ‘subset’ arrays. Then do the comparison.

.

[Louise Wilson]

It's not true, because if you look at the actual datetime values in the table, they do occur between start and last. All of them do!

[Louise Wilson]

Hmm, I tried that already but I must have done it wrong. Thank you so much!!

[Star Strider]

As always, my pleasure!

.

[Louise Wilson]

Is there by any chance a way to 'correct' the current datetime format, for example to switch month and day around?

[Walter Roberson]

You should do that at the time you read in the data into your table.

filename = 'FileNameGoesHere.txt';   %the original file not the mat file
opt = detectImportOptions(filename);
now = datetime('now');
fmt = now.Format;
opt = setvaropts(opt, 'DateTime', 'InputFormat', 'dd/MM/uuuu HH:mm:ss', 'Format', fmt);
YourData = readtable(filename, opt);

[Louise Wilson]

Thanks, it's actually a .mat file which is huge so I wasn't sure how to go about it. I will make sure to do that from now on.

[Star Strider]

For a .mat file, it is likely easiest to keep the format they are already in, and convert ‘start’ and ‘end’ to that format. This just make it easier to understand, removing any ambiguity.

The datetime variables are stored in a generic form that is independent of the selected format, so they won¹t change.

.

[Louise Wilson]

Thanks, I ended up doing this, because they are the wrong way round. Learned my lesson! Don't trust American equipment for your datetime :-P

%date format is mixed up in table (month and day wrong way round)
correct_date=datetime(2021,08,11,'Format','dd/MM/yyyy');
for i=1:height(gps)
gps.DateTime(i)=correct_date+timeofday(gps.DateTime(i));
end

[Star Strider]

That will work, however the loop may not be necessary if the times are already matching the dates and just need to be combined with the dates.

.

[Walter Roberson]

No, it is completely unnecessary for the Format to match.

The problem is not a mismatch in format: the problem is that the data in the file has month 11 day 8 stored in it, instead of month 8 day 11.

When readtable or textscan with datetime specification are used without a specific Format, they try to guess the format. If they see 11/08/2021 then they check all of the entries to see if the hypothesis of MM/dd/yyyy works. If it does not find any entries with leading number greater than 12, then it assumes that the initial number is month. If it finds leading values greater than 12 it proceeds to check the second pair to see if any of those are greater than 12; if not then it guesses dd/MM. If entries in the first and second both have values greater than 12 it will revert to the MM/dd guess.

In the case of this subset, all of the entries were 11/08 in the original file (before reading into the table) and so it found no evidence that MM/dd was wrong and stuck with that. If the range of dates on input had been a bit greater it would have automatically detected dd/MM.

[Star Strider]

I agree that with respect to the inner workings of the code it’s not necessary for the formats to match, it is necessary for the format to match in order to hard-code (in this instance) the ‘start’ and ‘stop’ times . That the formats were ambiguous — with the resulting confusion — was the problem here. August is not November !

.

[Walter Roberson]

No, changing the Format of the start and last times here would not help.

You have three choices in this situation:

1. Go back and fix the data source so it has the month and day the right way around, such as by using InputFormat at the time of readtable(); OR
2. Code the start and last times deliberately wrong to match the data: start = datetime(2021,11,08,09,58,25); last = datetime(2021,11,08,10,01,00); OR
3. Fix-up the data already read by switching day and month

To fix-up:

%gps datetime has reversed day and month
fmt = start.Format;
[year, mon, day] = ymd(gps.DateTime);
gps.DateTime = datetime(year, day, mon, 'Format', fmt) + timeofday(gps.DateTime);

[Louise Wilson]

Wonderful thank you! Thank you for the explanation.

[Walter Roberson]

Sorry, had a typo in my fix-up code.

[Louise Wilson]

All good, it works perfectly :-) Thank you. The data I am working with is huge and a lot of time consuming processing has already been done so the fix up solution is preferrable by far!

[Steven Lord]

If that doesn't work, perhaps the data in the file was more precise than you thought it was.

t1 = datetime(2021, 11, 10, 14, 25, 00)

t1 = datetime
   10-Nov-2021 14:25:00

t2 = datetime(2021, 11, 10, 14, 25, 0.4)

t2 = datetime
   10-Nov-2021 14:25:00

t1 >= t2 % correctly false even though they display the same

ans = logical
   0