How to tell Matlab to ignore NaNs in calculations/loops

Hi, I am trying to make some operations with Matlab but I need to ignore the generatedmissing Values / NaNs Matlab gives out after reading in the data from excel. Since now the code is only taking the empty cells and copy them to the new spreadsheet. Instead the NaNs should be ignored. (I am replicating a Finance paper which makes use of the momentum strategy described in Jegadeesh/Titman(1993).) The code is posted below

The code is taken to some part from paper which is about quite the same topic. I have made only some few channges. I Can't delete all the columns which have some NaNs in it.

Thank you for any helpful comments

The code as used is

%  [stockdata] = xlsread('Argentina.xlsm',-1);
%  [stockid] = xlsread('Argentina.xlsm',-1);

s=-1+(1-(-1)).*rand(715,215);
stockdata=transpose(s);
stockdata(stockdata > 0.5) = NaN;
stockid=[1:715];

[totalmonths,totalstocks] = size(stockdata);% Number of months, number of stocks
period1 = 6; %order period
period2 = 6; % holding period
startmonth=14;
for i=1:totalmonths
    nbrstocks(i) = totalstocks - sum(isnan(stockdata(i,:)));
end
for i = 1:length(period1)
    for j = 1:length(period2)
        p1 = period1(i);
        p2 = period2(j);
        Rwinner = zeros(totalmonths,1);
        Rloser = zeros(totalmonths,1);
        idwinner = zeros(totalmonths,round(max(nbrstocks)*0.33));% empty matrix for winners
        idloser = zeros(totalmonths,round(max(nbrstocks)*0.33));% empty matrix for losers
        for k = startmonth:totalmonths+1-p1-p2 %  loop from start till end
            start1 = k;
            stop1 = start1+p1-1;
            ordermonths = start1:stop1;
            start2 = stop1+1;
            stop2 = start2+p2-1;
            holdmonths = start2:stop2;
            nstocks = nbrstocks(k);
            % order R for the ordering weeks
            data1 = stockdata(ordermonths, 1:nstocks);
            R1 = ones(1,nstocks);
            for m = 1:length(ordermonths)
                R1 = R1.*data1(m,:);
            end
            R1 = R1-1;
            % select for the winners and losers
            [B,idx] = sort(R1,'descend');
            ncandidates = round(nstocks*0.33);
            idwinner(k,1:ncandidates) = stockid(idx(1:ncandidates));
            idloser(k,1:ncandidates) = stockid(idx(end-ncandidates+1:end));
            % caculate the R for winners and losers in holding months
            data2winner = stockdata(holdmonths, idx(1:ncandidates));
            data2loser = stockdata(holdmonths, idx(end-ncandidates+1:end));
            R2w = ones(1,ncandidates);
            R2l = ones(1,ncandidates);
            for m = 1:length(holdmonths)
                R2w = R2w.*data2winner(m,:);
                R2l = R2l.*data2loser(m,:);
            end
            R2w = R2w-1;
            R2l = R2l - 1;
            Rwinner(k) = mean(R2w);
            Rloser(k) = mean(R2l);
        end
        xlswrite(fullfile(['strategy_' num2str(p1) '_' num2str(p2) '_''.xlsx']),Rwinner,'Rwinner');
        xlswrite(fullfile(['strategy_' num2str(p1) '_' num2str(p2) '_''.xlsx']),Rloser,'Rloser');
        xlswrite(fullfile(['strategy_' num2str(p1) '_' num2str(p2) '_''.xlsx']),idwinner,'idwinner');
        xlswrite(fullfile(['strategy_' num2str(p1) '_' num2str(p2) '_''.xlsx']),idloser,'idloser');
    end
end

Answer 1

[Oleg Komarov]

If you have the Statistics or the Financial toolbox, use nanmean().

EDIT

What are you trying to do with:

% order R for the ordering weeks
data1 = stockdata(ordermonths, 1:nstocks);
R1 = ones(1,nstocks);
for m = 1:length(ordermonths)
    R1 = R1.*data1(m,:);
end
R1 = R1-1;

the cumulative return? Then set the NaNs to 1.

idx        = isnan(data1);
data1(idx) = 1;
R1         = prod(data1) - 1;

no need to loop, since prod() is vectorized.

Apply same concept here:

% caculate the R for winners and losers in holding months
data2winner = stockdata(holdmonths, idx(1:ncandidates));
data2loser = stockdata(holdmonths, idx(end-ncandidates+1:end));
R2w = ones(1,ncandidates);
R2l = ones(1,ncandidates);
for m = 1:length(holdmonths)
    R2w = R2w.*data2winner(m,:);
    R2l = R2l.*data2loser(m,:);
end
R2w = R2w-1;
R2l = R2l - 1;

AND then use nanmean().

[Tobi83]

Thanks for your quick answer. I am not sure if I got you right but I don't see how nanmean would solve the problem because I think the error occurs some rows before, probably the sorting process is the problem. But again I am not sure about that.

[Oleg Komarov]

Well, I cannot test your script since I don't have the data.

Post a brief sample of data that generates the problem.

[Tobi83]

the stockdata matrix looks like this but the complete one has 715 columns and 215 rows. the following shows the problem with a smaller matrix

r = rand(40,30);
r(r > 0.5) = NaN;

respectively the stockid matrix is a one row 30 columns matrix of idnumbers from 1 to 30.

Thanks again for your assistance

[Oleg Komarov]

Still, I don't have stockid() to go further.

[Tobi83]

Thanks for keeping your eye on the ball
the following can be used for both stockid and stockdata

    s=-1+(1-(-1)).*rand(715,215);
    stockdata=transpose(s);
    stockdata(stockdata > 0.5) = NaN;
    stockid=[1:715];

[Oleg Komarov]

See my edit.

Answer 2

[Albert Yam]

How about just going low level. At the start of the loop (or where applicable),

if any(isnan(VARIABLE))
    continue
end

[Tobi83]

I will try it and keep you updated. Thanks so far for your help

[Tobi83]

So far I tried to insert the command at the position as shown above. The result is that the vector doesn't consist of NaNs anymore but of zeros, which isn't plausible. So either it doesn't work or I got you (or the code) wrong

if any(isnan(R1)) continue;
[B,idx] = sort(R1,'descend');
end

However I am grateful that you share your ideas with me.

Tobi

[Albert Yam]

Looking at others' answers, not sure what you want to skip. This will work if you put it in the right place. Not sure where the right place is.

if any(isnan(VARIABLE)) % if this is true, run the 'continue'
    continue % this skips this iteration of the loop
    %any code here is skipped, due to the continue
end
[B,idx] = sort(R1,'descend'); %should go here if you want to run it

[Tobi83]

Thanks for your help again. I tried it like this

if any(isnan(R1)) 

continue 
[B,idx] = sort(R1,'descend');
end

but this gave me an RWinner of zeros. I used this as data in combination with the code above

s=-1+(1-(-1)).*rand(715,215);
  stockdata=transpose(s);
  stockdata(stockdata > 0.5) = NaN;
  stockid=[1:715];

Answer 3

[Sebastian Holmqvist]

data_vec = [1 2 3 4 5 NaN 7 8 NaN 10 12];
data_vec(~isnan(data_vec))
ans =
 1 2 3 4 5 7 8 10 12

[Tobi83]

Thank you, but I don't know how I can use that in the code above. What would you propose?

[Sebastian Holmqvist]

Well, this code excludes the NaN elements. So it all depends on what you want to do. I can't make much of your code (since I don't know the input data or what you want to produce) so it's hard for me to make an educated guess. A guess (probably a malfunctioning one) would be to insert

 R1 = R1(~isnan(R1));

before

 [B,idx] = sort(R1,'descend');

[Tobi83]

Thanks again. This doesnt't work because the number of elements must agree in

R1 = R1.*data1(m,:);

You simulate data which looks like the one I am using by copying the following and using it instead of the xlsread commands:

s=-1+(1-(-1)).*rand(715,215);
stockdata=transpose(s);
stockdata(stockdata > 0.5) = NaN;
stockid=[1:715];

Answer 4

[borge]

Hi Tobi83,

I am attempting to do a similar analysis andI have attempted to a compile a code similar to yours, but ran into the same problem, so I was wondering how your code ended up looking like?

Many thanks in advance!

Best, borge