I have a data matrix of size 890*9 , now I want to find out the correlation coefficient matrix, So I use the corrcoef command and get a 9*9 matrix. But the problem is I get a 9*9 matrix which is full of NaN. What am I doing wrong

The attached file is the 890*9 matrix.

Answer 1

[John D'Errico]

A useful little tool I have for these questions displays some information about your array.

data has 890 rows and 9 columns
          Var.#1     Var.#2     Var.#3     Var.#4     Var.#5     Var.#6     Var.#7     Var.#8     Var.#9
   Min          0          5       0.96          0     -638.7       -998          0     -756.9       2.82
  1.0%          0          7       1.22          0     -118.9     -778.1          0      -8.29       5.77
  5.0%          0         90       2.26        5.7      889.3         83          0       0.04       8.89
 10.0%          0        184       2.81       13.1       1302      548.4        0.1       0.98      10.04
 25.0%        240        643       4.58      26.56       2068       1594        8.3        1.9       12.8
 50.0%       1421       3100       8.04      52.68       3436       3428       77.2       3.03       16.2
 75.0%        NaN        NaN        NaN      106.2       7267       8100        728       4.66      19.74
 90.0%        NaN        NaN        NaN      212.8  2.156e+04  2.508e+04       3778       7.04      26.06
 95.0%        NaN        NaN        NaN      332.6  3.778e+04        NaN        NaN       9.29      37.04
 99.0%        NaN        NaN        NaN        NaN        NaN        NaN        NaN       7928        NaN
   Max        NaN        NaN        NaN        NaN        NaN        NaN        NaN        NaN        NaN

So, if we look at your array, we find that it is itself riddled with NaNs. In fact, every column of y has at least one element that is a NaN, and some of those columns are more than 25% NaNs. So I would expect to see a result that is essentially a 9x9 matrix of NaNs.

sum(isnan(y),1)
ans =
 262   378   231    15    32    57    52     4    25

size(y)
ans =
 890     9

sum(isnan(y),1)/890*100
ans =
  29.438  42.472  25.955  1.6854  3.5955  6.4045  5.8427  0.44944  2.809

So column 2 has more than 42% NaN elements.

My suggestion is that when you see something strange, like a correlation coefficient matrix that is full of NaNs, that you then look at your data that went into creating that result. Always apply common sense.

[Parth Parakh]

I was aware of these NaN's in the data , but I thought the code for corrcoef finds its way around it somehow. I now realise how stupid it was to think that way. Anyway, Thanks a lot! cheers