replacing the NAN with value

hi. i have 2 matrices V & I. some elements of I are zero. then R=V/I returns unavailable values. i need to keep the size of 3 matrices constant. so what do you suggest me for replacing NAN values? does it make sense to replace zeros with positive minimum value of I? thank you for your help

Answer 1

[Image Analyst]

Study this snippet. Adapt as needed.

m=[1,3,nan, 5,7,nan,2]
% Find the nans
nanLocations = isnan(m)
% Pick some other value to set the nans to.
alternateValue = 42;
% Do the replacemenet.
m(nanLocations) = alternateValue

In the command window:

m =
     1     3   NaN     5     7   NaN     2
nanLocations =
     0     0     1     0     0     1     0
m =
     1     3    42     5     7    42     2

[maryam]

Dear Image Analyst my problem is finding a value to replace nans. i have no idea what is the the most appropriate value for replacing

[Image Analyst]

Well then I certainly wouldn't know. Maybe you just want to do bilinear or bicubic interpolation from surrounding good areas?

Answer 2

[Star Strider]

The NaN values are the result of a 0/0 division, so the same elements of ‘V’ and ‘I’ are both zero in those situations. That is an indeterminate result.

I do not know all of what you are doing, but since the value of ‘R’ (the resistor you are measuring the voltage drop across) will likely not change, I would replace the NaN values with whatever the average value for ‘R’ is in your other calculations.

[maryam]

would you plz check my excel file? when current is zero, Voltage may have different values.

[Star Strider]

I can’t make any sense out of it.

I have no idea how you generated those data. I also have no idea how you had a zero voltage drop across your resistor with a finite current going through it, unless the resistor became a short circuit, or finite voltage with zero current, unless your resistor became an open circuit.

I completely fail to understand the first column, ‘X’, or whether it is important in understanding the rest of your data.

Something is definitely not correct in whatever you are doing. You probably need to perform your experiment again.

[maryam]

at first my system (like a switch) is in closed situation, so the voltage drop across it is zero, but current has a value. when two parts of that starts separating current tends to become zero and open voltage will be appear across it. the first column of excel file is time, it is not important here. so do you have more suggestion? tnx

Answer 3

[Stephen23]

What is wrong with keeping the NaN values? Why would you want to get rid of them?

NaN's tell you very important information, and it would be programmatically very poor practice to hide a divide-by-zero with some fake data.

Do not replace them with "some positive minimum value".

If later during your data processing you must use a function that does not accept NaN's (e.g. spline fitting), then you can deal with it then in the most appropriate way for the specific situation. You might need an error message, or to remove them using isnan... but if you have replaced the values with "some positive minimum value" then you have lost information about your data that you cannot retrieve later.

[maryam]

i want to use R=V/I matrix in neural network. with NaN values the network doesn't work.

[Stephen23]

It really depends on how much you know about the system that you are working with. Does it change? Do you expect the R values to change between measurements? Is there a temperature dependency in the system? How accurate are your measurements?

If you have a fixed-value resistance and you are not worried about a bit of drift in your values due to effects from different currents or thermal effects, then you can assume that the R is constant and take the mean of the finite R-values.

If you need more precision than this, then those data are critical, because they are indeterminate values. Only knowing more information about the system can tell you how to handle these NaN's, but it is likely that they should be ignored in the neural network.

[maryam]

i have a physical system that measure current through opening operation. when two parts separate,current will be zero (zero values in Matrix I). R values depend on current, so they arent constant. these zeros produce very big changes in R. i dont want to lose these points. i just want to replace them with an appropriate value. thank you for your patience.

[maryam]

i would be very appreciate if you check my excel file.

Answer 4

[Guillaume]

Looking at your excel file, I think the first problem is that excel #DIV/0! can mean several things. It can mean NaN which is what matlab interprets it as, or it could mean +Inf or -Inf, which it is in your case.

From a physical perspective, when your switch is open, the resistance is infinite (hence current is 0). So the first thing, I'd do in matlab is recalculate the resistance in matlab. If you just import your excel file with

m = xlsread('42.xlsx');

then

m(:, 5) = m(:, 2) ./ m(:, 3); %calculate resistance

No NaN anymore, but +Inf and -Inf, which is physically correct. That may not help you for further processing, but when you say 'I just want to replace them with an appropriate value', I'm not sure what is more appropriate than infinite. You could replace them with a very big value (but how big is big?), but that would be less correct:

m(~isfinite(m(:, 5), 5) = someverybigvalue; %replace inf by very big R

[maryam]

that is exactly my question! i should do further analysis on these data (for example with neural network). if i remove them i will lose important information. but i dont know how to replace them

[Guillaume]

Well, I don't think you're going to get an answer on what to replace them with here. This is not really a matlab problem anymore. You've been shown how to replace them.

You're the best place to decide what to replace them with, and it all depends on what further analysis you want to do. Or maybe, use a different analysis tool, one that can cope with Inf.

I still contend that the proper value should be Inf, anything other is just a lie.

I also don't see why you can't just remove them, if it cause problem. If I understood correctly, that data corresponds to a completely different state of the system (open circuit). It's not going to give any information about the closed circuit.

[maryam]

thank you for your reply. i have 100 matrices like one i uploaded. they have different numbers of zero. if i remove zeros the size of all matrices will changed which cause problem in coding with NN.

[Guillaume]

You could always trim all your matrices to the size of the smallest one. Eliminate data at random from the longer ones.

Assuming your 100 matrices are in cell array c:

%c contain matrices, with NaN or Inf in the fifth column
c = cellfun(@(m) m(isfinite(m(:, 5)), :), c, 'UniformOutput', false); %remove NaN/Inf
minheight = min(cellfun(@(m) size(m, 1), c); %get height of smallest matrix
c = cellfun(@(m) m(sort(randperm(size(m, 1), minheight)), :), c, 'UniformOutput', false); %reduce all matrices to the same size by minheight random rows in each

Otherwise you could do some interpolation as Image Analyst suggest. However, that would just be inventing data that does not exists.