Replacing unassigned values in a vector with nans instead of zeros

Morning everyone,

I'm resizing vectors in a loop. Not good I know and probably why it takes 30 minutes to run, but beside from reading through all the files a first time to determine the correct length of the vectors and then reading through a second time to fill in the vectors I can't see a better way. As there is no easy method of determining the length I need.

However, currently if a file can't be read - some of the files are dodgy and I put a try-catch statement around them - then unassigned values in the vector automatically equal zero. Is there a way to change that behaviour so that an unassigned vector value is a nan?

For example

r(1) = 2;
r(3) = 4;
>> r
r =

     1     0     2

is there a way to make r(2) a nan without explicitly assigning it?

[Chad Greene]

Is preallocating r an option? I'm thinking something like this:

r = nan(1,3);
r(1) = 2;
r(3) = 4

[jlt199]

preallocating to the correct number is not an option as it is unknown

Answer 1

[Steven Lord]

Preallocate r to be a NaN vector with a number of elements equal to some upper bound on the number of elements that it could possibly contain (for instance, if you have 10 files and each could add 7 elements to the vector preallocate it to be nan(1, 70)) then fill it in and trim off the excess elements after the for loop is complete.

[jlt199]

Thanks Steven, Is this a method you would use or would you bite the bullet and read though the files twice (first to determine the length of the vectors)?

[Stephen23]

@jlt199: it is quite a reasonable method: preallocate the largest array that you will need, and then trim it afterwards. You might be surprised at how efficient this is.

[Guillaume]

I/O is slow, so reading the same file twice (even though the computer may cache it in memory for the 2nd read) is not a good idea.

When reading files, there's always going to be some amount of 'allocating as you go'. This does not mean that it has to be inefficient. If you give us the details of how you're reading the files, we may be able to tell you a more efficient way of reading them.

[jlt199]

Thanks Guillaume, I'm reading them using the following code:

fid = 'W:/Algorithm Dept/Flat Plat Defects/Circular Defects/';
files = dir([fid,'*.txt']);
for i = 1:length(files)
  ML = readtable([fid,file.name],'HeaderLines',8,'ReadVariableNames',false);
...
end

[Guillaume]

The reading is efficient, but I don't see how that portion of code links to your question. Assuming you're concatenating the tables vertically and you want nans for the missing files, this is how I'd do it:

%...
ML = cell(numel(files));
for filenumber = 1:numel(files)
   try
      ML{filenumber} = readtable(...);
   catch
      ...
   end
end
failedread = cellfun(@isempty, ML);
maxheight = max(cellfun(@height, ML(~failedread)));
width = cellfun(@width, ML(~failedread));
assert(numel(width) > 0 && all(width == width(1)), 'table concatenation will fail')
ML{failedread} = array2table(nan(maxheight, width(1)));
allML = vertcat(ML{:});

Answer 2

[Jonathan Avesar]

I also have this question. I can't pre-allocate because I don't know how large the matrix will be in advance.