transform the while loop into a loop for

Hello ,

I would like to turn the following block of code into a loop for

while feof(fid) == 0
        tline = fgetl(fid);
        TargetVar = regexp(tline,' ','split');


        if length(TargetVar)>2
  
        [xc, reste] =strtok(tline, ' ');
        [yc, reste] =strtok(reste, ' ');
        [zc, reste] =strtok(reste, ' ');
        [Rayon, reste] =strtok(reste, ' ');

        h=h+1;
        Pts(h,1:3)= [str2num(xc) str2num(yc) str2num(zc)] ;
        Ray(h,1)=str2num(Rayon);
        Vol(h,1)=((4/3)*pi)*(Ray(h)^3);


        end
        

    end

Answer 1

[Image Analyst]

Try this:

maxLines = 9999999; % More than you expect to ever need.
for k = 1 : maxLines
	tline = fgetl(fid);
	if ~ischar(textLine)
		% End of file so break out of loop.
		break;
	end
	
	% The rest of your code follows (I didn't check it).
	TargetVar = regexp(tline,' ','split');	
	
	if length(TargetVar)>2		
		[xc, reste] =strtok(tline, ' ');
		[yc, reste] =strtok(reste, ' ');
		[zc, reste] =strtok(reste, ' ');
		[Rayon, reste] =strtok(reste, ' ');
		
		h=h+1;
		Pts(h,1:3)= [str2num(xc) str2num(yc) str2num(zc)] ;
		Ray(h,1)=str2num(Rayon);
		Vol(h,1)=((4/3)*pi)*(Ray(h)^3);		
	end
end

[PIERRE OLIVIER SCHOUEL ATANGANA]

Thanks so much

[Walter Roberson]

This is not equivalent code. It will give out after roughly

lines = (9999999+1);
chars_per_line = 50;
max_file_size_estimate_gigabytes = lines * chars_per_line / 2^30

max_file_size_estimate_gigabytes = 0.4657

I can store files 1000 times larger than that on my system, without difficulty.

Only transform while loops into for loops if you have a maximum number of iterations (or fixed number of iterations): if you cannot put an upper bound on the number of iterations, then leave it as a while loop.

[Image Analyst]

I agree that while is preferable. I just gave the for loop way because I guess he wanted it for comparison purposes.

Often people know how big their files are. Like if you know you're writing data for up to hundred files and each file will have 5 numbers for it, then in your output text file you won't have more than 100 lines in it and it won't be hundreds of GB. The comment says that you put the line count at way more than you know you'll ever need, and in that case, the code works.

maxLines = 9999999; % More than you expect to ever need.

Answer 2

[Walter Roberson]

for is only for use with a fixed number of iterations, but your code has no inherent limits on the number of lines it reads from a file.

In theory you could be running MATLAB on Linux or MacOS with a ZFS file system, which permits invididual files as large as 2^64 - 1 bytes. If you transform the while into a for you would have to loop

for LINE = 1:inf

but MATLAB constrains the number of iterations for a for loop to be at most flintmax('double') -- 2^54 . So you would have to use nested loops to get the rest of the potential iterations before you ran into the potential file size limit.

Answer 3

[PIERRE OLIVIER SCHOUEL ATANGANA]

Hello,

please how to parallelize matlab code which has a for included in a break. eg:

maxLines = 9999999

for k = 1 : maxLines

tline = fgetl(fid);

if ~ischar(textLine)

% End of file so break out of loop.

break;

end

%there are all instructions below

end

[Image Analyst]

Do you even have the Parallel Computing Toolbox? Type ver to find out. But even if you do, I'm not sure it can be parallelized. But I haven't played around with parallelizing code much.

If you want, you can use fileread() to suck up the whole file into one character variable.

[Walter Roberson]

What would be run in parallel? Reading different files? Reading different lines from the same file?

[PIERRE OLIVIER SCHOUEL ATANGANA]

Hello,

it is reading the different files which is executed in parallel.

[Walter Roberson]

If you want to run multiple files in parallel, then you could

parfor fileIdx = 1 : number_of_files
   thisfile = filenames{fileIdx};
   fid = fopen(thisfile);
   ... do the reading and mathematical processing
   fclose(fid)
   results{fileIdx} = {Pts, Ray, Vol};
end

However if you are doing this, there is no reason to swtich from while loop to for loop to read the lines.

If the files fit into memory, there are notably more efficient ways of processing the input that do not use for or while to read the values for any particular file.

[Image Analyst]

Pierre, you never answered my question and said whether you in fact HAVE the Parallel Processing Toolbox. Type ver and show us what it says (except for your license number of course).

[PIERRE OLIVIER SCHOUEL ATANGANA]

Good evening,

When I enter the worm command on the MATLAB prompt this is what I get

-----------------------------------------------------------------------------------------------------

MATLAB Version: 9.4.0.813654 (R2018a)

MATLAB License Number:

Operating System: Microsoft Windows 8.1 Professionnel Version 6.3 (Build 9600)

Java Version: Java 1.8.0_144-b01 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

-----------------------------------------------------------------------------------------------------

MATLAB Version 9.4 (R2018a)

Simulink Version 9.1 (R2018a)

Aerospace Blockset Version 3.21 (R2018a)

Aerospace Toolbox Version 2.21 (R2018a)

Antenna Toolbox Version 3.1 (R2018a)

Audio System Toolbox Version 1.4 (R2018a)

Automated Driving System Toolbox Version 1.2 (R2018a)

Bioinformatics Toolbox Version 4.10 (R2018a)

Communications System Toolbox Version 6.6 (R2018a)

Computer Vision System Toolbox Version 8.1 (R2018a)

Control System Toolbox Version 10.4 (R2018a)

Curve Fitting Toolbox Version 3.5.7 (R2018a)

DO Qualification Kit Version 3.5 (R2018a)

DSP System Toolbox Version 9.6 (R2018a)

Data Acquisition Toolbox Version 3.13 (R2018a)

Database Toolbox Version 8.1 (R2018a)

Datafeed Toolbox Version 5.7 (R2018a)

Econometrics Toolbox Version 5.0 (R2018a)

Embedded Coder Version 7.0 (R2018a)

Filter Design HDL Coder Version 3.1.3 (R2018a)

Financial Instruments Toolbox Version 2.7 (R2018a)

Financial Toolbox Version 5.11 (R2018a)

Fixed-Point Designer Version 6.1 (R2018a)

Fuzzy Logic Toolbox Version 2.3.1 (R2018a)

GPU Coder Version 1.1 (R2018a)

Global Optimization Toolbox Version 3.4.4 (R2018a)

HDL Coder Version 3.12 (R2018a)

HDL Verifier Version 5.4 (R2018a)

IEC Certification Kit Version 3.11 (R2018a)

Image Acquisition Toolbox Version 5.4 (R2018a)

Image Processing Toolbox Version 10.2 (R2018a)

Instrument Control Toolbox Version 3.13 (R2018a)

LTE HDL Toolbox Version 1.1 (R2018a)

LTE System Toolbox Version 2.6 (R2018a)

MATLAB Coder Version 4.0 (R2018a)

MATLAB Compiler Version 6.6 (R2018a)

MATLAB Compiler SDK Version 6.5 (R2018a)

MATLAB Distributed Computing Server Version 6.12 (R2018a)

MATLAB Report Generator Version 5.4 (R2018a)

Mapping Toolbox Version 4.6 (R2018a)

Model Predictive Control Toolbox Version 6.1 (R2018a)

Model-Based Calibration Toolbox Version 5.4 (R2018a)

Neural Network Toolbox Version 11.1 (R2018a)

OPC Toolbox Version 4.0.5 (R2018a)

Optimization Toolbox Version 8.1 (R2018a)

Parallel Computing Toolbox Version 6.12 (R2018a)

Partial Differential Equation Toolbox Version 3.0 (R2018a)

Phased Array System Toolbox Version 3.6 (R2018a)

Polyspace Bug Finder Version 2.5 (R2018a)

Polyspace Code Prover Version 9.9 (R2018a)

Powertrain Blockset Version 1.3 (R2018a)

Predictive Maintenance Toolbox Version 1.0 (R2018a)

RF Blockset Version 7.0 (R2018a)

RF Toolbox Version 3.4 (R2018a)

Risk Management Toolbox Version 1.3 (R2018a)

Robotics System Toolbox Version 2.0 (R2018a)

Robust Control Toolbox Version 6.4.1 (R2018a)

Signal Processing Toolbox Version 8.0 (R2018a)

SimBiology Version 5.8 (R2018a)

SimEvents Version 5.4 (R2018a)

Simscape Version 4.4 (R2018a)

Simscape Driveline Version 2.14 (R2018a)

Simscape Electronics Version 2.13 (R2018a)

Simscape Fluids Version 2.4 (R2018a)

Simscape Multibody Version 5.2 (R2018a)

Simscape Power Systems Version 6.9 (R2018a)

Simulink 3D Animation Version 8.0 (R2018a)

Simulink Check Version 4.1 (R2018a)

Simulink Code Inspector Version 3.2 (R2018a)

Simulink Coder Version 8.14 (R2018a)

Simulink Control Design Version 5.1 (R2018a)

Simulink Coverage Version 4.1 (R2018a)

Simulink Design Optimization Version 3.4 (R2018a)

Simulink Design Verifier Version 3.5 (R2018a)

Simulink Desktop Real-Time Version 5.6 (R2018a)

Simulink PLC Coder Version 2.5 (R2018a)

Simulink Real-Time Version 6.8 (R2018a)

Simulink Report Generator Version 5.4 (R2018a)

Simulink Requirements Version 1.1 (R2018a)

Simulink Test Version 2.4 (R2018a)

Spreadsheet Link Version 3.3.3 (R2018a)

Stateflow Version 9.1 (R2018a)

Statistics and Machine Learning Toolbox Version 11.3 (R2018a)

Symbolic Math Toolbox Version 8.1 (R2018a)

System Identification Toolbox Version 9.8 (R2018a)

Text Analytics Toolbox Version 1.1 (R2018a)

Trading Toolbox Version 3.4 (R2018a)

Vehicle Dynamics Blockset Version 1.0 (R2018a)

Vehicle Network Toolbox Version 4.0 (R2018a)

Vision HDL Toolbox Version 1.6 (R2018a)

WLAN System Toolbox Version 1.5 (R2018a)

Wavelet Toolbox Version 5.0 (R2018a)

>>

[Image Analyst]

OK, you have an enormous number of toolboxes including the Parallel Computing Toolbox so you should be able to use parallel processing. Have you gone through the demos/examples provided with that toolbox?

[PIERRE OLIVIER SCHOUEL ATANGANA]

Hello,

I don't know how to get the demos / examples that come with this toolkit.

Or you have to go to the Matlab documentation.

[Image Analyst]

Yes. Look at this page:

https://www.mathworks.com/help/parallel-computing/index.html

Especially look at the Getting Started link and the other links on the left hand side of the page.

[PIERRE OLIVIER SCHOUEL ATANGANA]

Hello,

I still have problems for the parallelization of this block of code, despite the solution you offered me.

The difficulty is in the following instructions:

sousEns=Pts(members,:);
sousRay=Ray(members);

[Walter Roberson]

Those two lines do not appear in what you have posted so far, so we do not have the context for them. We do not know what members is, and we do not know this is after the parfor where you would be working with the accumulated results -- we can't be sure that Pts and Ray are numeric arrays by then because we do not have the context.

[PIERRE OLIVIER SCHOUEL ATANGANA]

Please never another worry too,

How do I ignore I / O instructions (write, read, write to file, and read to file) in a MATLAB program?

[Walter Roberson]

What do you want to have happen in place of the i/o operations?

Answer 4

[PIERRE OLIVIER SCHOUEL ATANGANA]

In fact the context is that of the calculation of the ellipsoid from the coordinates of each ball. the block that I want to parallelize is the one that grouping balls from the coordinates.

When I change the for to parfor these 02 instructions cause me problems

[Walter Roberson]

Which two instructions?

I get the impression that you did not use the strategy I showed in

https://www.mathworks.com/matlabcentral/answers/1444364-transform-the-while-loop-into-a-loop-for#comment_1722509

[PIERRE OLIVIER SCHOUEL ATANGANA]

Good evening,

Depending on the solutions you offered me, I have several kinds of errors during compilation:

1.The entire array or structure 'Pts' and Ray is a broadcast variable.This might result in unncessary communication overhead

[PIERRE OLIVIER SCHOUEL ATANGANA]

2.Error unsing mex

Finally I can't quite use this:

mex my_code.m
addAttachedFiles(gcp,'my_code.m')

[Walter Roberson]

Pts and Ray are not broadcast variables in the outline I posted.

You are asking us to debug code that we cannot see.

[Walter Roberson]

You never mex a .m file. You mex code that is written in other languages, such as

mex my_code.c

mex is not the command to compile .m code into binary; that would be mcc https://www.mathworks.com/help/compiler/mcc.html

If you were to generate code from a .m file, then you would attach the generated object, such as a .mexw64 file, not the .m file.

[PIERRE OLIVIER SCHOUEL ATANGANA]

Good evening,

Here is the block of code I want you to help me debug:

parfor i=1:k
        members = (i == idx);
        disp(strcat('Cluster numero <',num2str(i),'>'));
        disp('De Centroid');
        disp(C(i,:));
        t_debut3=tic;
        
        sousEns=Pts(members,:);
        if(isempty(sousEns)==1)
      
            continue
        end
        sousRay=Ray(members);
        %disp('De Centroid');
        %disp(C(i,:));
        Echan=[];
end

[PIERRE OLIVIER SCHOUEL ATANGANA]

these are the next two instructions that I can't debug:

sousEns=Pts(members,:);
sousRay=Ray(members);

Is there a way to get around these two instructions?

[PIERRE OLIVIER SCHOUEL ATANGANA]

How to vectorize the following block:

for indx = 1:nbptsvisual
        for indy = 1:nbptsvisual
            poin = [x1(indx,indy) y1(indx,indy) z1(indx,indy)]';
            Pt = V1 * poin;
            x1(indx,indy) = Pt(1)+centro(1);
            y1(indx,indy) = Pt(2)+centro(2);
            z1(indx,indy) = Pt(3)+centro(3);
        end
        
    end

[Walter Roberson]

1.The entire array or structure 'Pts' and Ray is a broadcast variable.This might result in unncessary communication overhead

That is not an error, that is a warning.

In the context of the code you posted at https://www.mathworks.com/matlabcentral/answers/1444364-transform-the-while-loop-into-a-loop-for#comment_1739794 then the way you deal with that warning is to rewrite the loop as

parfor i=1:k
        disp(strcat('Cluster numero <',num2str(i),'>'));
        disp('De Centroid');
        disp(C(i,:));
end

and leave out the assignment to souEns and sousRay . Those assignments you are doing are assignments to local variables that will be discarded after the parfor, so there is no point in computing them. The only productive work your parfor is doing is displaying the cluster numbers and associated centroid.

[Walter Roberson]

Pt = V1 * poin;

What size is V1 ? It looks to me that it is most likely a 3 x 3 matrix. If so then,

%do this part ONCE
syms X1 Y1 Z1
syms v1 [3 3]
syms Centro [3 1]
poin = [X1, Y1, Z1].';
pt = v1 * poin + Centro(:)
ptfun  = matlabFunction(pt, 'vars', {v1, Centro, X1, Y1, Z1})

%now each iteration with new V1, centro, x1, y1, z1, do the following 

Pt = Ptfun(V1, centro(:), permute(x1, [3 1 2]), permute(y1, [3 1 2]), permute(z1, [3 1 2]));
x1 = permute(Pt(1,:,:), [2 3 1]);
y1 = permute(Pt(2,:,:), [2 3 1]);
z1 = permute(Pt(3,:,:), [2 3 1]);

[Walter Roberson]

Or... you could do

TM = [V1, centro(:)];
XYZ1 = [x1(:), y1(:), z1(:), ones(numel(x1),1)];

pt = TM * XYZ1;
x1 = reshape(pt(1,:), size(x1));
y1 = reshape(pt(2,:), size(y1));
z1 = reshape(pt(3,:), size(z1));

[PIERRE OLIVIER SCHOUEL ATANGANA]

thank you for this solution track,

I will implement it and report to you

[PIERRE OLIVIER SCHOUEL ATANGANA]

Hello,

I would like to vectorize the following code:

for j = 1:size(sousEns,1)
    
    xx=[];
    yy=[];
    zz=[];
    
    xx=sousRay(j)*sin(phi).*cos(theta)+sousEns(j,1);
    yy=sousRay(j)*sin(phi).*sin(theta)+sousEns(j,2);
    zz=sousRay(j)*cos(phi)+sousEns(j,3);
    
    n=size(xx(:),1);
    Echan2=[];
    Echan2(1:n,1)=xx(:);
    Echan2(1:n,2)=yy(:);
    Echan2(1:n,3)=zz(:);
    
    Echan=[Echan2;Echan];
    
    [xu,yu,zu] = sphere;
    x = xu*sousRay(j) + sousEns(j,1);
    y = yu*sousRay(j) + sousEns(j,2);
    z = zu*sousRay(j) + sousEns(j,3);
    c = ones(size(z))*1;
    hold on;
    
    surf(x,y,z,c);
end

[Walter Roberson]

    Echan=[Echan2;Echan];

You do not use that information, so it is not clear that it is worth calculating it.

You appear to draw a sphere at each location. Is it correct that you want that merged into a single graphics object? Doing that might be a bit tricky unless you switch to patch()

[PIERRE OLIVIER SCHOUEL ATANGANA]

Good evening,

Echan = [Echan2; Echan]; 

It is a matrix which contains the coordinates of each ball. And a ball is defined by three coordinates x, y and z

[Walter Roberson]

You do not use the information in Echan to draw the balls.

The information you accumulate in Echan has different coordinates than what you draw with (unless by coincidence.)

The reason I bother mentioning this is that if you are not going to use Echan to draw the balls, then there is no point in building that information and you can throw away a number of lines. Doing that would remove any need for us to worry about what the sizes of various arrays are.

But if you need to use those coordinates to draw the balls, and you want to vectorize it all, then we have to know size() of each variable involved. For example is size(phi) guaranteed to be the same as size(theta) ? Is phi a column vector and theta is a row vector? Are they scalars ?