Can I speed up this text file parsing code?
6 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I am reading in a large text files (~4MB) containing one column of of different readings. The part of my code that sorts the column of 5 reading types in to 5 different vectors is taking ~2 minutes to run. Is there a code change I can make to speed this up?
fid=fopen(fileName);
C = textscan(fid, '%f');
rawData=cell2mat(C);
toc
j=1;
k=1;
display('parsing data');tic
speed=[];voltage=[];current=[];force=[];displacement=[];
while j<=length(rawData)
displacement(k)=rawData(j);
force(k)=rawData(j+1);
current(k)=rawData(j+2) ;
voltage(k)=rawData(j+3);
speed(k)=rawData(j+4);
j=j+5;
k=k+1;
end
fclose(fid);
0 comentarios
Respuesta aceptada
Jan
el 20 de Jul. de 2011
Or try FSCANF instead:
Data = fscanf(fid, '%f', Inf);
fclose(fid);
len = numel(Data);
displacement = Data(1:5:len);
force = Data(2:5:len);
current = Data(3:5:len);
voltage = Data(4:5:len);
speed = Data(5:5:len);
Más respuestas (1)
Sean de Wolski
el 20 de Jul. de 2011
Preallocate, preallocate, preallocate!!!!!!! (just like m-lint (the orange squiggly line) is telling you)
v = 1:5:length(rawdata);
n = length(v);
force = zeros(n,1);
displacement = zeros(n,1);
current = zeros(n,1);
voltage = zeros(n,1);
speed = zeros(n,1);
for jj = v
displacement(k)=rawData(jj);
force(k)=rawData(jj+1);
current(k)=rawData(jj+2) ;
voltage(k)=rawData(jj+3);
speed(k)=rawData(jj+4);
k=k+1;
end
or even better just use reshape:
rawData = reshape(rawData,[],5)';
displacement = rawData(:,1);
force = rawData(:,2);
current = rawData(:,3);
voltage = rawData(:,4);
speed = rawData(:,5);
3 comentarios
Sean de Wolski
el 20 de Jul. de 2011
There's no reason for cells since all data is the same size. The reshape command will give you an nx5 matrix where each column is what you want...
Ver también
Categorías
Más información sobre Data Import and Export en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!