create multiple tables from a CSV in a for loop

hello everyone,
I have a large CSV file (too large to upload ~ 50 GB) that I stored in a table. the table columns are time,energy.
I'd like to save segments from this table into separate tables. for that I need the new table name to be changed in some way, lets say:
T=1000;
for i = 1:6
segment = find(old_table.time<T);
new_table(i) = old_table(segment,:); %note
T = T+1000;
end
the row with the note is what I'm trying to do but not sure how.

5 comentarios

Stephen23
Stephen23 el 14 de Sept. de 2021
Editada: Stephen23 el 14 de Sept. de 2021
"for that I need the new table name to be changed in some way"
No, you don't need to change the table name.
Changing the table name would be a slow, more complex, and very inefficient approach.
The neat, simple, and very efficient solution would be to use basic MATLAB indexing and a cell array.
"the row with the note is what I'm trying to do but not sure how."
As an alternative you could just use one tall array: https://www.mathworks.com/help///matlab/large-files-and-big-data.html
sani
sani el 14 de Sept. de 2021
thanks for your answer!
the problam is that when I'm trying to write it into a predetermined call array if fails since not all segments contain the same number of rows. I predetermined the row in the array to be 1E7 (which is more than the max segment I'm expecting).
the error is "unable to perform the assingment because the size of the left side in 1e7-by-2 and the right side is 56468-by-2"
I don't think you need to predetermine the segment size to store the table in cell array. You can just edit your code as follows.
new_table{i} = old_table(segment,:);
left_side(1:size(right_side,1), 1:size(right_side,2)) = right_side;
sani
sani el 19 de Sept. de 2021
Thank you all, it solved my problam!

Iniciar sesión para comentar.

Respuestas (0)

Categorías

Más información sobre Matrices and Arrays en Centro de ayuda y File Exchange.

Preguntada:

el 14 de Sept. de 2021

Comentada:

el 19 de Sept. de 2021

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by