Reading huge number of files quickly (I/O)
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
For analysis of EOD data for stocks, I need to read 3650 .csv files, but using a for loop takes around 30 seconds for 15 files, which would mean around 2 hours just to read data. Is there any way I can do it faster?
0 comentarios
Respuestas (1)
Walter Roberson
el 4 de Jun. de 2012
Are you currently reading using csvread(), or import(), or load() ?
If each file has exactly the same structure (same number of fields, some number of headers, same fields are character vs numeric) then you might be able to do better by using textscan()
If you are currently using xlsread() on MS Windows then you can almost certainly do better than you are doing now (but textscan() might be better still). xlsread() opens and closes Excel each time; Image Analyst has posted code several times showing how to open Excel just once and do several operations with it.
Ver también
Categorías
Más información sobre String Parsing en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!