Tall Array Data Fetch Optimization
2 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Hi everyone,
I am working on a Tall Data array backed by a Data Store, the array looks like follows;
My question is, when I try to fetch/mine data using the Data_Point as in the following (tdsDB is the Tall Array and dataPointIndxFirst and dataPointIndxLast parameters are set accordingly):
app.xxyydata = gather(app.tdsDB(app.tdsDB.Data_Point>=dataPointIndxFirst &...
app.tdsDB.Data_Point<=dataPointIndxLast,:));
but this takes several minutes to compute, but Data_Point is unique indexing value which increase along the data rows. And the Data_Point values start from 97137237, not from 1.
Compute time is around 5 mins even the parameters are set like: dataPointIndxFirst = 97137237 and dataPointIndxLast = 97137238 (very next point of dataPointIndxFirst).
My guess is MatLab searches through the whole Tall Array, which is not the optimal way. Is there a way that I can tell MatLab this Data_Point is arranged in an increasing order and acts like (or can be treated as) an index?
Any help to optimize compute time is appreciated....many thanks in advance.
0 comentarios
Respuestas (0)
Ver también
Categorías
Más información sobre Tall Arrays en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!