Tall Array Data Fetch Optimization

2 visualizaciones (últimos 30 días)
Buddhi Wimarshana
Buddhi Wimarshana el 31 de En. de 2021
Hi everyone,
I am working on a Tall Data array backed by a Data Store, the array looks like follows;
My question is, when I try to fetch/mine data using the Data_Point as in the following (tdsDB is the Tall Array and dataPointIndxFirst and dataPointIndxLast parameters are set accordingly):
app.xxyydata = gather(app.tdsDB(app.tdsDB.Data_Point>=dataPointIndxFirst &...
app.tdsDB.Data_Point<=dataPointIndxLast,:));
but this takes several minutes to compute, but Data_Point is unique indexing value which increase along the data rows. And the Data_Point values start from 97137237, not from 1.
Compute time is around 5 mins even the parameters are set like: dataPointIndxFirst = 97137237 and dataPointIndxLast = 97137238 (very next point of dataPointIndxFirst).
My guess is MatLab searches through the whole Tall Array, which is not the optimal way. Is there a way that I can tell MatLab this Data_Point is arranged in an increasing order and acts like (or can be treated as) an index?
Any help to optimize compute time is appreciated....many thanks in advance.

Respuestas (0)

Categorías

Más información sobre Tall Arrays en Help Center y File Exchange.

Productos


Versión

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by