Extracting Data from Tall Arrays using Logical Indexing

I have a dataset made up of tall arrays [X Y Z]. Ultimately, I would like to create some kind of mask or sorting function to extract specific regions of interest. The issue I am running into is how to create a filter using tall arrays and more spcifically, how to do so without using for loops.

For example, I would like to know all of the [X, Y, Z] data points that reside within a cylinder of radius (R) angled 30 degrees from the center (0,0,0). All other values can be removed.

The dataset currently resides in a datastore, to assist with this analysis, I imported a small amount of data for processing and converted [X Y Z] to spherical coordinates [azimuth, elevation, r].

I believe the correct approach is to use Logical Indexing for the three conditions [azimuth, elevation, r]. If someone could help me with the initial set-up of these conditions, i would greatly appreciate it.

Answer 1

[Benjamin Großmann]

I think you mentioned everything that you need for this task. Try this as a starting point:

clearvars, close all
clc

% some random data, shall be [x,y,z]
data = rand(100,3);

% transform to cylindrical coordinates
[theta,rho,z] = cart2pol(data(:,1), data(:,2), data(:,3));

% define the mask(s)
rho_min = 0;
rho_max = 0.5;
rho_mask = (rho >= rho_min) & (rho <= rho_max);

theta_min = 0;
theta_max = pi/6;
theta_mask = (theta >= theta_min) & (theta <= theta_max);

z_min = -inf;
z_max = inf;
z_mask = (z >= z_min) & (z <= z_max);

% combine the masks
mask = rho_mask & theta_mask & z_mask;

% Apply the mask
data_filtered = data(mask,:);

Edit: Corrected the theta_mask and added a z_mask, which is inactive via the infinite borders. Code is optimised for clearness, i.e. the z mask is superfluous. Let me know if you have performance issues or need further help.

[Benjamin Großmann]

You can use gather() on data_filtered to get the tall array into memory and test this by changing "data=rand(100,3);" to "data=tall(rand(100,3));".