In a logical array find how the density of TRUE values increases

Supposedly we have a logical array:

t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0];

The true values at the beginning are very few but the true values become more and more at the end of the array. Is there any way to identify the increasing density of the true values?

[Azzi Abdelmalek]

For this example, what are you expecting as result?

[Giorgos Papakonstantinou]

Good question! I am trying to find a criterion after which all false values will become true. And that criterion might be the density. So in the very specific example that I gave, after applying a criterion of density maybe t would become like this:

t=[0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1];

[Azzi Abdelmalek]

[Giorgos commented]

Thank all of you for your answers. I think that I will focus on the cumsum, I will check it a bit more and I will adopt it to my code. I will get back to you if I have any questions.

[Azzi Abdelmalek]

[Giorgos commented]

I still can not solve my actual problem. So I will post it here. I have a set of data in which fluctuate until a point after which the fluctuation fades out.

To be more specific you can see below my data plotted.

I have multiple datasets that are similar to the first plot of the figure. In the figure I have plotted only one case from my dataset. As you can see in the beginning of the plot the fluctuation is less and as we move to the right the fluctuation is significant. The magenta points are the extreme points. I have stored in a logical array the extreme values (the pink) indicated by TRUE and all the others (the blue) indicated by FALSE.

In every case I would like to spot the *point *after which the fluctuation becomes significant.

The method that Azzi provided in his last comment, to indicate the FALSE values in the logical array with -1 and then cumsum the array resulted in the second plot that you see.

The point that I want to pick lies in the local minima of the second plot. In the best case scenario if the magenta points are monotonously increasing then the point would be the global minimum of the second plot.

I am able to find the local minima of the second plot, but I am not able to find a rule on how to locate that point. To find the local minima I use the following function

function minima = findminima(x)
%FINDMINIMA  Find location of local minima
%  From David Sampson
%  See also FINDMAXIMA

minima = findmaxima(-x);

function maxima = findmaxima(x)
%FINDMAXIMA  Find location of local maxima
%  From David Sampson
%  See also FINDMINIMA

% Unwrap to vector
x = x(:);
% Identify whether signal is rising or falling
upordown = sign(diff(x));
% Find points where signal is rising before, falling after
maxflags = [upordown(1)<0; diff(upordown)<0; upordown(end)>0];
maxima   = find(maxflags);

Below I have zoomed in the right area of both plots, so you can visualize better the problem.

I would appreciate if take a look my problem. You can also download the data of both plots from this link

[Azzi Abdelmalek]

Giorgos, please, do not answer your question when you want to add a comment. Click on comment on this answer if you want to add a comment to any answer

[Giorgos Papakonstantinou]

Ok Azzi! Thank you!

Answer 1

[Image Analyst]

You have to define density over some area (number of elements) because, obviously, the density changes depending on how many elements it's calculated over. For example, to calculate the density over a window size of 5, try this code:

t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0];
density = conv(t, ones(1, 5)/5, 'valid')
plot(density, 'b-', 'LineWidth', 3);
grid on;
xlabel('Element', 'FontSize', 30);
ylabel('Density', 'FontSize', 30);

conv() is a function that will give the sum in the window and then divide by 5 so it gives the average value within the window of length 5.

[Giorgos Papakonstantinou]

Indeed as you mentioned it depends on the a specified window. Taking an arbitrary window is an option. However, I would like to extend my question and ask if there is any statistical rule, which will determine the window. And to broaden it even more, apart from the density that I mentioned, is there any other way to identify that the ones increase in the logical array?

As Azzi asked above? The question is what do I expect as a result? With the solution that you provided above I may introduce as a criterion a fixed density.

When density is larger than a fixed value, all the false values of the logical array will be converted to true and all the true values that are smaller than the fixed density are going to be converted to false.

[Image Analyst]

You can do that if you want.

% densityThreshold is some value you define.
aboveDensityThreshold = density > densityThreshold;  
% Assign original data to false in those locations.
t(aboveDensityThreshold) = false;

[Giorgos Papakonstantinou]

Thank you Image Analyst for your suggestion but I haven't solved yet my original problem neither with the "density" nor with the cumsum. I posted in the beginning of this thread my original question.

I would appreciate if you could take a look at it. And if there is something unclear I would be happy to explain it further.

Answer 2

[Azzi Abdelmalek]

What you did in your previous comment is

sort(t)

I do not think this is what you want. Can you explain mathematically what is density?

[Giorgos Papakonstantinou]

Certainly sort is not what I want. I just gave an example of how I want to be the final form of the logical array.

In the form that I gave the logical array the density of the TRUE values would be the ratio of the sum of TRUE values over the sum of TRUE and FALSE values. This I know how to define. The "window" as Image Analyst mentioned above is something that I do not know how to define.

So I extend my question and I ask if your opinion there is a mathematical/statistical rule which will help me to find the increasing occurrence of TRUE values in the logical array.

[Image Analyst]

It does not make sense to compute density when the window over which the density is defined changes from location to location. Maybe you just want to use cumsum() and plot that.

[Azzi Abdelmalek]

Edit

t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 ];
s=num2str(t)
s1=strrep(s,' ','')
n1=cellfun(@numel,regexp(s1,'1+','match'))
n2=cellfun(@numel,regexp(s1,'0+','match'))
stairs(cumsum(n1))
hold all
stairs(cumsum(n2))

[Azzi Abdelmalek]

Look at edit

Answer 3

[Azzi Abdelmalek]

%or maybe

t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 ];
s=num2str(t)
s1=strrep(s,' ','')
n1=cellfun(@numel,regexp(s1,'1+','match'))
n2=-cellfun(@numel,regexp(s1,'0+','match'))
out=zeros(1,numel([n1 n2]))
out(1:2:2*numel(n1)-1)=n1
out(2:2:2*numel(n2))=n2
stairs(cumsum(out))

%or maybe

close
t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 ];
t(t==0)=-1
stairs(cumsum(t))

[Giorgos Papakonstantinou]

I like the last one a lot! Seems very tidy and fast.

Answer 4

[Spencer Talbot]

I know this question is old, but I recently had a similar question. Here's what I came up with:

    t=[1 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0];
    density = zeros(length(t),2); % initialize the density matrix (1st column is # of consecutive TRUE instances, 2nd column is the index where that streak occurs)
    k = 1; % initialize the index for keeping track of density

    for i = 1:length(t)
        if t(i)
            density(k) = density(k,1) + 1; % if the current value is true, plus one count for density
            if density(k,1) == 1
                density(k,2) = i; % If it's the start of a new streak, record the index
            end
        else
            k = k + 1; % if a streak ends, move on to the next index to prepare for a new streak
        end
    end
    
    [max_density, idx] = max(density(:,1)); % find the value and starting index for the longest streak
    range_idx = density(idx,2):density(idx,2) + max_density - 1; % compute the range of indices for which the max density streak lasts