Asked by Andreas Nagl
on 5 Jul 2017

Hello,

I need to group data (peaks for many signals, below) into bins with irregular sizes:

[pks,locs,w,p] = findpeaks(data)

I know discretize(), but I don't have equal-sized bins. The goal: Find peaks in series of measurements and track shifts of peak positions (therefore the peak positions +/- tolerance give my bins). This means findpeaks(), peaks = bins +/-, findpeaks() in next dataset, put them into the bins. Then the whole process again with the latest dataset giving the next bins. I am dealing with lots of peaks (~ 100/measurement and 60.000 altogether).

So what I want in the end:

Peak 1

Pos 7.2 Height 5 Width 3

Pos 7.3 Height 5.3 Width 2.8

... ... ...

Peak 2

... ... ...

Coming from the C# world, where this would be solved with lists and loops I appreciate every hint, including keywords for further research (also which data structures to use).

Thanks!

Answer by John BG
on 6 Jul 2017

Edited by John BG
on 6 Jul 2017

Accepted Answer

Hi Andreas

While the common output of findpeaks would be something like the starter example

x = linspace(0,1,1000);

Pos = [1 2 3 5 7 8]/10;

Hgt = [4 4 4 2 2 3];

Wdt = [2 6 3 3 4 6]/100;

for n = 1:length(Pos)

Gauss(n,:) = Hgt(n)*exp(-((x - Pos(n))/Wdt(n)).^2);

end

PeakSig = sum(Gauss);

% plot(x,Gauss,'--',x,PeakSig)

your list of peaks has repeated numerals for groups of peaks, something like

Pos1=[1 1.2 1.7]/10

Hgt1=[3.9 4 4.1]

Wdt1=[2 2 2.1]/100

Pos2=[2]/10

Hgt2=4

Wdt2=6/100

Pos3=[2.9 3.01 3.02 3.25]/10

Hgt3=[4.1 4.2 3.8 4]

Wdt3=[3 3.1 3.2 2.9]/100

Pos4=[5 5.2]/10

Hgt4=[2 2.01]

Wdt4=[3 3.1]/100

Pos5=[7 6.8]/10

Hgt5=[1.9 2]

Wdt5=[3.9 4]/100

Pos6=[8]/10

Hgt6=3

Wdt6=6/100

according to your question the data related to peaks with same numerals is grouped into cells

locs={Pos1 Pos2 Pos3 Pos4 Pos5}

w={Wdt1 Wdt2 Wdt3 Wdt4 Wdt5}

p={Hgt1 Hgt2 Hgt3 Hgt4 Hgt5}

but while your data has sets of variable lengths for each locs{k} w{k} p{k}

findpeaks does not repeat peak numerals for peaks located on different locations.

Your variable pks for this example would be

pks=[1:1:5]

With such data format, to generate the table you ask for, the peak numerals have to repeat accordingly

L=[]

for k=1:1:size(locs,2)

L=[L repmat(k,1,numel(locs{k}))];

end

L

=

1 1 1 2 3 3 3 3 4 4 5 5

And to read the values in the table the contents of the cells has to change type

locs2=cell2mat(locs)

w2=cell2mat(w)

p2=cell2mat(p)

Once the data has been prepared, obtaining the table

T = table(L', locs2', p2', w2','VariableNames',{'PeakNo' 'Positions' 'Heights' 'Widths'})

T =

12×4 table

PeakNo Positions Heights Widths

______ _________ _______ ______

1 0.1 3.9 0.02

1 0.12 4 0.02

1 0.17 4.1 0.021

2 0.2 4 0.06

3 0.29 4.1 0.03

3 0.301 4.2 0.031

3 0.302 3.8 0.032

3 0.325 4 0.029

4 0.5 2 0.03

4 0.52 2.01 0.031

5 0.7 1.9 0.039

5 0.68 2 0.04

if you find this answer useful would you please be so kind to consider marking my answer as Accepted Answer?

To any other reader, if you find this answer useful please consider clicking on the thumbs-up vote link

thanks in advance

John BG

Andreas Nagl
on 6 Jul 2017

Thank you very much for your answer.

Could you specify what you mean by the term numeral? Especially in connection with the findpeaks() method?

John BG
on 23 Jul 2017

Hi Andreas

In your question one can read the format of your data

Peak 1

Pos 7.2 Height 5 Width 3

Pos 7.3 Height 5.3 Width 2.8

... ... ...

Peak 2

... ... ...

note that each peak may have a variable amount of positions.

Yet, the function find understands each 'position' as a different peak.

For such reason, in my answer I assign numerals that repeat accordingly.

The numerals are the matrix indices.

I have noticed that my answer was initially accepted, probably by you, and now it's not accepted.

If you find my answer to solve your question, would you be so kind to mark again it as the accepted answer?

Thanks in advance.

Sign in to comment.

Answer by Star Strider
on 5 Jul 2017

Edited by Star Strider
on 5 Jul 2017

Since you have a fixed number of peaks in your data (I assume they are always in the same order), I would store the peaks and other data in a cell array (or matrix) in a loop, for example:

for k1 = 1:N

[pks{k1},locs{k1},w{k1},p{k1}] = findpeaks(data{k1});

end

where ‘N’ are the number of data vectors you have.

If you want to use discretize (or possibly histcounts), note that the bins do not have to be equal sizes. You can specify the edges of the bins in a vector in both functions, so the bins can be different widths.

EDIT —

The loop I posted will allow you to get all the information you described in your edit that you want.

For information on how to use cell arrays such as I use in my code example, see the documentation on Cell Arrays (link).

Star Strider
on 11 Jul 2017

I have no idea what you are measuring. It looks to me that you have a signal (somewhere) corrupted by noise. I would do a Fourier transform of your data (use the fft (link) function) to see if there is underlying periodicity in your data, and if at least some of your noise is band-limited. If there is, you can design a filter to recover it, and depending on the frequency characteristics of the noise, eliminate most of the noise.

It is common to have noise in measured data, usually requiring at least some pre-processing to recover the underlying signal.

Andreas Nagl
on 11 Jul 2017

I don't mean to be rude, but I ask for tracking peaks for a reason.

And there is a reason why I am not asking about pre-processing, noise and fft. That is all already done. So unless you're an expert in crystallography I'd appreciate it if we could stick to my question/problem instead of redefining my problem.

Star Strider
on 11 Jul 2017

I have no idea what your peaks represent. I have no experience with crystallography.

If the peaks simply shift in amplitude but are always at the same positions with respect to your independent variable, then you can concatenate them in a matrix. That would allow you to track their amplitudes between experiments, and plot them.

If the peak positions are never stable, so that the peaks shift in amplitude and position of your independent variable, I know of no reliable way to track them between experiments. Perhaps summing them over the same ranges of your independent variable in each experiment would then be appropriate. You could do that with the reshape function, if the size of your vectors and the ranges of the independent variable match the requirements reshape imposes.

Perhaps posting to a crystallography forum would provide you with participants with the necessary expertise and experience to give you the information you need.

Sign in to comment.

Answer by Jan
on 24 Jul 2017

Edited by Jan
on 24 Jul 2017

Omit the details about cristallography or if the numbers are peaks, because for Matlat they are just numbers.

You have 60'000 sets of 100 numbers and want to find the clusters in it: The numbers which are near together. The data sets need not contain all numbers. You want the position and width of the clusters. Correct? If so, how is "width" defined? Standard deviation or maximum range? Can the clusters overlap -- or in your terminology: can the positions of the peaks vary such, that they could be seen as the one or the other final peak? Then the order of numbers might matter.

This might be a job for kmeans. Join all numbers (peak positions) to a vector and determine the 100 clusters. But perhaps there are not exactly 100 clusters. Then this answer is not the solution, but perhaps it helps you to get in the right direction. Or I misunderstand your question.

Star Strider
on 24 Jul 2017

Sign in to comment.

Answer by Andreas Nagl
on 28 Jul 2017

Edited by Andreas Nagl
on 28 Jul 2017

Dear all,

thank you very much for your contributions. What I ended up doing was a cell array of tables, one table for each peak. So by looping through the peaks I sorted them into the respective tables.

For a big number of peaks this can get rather slow (as I always look up the last position of a peak in it's table to account for shifts), but for me it's good enough.

So it's quite similar to John BG's solution (Thanks again!). I hope this helps!

However, I'd like to highlight Jan Simon's answer: Even though as it's not what I wanted initially, I will do that. It's a great idea, thanks!

Best, Andreas

Sign in to comment.

Opportunities for recent engineering grads.

Apply Today
## 2 Comments

## Image Analyst (view profile)

## Direct link to this comment

https://la.mathworks.com/matlabcentral/answers/347531-group-data-into-bins-with-irregular-sizes-track-peaks#comment_467089

## Andreas Nagl (view profile)

## Direct link to this comment

https://la.mathworks.com/matlabcentral/answers/347531-group-data-into-bins-with-irregular-sizes-track-peaks#comment_468185

Sign in to comment.