Compare and compile matching values from various data set.

Hi All, I have a problem where I need to compare various data set and only keep the values where the same matching values exist. Let me explain in a bit more detail.

Lets say I have multiple unique data sets:

data1.gamma = [1,2,3,4,5,6,7,8,9,10];
data1.eta   = [0.1,0.4,0.9,0,0.2,0.5,0.7,0.8,0.9,0.10];
data1.phi ........

data2.gamma = [0,2,0,4,5,9,7,10,9,6];
data2.eta   = [0.1,0.4,0.9,0,0.2,0.5,0.7,0.8,0.9,0.10];
data2.phi ........

and data3, data4 etc as such.

I would like to write a function that can to compare all the data sets for the values of

data.gamma

I would like to keep/store the values where the same values of data.gamma values exists in all data set(data1, data2, data3, data4...). I would also like to mention that the data sets (data1,data2,data3) might not be the same size.

I know and can compare and match two data sets. However I am not sure how to compare multiple data sets. Any idea how I would go about doing that?

Any help would be much appreciated.

Answer 1

[OCDER]

First, do not label variables dynmically like data1, data2, data3, etc.. as this will make your function very complicated. In your situation, using a nonscalar structure would be better such as data(1).gamma, data(2).gamma, data(3).gamma, etc.

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

Once you restructure your data, you could use intersect to determine common numbers:

data(1).gamma = [1,2,3,4,5,6,7,8,9,10];
data(1).eta   = [0.1,0.4,0.9,0,0.2,0.5,0.7,0.8,0.9,0.10];
data(2).gamma = [0,2,0,4,5,9,7,10,9,6];
data(2).eta   = [0.1,0.4,0.9,0,0.2,0.5,0.7,0.8,0.9,0.10];
data(3).gamma = [2,4,5,7,10];
data(3).eta   = [0.1,0.4,0.9,0,0.2,0.5,0.7,0.8,0.9,0.10];

SameGamma = data(k).gamma;
for k = 2:length(data)
    SameGamma = intersect(SameGamma, data(k).gamma);
end

SameGamma =
     2     4     5     7    10

[Abhi Ove]

Hi Donald many thanks for your advice. I receive the data as such, so can not necessarily change them ( as I have to return the data in that exact format)

Many thanks for helping me to identify identical values of gamma. I have to only compare 10 values rather than about 300 previously.

However I am still not sure about how to store the rest of the values (eta,phi,..) that corresponds to the same gamma value.

[OCDER]

Hi Abhi, here's how you would do it if you use structured data:

SameGamma = data(k).gamma;
for k = 2:length(data)
    SameGamma = intersect(SameGamma, data(k).gamma);
end

datafields = fieldnames(data);
for k = 1:length(data)
    [~, Idx, ~] = intersect(data(k).gamma, SameGamma);
    for f = 1:length(datafields)
       data(k).(datafields{f}) = data(k).(datafields{f})(Idx);
    end
end

If you want to uses data1, data2, data3,... data10, then you have to type it all out 10 times instead of using the for loop. OR, you need to use save/load workarounds:

save('temp.mat', '-regexp', 'data')
T = load('temp.mat')

* Do same calc as above, but:
    replace  :   data(k).gamma  
    with this:   T.(['data' num2str(k)]).gamma 

save('temp2.mat', 'T', '-struct'); 
load('temp2.mat') %Poof variables back out

Either way, it'll be cumbersome to work with dynamically named variables... Hope this helps.

[Abhi Ove]

Many thanks. I have taken your advice and made the whole process automated.

I understand the concern with individual data set that required somewhat human input during analysis. This was because I had to work with a set of data as I had received them. I found it easier to restructure the whole results data set using a separate script for better usability. Now everything flows a lot better.