How can I see if data fit a specified curve?

I have expression data, some are correlated, but some follow a pattern of y = 1/x (only the positive values of x and y).

How do I rapidly tell if the data fit y = 1/x (I have a few thousand pairs of data to try, hence not wanting to plot each one individually). Is there an easy way of doing this with curve fitting functions?

[Rik]

Do you want to see if single pairs follow this function, or if groups of pairs follow this function?

[Rachael Thompson]

I want to see if single pairs of genes (with expression measured in ~1000 samples) follow the function.

[Adam Danz]

When you say that some are correlated, I'm not sure I understand what that means. Do you mean for some data, x approximately equals y? A screen shot of your plots may be helpful.

[Rachael Thompson]

Yes! So some y=x and some y=1/x and some are just random clouds.

The y=x are easy to find with the spearman's correlation coefficient.

The ones where y=1/x are the ones I want to find.

Answer 1

[Adam Danz]

There are several ways to classify your data into the three groups: 1) random, 2) y=x, and 3) y=1/x. I tend to favor low-level approaches whenever possible. The demo below creates a dataset that fits your description and then classifies each data point according to those three categories. It does so by computing the error between each coordinate and the two functions y=x and y=1/x. If the error is above a set (subjective) threshold for both functions, the coordinate is classified as random. Otherwise, it is classified according to the function with the smallest error. If your data contain a considerable amount of noise, first trying increasing the errThreshold. If that doesn't help, we may need to use a more sophistocated algorithm.

% Create x-data
% the +.2 is to avoid values near 0 which cause problems with plotting 1/x
x = rand(1,1000)*5 + .2;  

% Create 3 groups of y-data
%1) y = x
%2) y = 1/x
%3) random
randXIdx = randperm(numel(x)); 
sections = floor(numel(x).*[.33, .66]); 
y = rand(size(x))*5 + .2;                                     % random y values
y(randXIdx(1:sections(1)-1)) = x(randXIdx(1:sections(1)-1));  %y = x
y(randXIdx(sections(1):sections(2))) = 1./x(randXIdx(sections(1):sections(2)));  %y = 1/x

% Look at data
clf()
plot(x,y, 'o')

% Compute the error between the two function y=x and y=1/x.
err1 = abs(y-x); 
err2 = abs(y - 1./x); 

% Choose a threshold.  Plotting the error may be helpful. 
% Error greater than threshold will be assigned to random class
%     clf()
%     plot(err1,'ro')
%     hold on
%     plot(err2, 'bx')
errThreshold = 0.05;    % my subjective judgement

% Classify the coordinates based on minimum error.
% If the error for both is beyond threshold, classify as random. 
group = zeros(size(y)); 
group(err1 >= errThreshold & err2 >= errThreshold) = 1; % group 1 is random group
group(err1 < errThreshold) = 2; % group 2 is y=x group
group(err2 < errThreshold) = 3; % group 3 is y=1/x groups (and y=x if the point belongs to both groups)

% Check that all points are assigned to a group
if any(group==0)
    error('Point not assigned to group.')
end

% Plot results
clf()
plot(x,y, 'ko')
hold on
plot(x(group==1),y(group==1), 'r.', 'DisplayName', 'rand')
plot(x(group==2),y(group==2), 'b.', 'DisplayName', 'y=x')
plot(x(group==3),y(group==3), 'g.', 'DisplayName', 'y=1/x')
legend()

Results of classification:

[Rik]

Neat solution, +1. Of course it would be possible to design more sofisticated measures than just the absolute distance, but this code design makes it easy to insert the method of choice.

[Adam Danz]

Thanks, Rik. With noisier data, a different approach may be better. This one only considers vertical error which may or may not be suitable. It would be helpful to see the real data.

I'm wondering how helpful the Classification Learner App would be. I haven't used it yet but have been meaning to dig into that.

[Rachael Thompson]

Thank you for your help!!

I altered the err2 calculation a little as relationships didn't exactly fit y = 1/x that I wanted to find (they were almost following the x and y axis in a right angle shape), but plotting out the data with the final graph you created meant I could gradually alter the equation to something that fit!

[Adam Danz]

Great job getting it to work on your data! Thanks for the feedback.

[Rachael Thompson]

Thank you for your help and all the explanation in the code so that I could understand and alter it!