Identify two linear trends from the data

Hello, I have the following data as an example which is attached in this question. I would like to obtained two linear trends and estimate the break point between the two curves like the image below (which is one example I have already obtained). Both lines should represent the best fit with lower residuals. I will have to automatized the procedure based on different conditions but this example would be just one case.

load data_example.mat

x=data_example(:,1) % x data is already in log values

y=data_example(:,2) % y data is already in log values

I would appreciate the help.

Answer 1

[John D'Errico]

The amount of noise in your data makes it almost impossible to estimate the two trends in any intelligent way, even assuming there are indeed two distinct trends, and almost impossible to identify a breakpoint at all well.

But you can try some simple things. For example, a simple pair of weighted regressions might work.

load data_example.mat
x = data_example(:,1);
y = data_example(:,2);
xscale = 2*(x - min(x))/(max(x) - min(x)) - 1;
w1 = exp(-2*xscale);
w2 = exp(2*xscale);

P1 = fit(x,y,'poly1','weights',w1);
P2 = fit(x,y,'poly1','weights',w2);

plot(P1)
hold on
plot(P2)
plot(x,y,'.')

As you can see, we do get two distinct lines out. The problem is, your data is just so noisy. And the result will be strongly influenced by how I construct the weight vectors.

I can do a little better by use of my SLM toolbox. It is available for free download from the File Exchange. you will need the optimization toolbox though.

slm = slmengine(x,y,knots = 3,plot = 'on',degree = 1,interiorknots = 'free');
slm.knots
ans =
       1.3863
       1.9257
       2.1748

The break point lives at x=1.9257. You can see the two lines below:

Again though, the extent of noise in your data makes any such determination highly problematic.