You have ONE equation, not a system of ODEs.
(a * dy/dt) + (b * y) = (c * dx/dt) + (d * x)
But suppose you knew the true solution for those coefficients, estimated as [a, b, c, d], as the "optimal" set of coefficients. Can you know then that [2*a, 2*b, 2*c, 2*d] is not just as good of a solution? In fact, pick any constant k, and we can see that the set [k*a, k*b, k*c, k*d] is a solution as good as any other. k can even be zero.
That tells you that your problem is not well posed. You can choose ANY of those 4 coefficients, and arbitrarily set it to 1, or to any constant. I'd pick a==1, which is as good as any other. So your problem now reduces to something like this:
dy/dt + b*y = c*dx/dt + d*x
Next, although we don't have your data in hand to show how it might work, you have data of the form of a list of values for t, as well as x and y at each point in time.
So just interpolate x(t) and y(t) as splines. Then use them to infer the values of b, c, and d. Assuming t,x, and y are vectors, it would look like this:
xoft = spline(t,x);
xdot = fnder(xoft)
yoft = spline(t,y);
ydot = fnder(xoft)
A = [-y(:),fnval(xdot,t(:)),x(:)];
bcd = A\fnval(ydot,t(:));
bcd will be a vector of length 3, contining the values of b, c, and d. Remember that a was 1.
fnval and fnder should be part of the curve fitting toolbox as I recall.
