Interpreting eigenvalues and eigenvectors when using symbolic toolbox

When using the symbolic math toolbox (symbolic "d" in this case) I sometimes end up with results (using different stochastic matrices) where I get more eigenvalues returned than eigenvectors which does not make sense to me. Might it be that since the eigenvalues are returned in the same dimension as the input matrix the first eigenvalue in this case 0 is to be ignored. I.e we get three eigenvalues (1, -(d*(3 + 15^(1/2)*1i))/6, (d*(- 3 + 15^(1/2)*1i))/6) corresponding to the three eigenvectors in the example provided below?

Thanks in advance for any input!

Btw I did notice that if I change the symbolic 'd' to a scalar e.g d= 0.5 I get 4 eigenvectors and 4 eigenvalues.

function [eig_mat,eig_val,M] = pagerank_function(linkMatrix,d)
    n = size(linkMatrix,1)
    M = times(d, linkMatrix) + times((1-d)/n , ones(n))
    % diagonal matrix eigenvalues D, eigenvectors mtx U
    [U,D] = eig((M))
    extract_ev = diag(D)
    %x = round(U, 2);
    eig_mat = simplify(U)
    eig_val = simplify(extract_ev)
end

Where I input the following matrix :

D125 = [0,1/3,1/3,1/3;   
      0,0,1,0;
      1,0,0,0;
      0,0,1,0];

And run the function using ;

            
d = sym('d');      

[eig_mat,eig_val,M] = pagerank_function(D125,d);

% Latex code
latex_evtable = latex(sym([eig_mat]))
latex_etable = latex(sym([eig_val]))
latex_matrix = latex(sym(M))
Mtx_maker = latex(sym(D125))

Answer 1

[Christine Tobler]

I'm getting both 4 eigenvalues and 4 eigenvectors when running your code:

linkMatrix = [0,1/3,1/3,1/3;   
      0,0,1,0;
      1,0,0,0;
      0,0,1,0];

d = sym('d');  

n = size(linkMatrix,1)

n = 4

M = times(d, linkMatrix) + times((1-d)/n , ones(n))

M = 

% diagonal matrix eigenvalues D, eigenvectors mtx U
[U,D] = eig((M))

U = 

D = 

extract_ev = diag(D)

extract_ev = 

%x = round(U, 2);
eig_mat = simplify(U)

eig_mat = 

eig_val = simplify(extract_ev)

eig_val = 

Can you describe more of what outputs you are seeing?

[Fredrik Scheie]

Thanks for your answer. On second running I did get 4 eigenvalues, 4 eigenvectors as well when running the function. However, when I tried the following stochastic matrix the output gave 4 eigenvalues and 3 eigenvectors.

D102 = [0,1/2,0,1/2;   
      0,0,1,0;
      0,1,0,0;
      0,1,0,0];

Which gives the following eigenvalues, eigenvector pair converted to latex.

Might it be that the algebraic multiplicity is equal to 2 in this case for the characterisitc equation such that zero appears as a solution twice? Then the first eigenvector represents the repeated eigenvalue of zero. Notice that removing the simplify expression from the function and setting 'd' to a scalar e.g 0.7 does return 4 eigenvalues and 4 eigenvectors for the same matrix. Guess I am wondering what it is that makes the symbolic toolbox representation choose 3 instead of 4 eigenvectors?

Any help would be most welcome.

[Christine Tobler]

As you assumed, the difference is between symbolic computation and numerical computation. Let's look at a simple matrix with two eigenvalues that both have algebraic multiplicity 2:

A = blkdiag([2 1; 0 2], 3, 3)

A = 4×4

2 1 0 0 0 2 0 0 0 0 3 0 0 0 0 3

The geometric multiplicity of the two eigenvalues is different though - eigenvalue 2 has geometric multiplicity 1, while eigenvalue 3 has geometric multiplicity 2. We can see this numerically by computing null(A - lambda*I)

null(A - 2*eye(4))

ans = 4×1

-1 0 0 0

null(A - 3*eye(4))

ans = 4×2

0 0 0 0 0 1 1 0

The symbolic toolbox only returns these 3 eigenvectors, and returns the eigenvalues based on algebraic multiplicities, so the two arrays can have different sizes. To show which eigenvalue maps to which set of eigenvectors, a third output gives this mapping:

[U, D, p] = eig(sym(A, 'f')) % Turn A into a symbolic variable (no unknowns)

U = 

D = 

p = 1×3

1 3 4

d = diag(D);
d(p) % this vector specifies which eigenvalue is represented in each column of U

ans = 

When we use non-symbolic A, however, eig uses numeric computation, which includes round-off. The concept of a matrix that is not diagonalizable (where some geometric multiplicity is smaller than its equivalent algebraic multiplicity), there is always a tiny change in that matrix which will make it diagonalizable. For example:

Amod = A;
Amod(2, 1) = eps;
[U, D, p] = eig(sym(Amod, 'f'))

U = 

D = 

p = 1×4

1 2 3 4

Therefore in numerical computation, every matrix is treated as diagonalizable, and we instead talk about the eigenvalue condition number to say how "close to not diagonalizable" the matrix is - analogous to how the regular condition number tells us how "close to singular" the matrix is:

[U, D] = eig(A)

U = 4×4

1.0000 -1.0000 0 0 0 0.0000 0 0 0 0 1.0000 0 0 0 0 1.0000

D = 4×4

2 0 0 0 0 2 0 0 0 0 3 0 0 0 0 3

condeig(A)

ans = 4×1

1.0e+15 * 2.2518 2.2518 0.0000 0.0000

You can also see that the first two columns of U are essentially the same, meaning that U is very close to singular, and inverting it would be a bad idea.

[Fredrik Scheie]

I am not sure I understand the part about numerical computations where all matrices are treated as diagonizable. Is it the round off in numerical computation which can round off essentially two very similar eigenvalues and produce in the case of your A matrix 4 eigenvectors instead of the 3 produced using the symbolic calculations which calculates the eigenvectors using the null space?

[Christine Tobler]

Yes, that's the gist of it. Basically, numerical computation of EIG is based on a series of orthogonal similarity transformations applied to matrix A (Anew = Q*A*Q'). The transformations being orthogonal is important, since it means the amount of round-off error is kept minimal compared to doing Anew = X*A*inv(X) with a non-orthogonal matrix. But while the round-off is minimal, it still exists and means that we can't meaningfully tell the difference between two exactly identical eigenvalues and two eigenvalues that are just very close to each other.

If you're interested in more details, you might take a look at the Schur decomposition, which is an intermediate step in numerically computing the eigenvalue decomposition of a matrix. It decomposes A = U*T*U' where T is upper triangular (if you use the 'complex' flag) and U is orthogonal. The eigenvectors are then computed from T in a second step.