How to perform Eigenvalue Decomposition e.g, eig() on multiple GPUs？

Since the matrix is very large，about 60kx60k herimitian matrix.

If do eig（）， it needs about 140G memory and exceeds a single GPU's memory, normally is 24G or 40G.

Answer 1

[Walter Roberson]

That is not possible.

The current status of support for splitting work across GPUs is described at https://www.mathworks.com/help/parallel-computing/run-matlab-functions-on-multiple-gpus.html#MultiGPUExample-4

The last time I saw staff mentioning this possibility, it was really only for some aspects of Deep Learning. The current documentation hints it might be a bit more flexible now.

I doubt that it will work for your purpose, but I do not know enough about it to be sure of that.

Answer 2

[Steven Lord]

Do you need all the eigenvalues and eigenvectors or do you just need a few of them? If the latter, consider using the eigs function instead of eig, perhaps (if your matrix has some structure that you can exploit) using the Afun syntax to avoid explicitly creating that huge matrix.

Answer 3

[Joss Knight]

eigendecomposition is a highly serial algorithm so that's why simple multi-process solutions aren't easy to find and why the GPU's performance against the CPU does not compare particularly well to other LAPACK algorithms.

However, MATLAB is a programming language with a rich parallel language, so anything that is possible is possible in MATLAB.

It's plausible that using a distributed array and forgetting about GPUs will be your best best for now.