Sparse Block diagonal matrix

I have an N by M matrix and I want to store its columns in block diagonal form. This matrix is enormous, so if I can directly store it as a sparse matrix that would be helpful.

To give more detail:

X = ones(30, 1000);

And I want this without having to type it out (because the matrix actually has 1 million columns).

sparse(blkdiag(X(:,1), X(:,2),  X(:,3), .... ,  X(:,1000)).

By the way, my best attempt is this:

a = cell(M,1);
for i = 1:M
    [a{i}]= deal(sparse(X(:,i)));
end
X=blkdiag(a{:});

But I'm using a loop and am worried this is inefficient.

Answer 1

[Oleg Komarov]

The easy way:

X = num2cell(X,1);
sparse(blkdiag(X{:}))

Or, creating the sparse matrix directly:

[r,c] = size(X);
i     = 1:numel(X);
j     = repmat(1:c,r,1);
B     = sparse(i',j(:),X(:));

The second method is not only ~30-40 times faster but avoids creating the intermediate blkdiag() double matrix, which can easily cause out of memory.

Honestly, I don't know if you have that kind of memory:

(30 * 1e6) rows * 1e6 cols * 8 bytes ~ 218.2 TB

Therefore, go for the second method.

Answer 2

[Alec]

Or, if you can, call `sparse` before `blkdiag`:

X = ones(30, 1000); 
X = num2cell(sparse(X),1);
B = blkdiag(X{:});

Answer 3

[Daniel Fortunato]

You can also use MATLAB's internal sparse block-diagonal routine, which will create a sparse block-diagonal matrix even from dense blocks:

B = matlab.internal.math.blkdiag(X{:});