How to apply PCA correctly?
189 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Sepp
el 12 de Dic. de 2015
Comentada: the cyclist
el 25 de Jun. de 2024
Hello
I'm currently struggling with PCA and Matlab. Let's say we have a data matrix X and a response y (classification task). X consists of 12 rows and 4 columns. The rows are the data points, the columns are the predictors (features).
Now, I can do PCA with the following command:
[coeff, score] = pca(X);
As I understood from the matlab documentation, coeff contains the loadings and score contains the principal components in the columns. That mean first column of score contains the first principal component (associated with the highest variance) and the first column of coeff contains the loadings for the first principal component.
Is this correct?
But if this is correct, why is then X * coeff not equal to score?
1 comentario
Respuesta aceptada
the cyclist
el 12 de Dic. de 2015
Editada: the cyclist
el 25 de Jun. de 2024
==============================================================================
EDIT: I recommend looking at my answer to this other question for a more detailed discussion of topics mentioned here.
==============================================================================
Maybe this script will help.
rng 'default'
M = 7; % Number of observations
N = 5; % Number of variables observed
X = rand(M,N);
% De-mean
X = bsxfun(@minus,X,mean(X));
% Do the PCA
[coeff,score,latent] = pca(X);
% Calculate eigenvalues and eigenvectors of the covariance matrix
covarianceMatrix = cov(X);
[V,D] = eig(covarianceMatrix);
% "coeff" are the principal component vectors.
% These are the eigenvectors of the covariance matrix.
% Compare the columns of coeff and V.
% (Note that the columns are not necessarily in the same *order*,
% and they might be slightly different from each other
% due to floating-point error.)
coeff
V
% Multiply the original data by the principal component vectors
% to get the projections of the original data on the
% principal component vector space. This is also the output "score".
% Compare ...
dataInPrincipalComponentSpace = X*coeff
score
% The columns of X*coeff are orthogonal to each other. This is shown with ...
corrcoef(dataInPrincipalComponentSpace)
% The variances of these vectors are the eigenvalues of the covariance matrix, and are also the output "latent". Compare
% these three outputs
var(dataInPrincipalComponentSpace)'
latent
sort(diag(D),'descend')
17 comentarios
evelyn
el 21 de Jun. de 2024
Movida: the cyclist
el 25 de Jun. de 2024
appreciate the explanation.
Has anyone try when data is complex-value number.
When I generate complex-value number data, the result is equite different when using 'pca function' directly and using eigenvectors of covariance matrix. Could anyone help me! code is shown below.
M = 13; % Number of observations
N = 4; % Number of variables observed
PCA_component_Num = 4;
data_matrix = rand(M,N) + 1i * randn(M,N);
X = data_matrix;
% De-mean
centredX = bsxfun(@minus,X,mean(X))
%% PCA using Matlab function directly
[coeff,score,latent] = pca(centredX);
% latent is eigenvalues of the covariance matrix of X.
% "coeff" are the principal component vectors.
dataInPrincipalComponentSpace = centredX * coeff;
% Multiply the original data by the principal component vectors
% to get the projections of the original data on the
% principal component vector space. This is also the output "score".
% Compare ...
dataInPrincipalComponentSpace
score
% The columns of X*coeff are orthogonal to each other. This is shown with ...
corrcoef(dataInPrincipalComponentSpace)
%% eigenvector
C = cov(centredX);
[W, Lambda] = eig(C);
% Compare the columns of coeff and V.
% (Note that the columns are not necessarily in the same *order*,
% and they might be *lightly different from each other
coeff
W
% The variances of these vectors are the eigenvalues of the covariance matrix, and are also the output "latent". Compare
% these three outputs
var(dataInPrincipalComponentSpace)'
latent
sort(diag(Lambda),'descend')
ev = (diag(Lambda))'; % 提取特征值
ev = ev(:, end:-1:1); % eig计算出的特征值是升序的,这里手动倒序(W同理)
W = W(:, end:-1:1);
sum(W.*W, 1) % 可以验证每个特征向量各元素的平方和均为1
Wr = W(:, 1:4); % 提取前两个主成分的特征向量
% compare Tr with dataInPrincipalComponentSpace
% both PCA component
Tr = centredX * Wr % 新坐标空间的数据点
%% SVD
[U, S, V] = svd(centredX);
PCA_result = centredX* V(:,1:PCA_component_Num)
'PCA_result', 'dataInPrincipalComponentSpace' and 'Tr' are different from each other.(they are same with real-value data)
Besides, I see someone using 'centredX* V(:,1:PCA_component_Num)' while someone using 'U(:,rk)*S(rk,:)*V'' to get principle component. Which one is correct?
the cyclist
el 25 de Jun. de 2024
Short answer
I don't know.
Longer answer
I've never done PCA on complex numbers, and I've never even heard of someone doing PCA on complex numbers. (I didn't find anything on a quick web search, and ChatGPT seems to hallucinate on some of the questions I've asked.)
At first, I thought the issue with your version might be due to the fact that you did not use all the principal components in some of the computation. I also thought there might be severe numerical instability because of cancelation in the random input matrix. But, I don't think either of those explanations are correct. (I've tried other input matrices, and get the same issues.)
I definitely don't understand why one would get different results from SVD, eig(cov matrix), and PCA.
I think this is worth crafting a smaller example with non-random input, and posting a brand-new question.
Más respuestas (2)
Yaser Khojah
el 17 de Abr. de 2019
Dear the cyclist, thanks for showing this example. I have a question regarding to the order of the COEFF since they are different than the V. Is there anyway to see which order of these columns? In another word, what are the variables of each column?
8 comentarios
Yuan Luo
el 8 de Nov. de 2020
why X need to be de-meaned? since pca by defualt will center the data.
the cyclist
el 26 de Dic. de 2020
Sorry it took me a while to see this question.
If you do
[coeff,score] = pca(X);
it is true that pca() will internally de-mean the data. So, score is derived from de-meaned data.
But it does not mean that X itself [outside of pca()] has been de-meaned. So, if you are trying to re-create what happens inside pca(), you need to manually de-mean X first.
Greg Heath
el 13 de Dic. de 2015
Hope this helps.
Thank you for formally accepting my answer
Greg
0 comentarios
Ver también
Categorías
Más información sobre Dimensionality Reduction and Feature Extraction en Help Center y File Exchange.
Productos
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!