BIZNIQUE / LINEAR ALGEBRA / 19
Connect PCA with singular value decomposition
Connect PCA with singular value decomposition
Understand the statistical goal and the algebraic tool without treating them as competing methods.
Embedded Files
THE IDEA
Read the example.
Read the example.
For a column-centred data matrix, right singular vectors give principal directions. The corresponding sample covariance eigenvalues are σ²/(n−1). Truncating the decomposition gives a lower-rank approximation.
X = UΣVᵀ; covariance = XᵀX / (n−1)
Use it well
Use it well
SVD does not centre data automatically. Applying it to an uncentred matrix answers a different question from ordinary centred PCA.
TRY IT YOURSELF
A quick check.
A quick check.
With n = 5 observations and singular value σ = 4, what sample covariance eigenvalue corresponds to that direction?
Check your reasoning
Check your reasoning
σ²/(n−1) = 16/4 = 4.
Page updated
Google Sites
Report abuse