Singular value decomposition
SVD factors a real matrix into orthogonal directions, nonnegative scale factors, and another orthogonal change of coordinates. It applies to rectangular matrices too.
Definition
SVD factors a real matrix into orthogonal directions, nonnegative scale factors, and another orthogonal change of coordinates. It applies to rectangular matrices too.
Intuition
A linear map can be understood as rotate or reflect, scale, then rotate or reflect.
Worked example
For diag(3, 1), keeping only the singular value 3 gives a rank-one approximation diag(3, 0).
The math
. Truncating to the largest k singular values gives a best rank-k approximation in spectral and Frobenius norms.
In machine learning
SVD supports compression, pseudoinverses, least squares, and diagnosing numerical rank.
Sources
More in Linear algebra
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.