As we know, diagonalization is a very useful technique of decomposing a matrix so that we can do computations more easily. However, there are two main limitation of diagonalization:[br][list][*]It only applies to square matrices.[/*][*]Not every square matrix is diagonalizable.[br][/*][/list][br]But it turns out that there exists a matrix decomposition which is equally if not more useful than diagonalization that works for [b]ANY[/b] matrix (even the non-square ones!). This is the so-called [b]singular value decomposition (SVD)[/b]:[br][br]For any real [math]m\times n[/math] matrix [math]A[/math], there exists an [math]m\times m[/math] orthogonal matrix [math]U[/math] and an [math] n\times n[/math] orthogonal matrix [math]V[/math] such that [math]A=U\Sigma V^T[/math], where [math]\Sigma[/math] is a diagonal [math]m\times n[/math] matrix. Moreover, the diagonal entries of [math]\Sigma[/math], denoted by [math]\sigma_i[/math] for the [math]i^\text{th}[/math] row ([math]i=1,\ldots,m[/math]), are nonnegative and can be arranged in descending order. The positive [math]\sigma_i[/math] are called the [b]singular values[/b] of [math]A[/math] and the column vectors of [math]U[/math] and [math]V[/math] are called the left and right [b]singular vectors[/b] of [math]A[/math] respectively.[br]

Let [math]A=\begin{pmatrix}2 &-2 & 1 \\ -4 & -8 & -8\end{pmatrix}[/math]. The SVD of [math]A[/math] is as follows:[br][br][math]A=U\Sigma V^T=\begin{pmatrix}0 & 1\\-1 & 0\end{pmatrix}\begin{pmatrix}12 & 0 & 0 \\ 0 & 3 & 0\end{pmatrix} \begin{pmatrix}\frac 13 & \frac 23 & \frac 23 \\ \frac 23 & -\frac 23 & \frac 13 \\ \frac 23 & \frac 13 & -\frac 23\end{pmatrix}^T[/math][br][br](Note: SVD is not unique.)[br][br][br]This is an useful [url=https://matrixcalc.org/en/vectors.html]online SVD calculator[/url].[br][br]