A set of vectors [math]\left\{w_1,w_2,\ldots,w_p\right\}[/math] in [math]\mathbb{R}^n[/math] is an [b]orthonormal set[/b] if it is an orthogonal set of unit vectors. It is an [b]orthonormal basis[/b] for a subspace [math]W[/math] of [math]\mathbb{R}^n[/math] if it spans [math]W[/math].[br][br]Let [math]U[/math] be an m x n matrix. Suppose its column vectors [math]\mathbf{u}_1, \mathbf{u}_2,\ldots, \mathbf{u}_n[/math] in [math]\mathbb{R}^m[/math] form an orthonormal set. It is equivalent to [math]U^TU=I[/math] because the [math](i,j)[/math]-entry of [math]U^TU[/math] is [math]\mathbf{u}_i\cdot\mathbf{u}_j[/math] for [math]i,j=1,2,\ldots,n[/math] and [math]\left\{\mathbf{u}_1, \mathbf{u}_2,\ldots, \mathbf{u}_n\right\}[/math] is an orthonormal set if and only if [math]\mathbf{u}_i\cdot\mathbf{u}_j=0[/math] for [math]i\ne j[/math] and [math]\mathbf{u}_i\cdot\mathbf{u}_j=1[/math] for [math]i=j[/math].[br][br]Here are some nice properties of [math]U[/math]: For any vector [math]x,y[/math] in [math]\mathbb{R}^n[/math],[br][br]1. [math](Ux)\cdot (Uy)=x\cdot y[/math][br]2. [math]\|Ux\|=\|x\|[/math][br][br](Note: If we view [math]x\mapsto Ux[/math] be a linear transformation from [math]\mathbb{R}^n[/math] to [math]\mathbb{R}^m[/math], then this linear transformation preserves the inner product and norm.)[br][br][u]Proof[/u]: [br]1. [math]U^TU=I\Rightarrow x^TU^TU=x^TI=x^T\Rightarrow x^TU^TUy=x^Ty\Rightarrow (Ux)^T(Uy)=x^Ty[/math][br]By definition, [math](Ux)^T(Uy)=(Ux)\cdot (Uy)[/math] and [math]x^Ty=x\cdot y[/math]. Hence, [math](Ux)\cdot (Uy)=x\cdot y[/math][br][br]2. Using (1), [math](Ux)\cdot(Ux)=x\cdot x\Rightarrow \|Ux\|^2=\|x\|^2\Rightarrow \|Ux\|=\|x\|[/math][br][br]In particular, when [math]m=n[/math], [math]U[/math] is an n x n square matrix such that [math]U^TU=I[/math], which means [math]U[/math] is invertible and [math]U^{-1}=U^T[/math]. Such a square matrix is called an [b]orthogonal matrix[/b]. From above, we know that the linear transformation [math]T\left(x\right)=Ux[/math] preserves the inner product and norm. Moreover, it also preserve volume because [math]\det(U)=\pm 1[/math]. (Why?)[br][br][br]If [math]\left\{w_1,w_2,\ldots,w_p\right\}[/math] is an orthonormal basis for a subspace [math]W[/math] of [math]\mathbb{R}^n[/math], then for any [math]v[/math] in [math]\mathbb{R}^n[/math], then it is very convenient to write down its orthogonal projection onto [math]W[/math]:[br][br][math]\text{proj}_W v=(v\cdot w_1)w_1+(v\cdot w_2)w_2+\cdots+(v\cdot w_p)w_p[/math][br][br]If [math]U=\left(w_1 \ w_2 \ \cdots \ w_p \right)[/math], then [math]\text{proj}_W v=UU^Tv[/math].[br][br][u]Proof[/u]: Easy[br][br][br][br]

Given a basis for a subspace of [math]\mathbb{R}^n[/math], we have a simple way called [b]the Gram-Schmidt process[/b] to produce an orthogonal or orthonormal basis for the subspace. The idea is as follows:[br][br]Suppose you are given a basis [math]\left\{u_1,u_2,\ldots,u_p\right\}[/math] of a subspace [math]W[/math] of [math]\mathbb{R}^n[/math]. Start with [math]u_1[/math] and let it be the first one in the orthogonal basis that we are going to produce and call it [math]v_1[/math] i.e. [math]v_1=u_1[/math].[br][br]Let [math]W_1=\text{Span}\{v_1\}[/math]. We consider the orthogonal decomposition [math]u_2[/math] and get the component [math]v_2[/math] in [math]W_1^\perp[/math] i.e. [math]v_2=u_2-\text{proj}_{W_1}u_2=u_2-\frac{u_2\cdot v_1}{v_1\cdot v_1}v_1[/math][br][br]Then [math]\{v_1,v_2\}[/math] is an orthogonal set.[br][br]Let [math]W_2=\text{Span}\{v_1,v_2\}[/math]. We consider the orthogonal decomposition [math]u_3[/math] and get the component [math]v_3[/math] in [math]W_2^\perp[/math] i.e. [math]v_3=u_3-\text{proj}_{W_2}u_3=u_3-\left(\frac{u_3\cdot v_1}{v_1\cdot v_1}v_1+\frac{u_3\cdot v_2}{v_2\cdot v_2}v_2\right)[/math][br][br]Then [math]\{v_1,v_2,v_3\}[/math] is an orthogonal set. We repeat this procedure. At the i[sup]th[/sup] step, we let [math]W_{i-1}=\{v_1,v_2,\ldots,v_{i-1}\}[/math] and[br][br][math]v_i=u_i-\text{proj}_{W_{i-1}}u_i=u_i-\left(\frac{u_i\cdot v_1}{v_1\cdot v_1}v_1+\frac{u_i\cdot v_2}{v_2\cdot v_2}v_2+\cdots+\frac{u_i\cdot v_{i-1}}{v_{i-1}\cdot v_{i-1}}v_{i-1}\right)[/math][br][br]After p steps, we obtain the orthogonal basis [math]\left\{v_1,v_2,\ldots,v_p\right\}[/math] for [math]W[/math]. If you want to make it an orthonormal basis, then you can simply normalize each vector to [math]\left\{\hat{v}_1,\hat{v}_2,\ldots,\hat{v}_p\right\}[/math], where [math]\hat{v}_i=\frac1{\|v_i\|}v_i[/math] for [math]i=1,2,\ldots,p[/math].[br][br][u]Remark[/u]: If you change the order of the vectors in the given basis, you will get a different orthogonal basis from the Gram-Schimdt process. [br][br][br][u]Example[/u]: Find an orthonormal basis for [math]\text{Span}\left\{\begin{pmatrix}1\\2\\-3\\0\end{pmatrix},\begin{pmatrix}5\\0\\4\\7\end{pmatrix},\begin{pmatrix}3\\1\\-5\\9\end{pmatrix}\right\}[/math] in [math]\mathbb{R}^4[/math].[br][br]You can try this online [url=https://www.emathhelp.net/calculators/linear-algebra/gram-schmidt-calculator/]Gram-Schimdt calculator[/url].[br][br][br]