Chapter 49. Orthonormal Bases

An orthonormal basis is a basis made from unit vectors that are mutually orthogonal. It is one of the most useful structures in linear algebra because it combines two properties at once: every vector can be expressed uniquely in the basis, and the coefficients are found directly by inner products.

In an ordinary basis, coordinates may require solving a linear system. In an orthonormal basis, coordinates are obtained by taking dot products. This makes orthonormal bases central in projection, least squares, Fourier analysis, numerical linear algebra, signal processing, and spectral theory. Standard references define an orthonormal set as an orthogonal set of unit vectors, and such a set is linearly independent.

49.1 Unit Vectors

Let $V$ be an inner product space. A vector $v \in V$ is a unit vector if

\|v\| = 1.

Since the norm is induced by the inner product,

\|v\| = \sqrt{\langle v,v\rangle}.

Thus $v$ is a unit vector exactly when

\langle v,v\rangle = 1.

For example, in $\mathbb{R}^2$ ,

e_1 = \begin{bmatrix} 1\\ 0 \end{bmatrix}, \qquad e_2 = \begin{bmatrix} 0\\ 1 \end{bmatrix}

are unit vectors.

The vector

v = \begin{bmatrix} 3\\ 4 \end{bmatrix}

is not a unit vector because

\|v\| = \sqrt{3^2+4^2}=5.

To make it a unit vector, divide by its length:

\frac{v}{\|v\|} = \frac{1}{5} \begin{bmatrix} 3\\ 4 \end{bmatrix} = \begin{bmatrix} 3/5\\ 4/5 \end{bmatrix}.

This process is called normalization.

49.2 Normalization

If $v\ne 0$ , its normalization is

\widehat{v} = \frac{v}{\|v\|}.

Then

\|\widehat{v}\| = \left\| \frac{v}{\|v\|} \right\| = \frac{1}{\|v\|}\|v\| = 1.

Thus $\widehat{v}$ is a unit vector in the same direction as $v$ .

Normalization changes the length of a vector but not its direction. It is the operation that converts an orthogonal basis into an orthonormal basis. If $v_1,\ldots,v_k$ are nonzero orthogonal vectors, then

\frac{v_1}{\|v_1\|}, \frac{v_2}{\|v_2\|}, \ldots, \frac{v_k}{\|v_k\|}

form an orthonormal set. This follows because scaling nonzero orthogonal vectors preserves orthogonality and gives each vector length one.

49.3 Orthonormal Sets

A set of vectors

\{q_1,q_2,\ldots,q_k\}

is orthonormal if

\langle q_i,q_j\rangle = \begin{cases} 1, & i=j,\\ 0, & i\ne j. \end{cases}

Equivalently,

\langle q_i,q_j\rangle = \delta_{ij},

where $\delta_{ij}$ is the Kronecker delta.

The condition contains two requirements. When $i=j$ ,

\langle q_i,q_i\rangle = 1,

so every vector has length one. When $i\ne j$ ,

\langle q_i,q_j\rangle = 0,

so distinct vectors are orthogonal.

For example,

q_1 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1\\ 1 \end{bmatrix}, \qquad q_2 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1\\ -1 \end{bmatrix}

form an orthonormal set in $\mathbb{R}^2$ . Indeed,

\|q_1\|=1, \qquad \|q_2\|=1,

and

q_1^Tq_2 = \frac{1}{2}(1-1) = 0.

49.4 Orthonormal Bases

An orthonormal basis is an orthonormal set that is also a basis.

Thus $q_1,\ldots,q_n$ form an orthonormal basis for $V$ if:

Property	Meaning
Spanning	Every vector in $V$ is a linear combination of the $q_i$
Linear independence	The representation is unique
Orthogonality	Distinct basis vectors have inner product zero
Normalization	Each basis vector has length one

In a finite-dimensional inner product space, any orthonormal set with $\dim V$ vectors is automatically an orthonormal basis. This follows because every orthonormal set is linearly independent, and a linearly independent set with the dimension of the space is a basis.

49.5 Orthonormal Sets Are Linearly Independent

Every orthonormal set is linearly independent.

Let

\{q_1,\ldots,q_k\}

be an orthonormal set, and suppose

c_1q_1+\cdots+c_kq_k=0.

Take the inner product with $q_j$ . Then

\left\langle c_1q_1+\cdots+c_kq_k, q_j \right\rangle = \langle 0,q_j\rangle.

By linearity,

c_1\langle q_1,q_j\rangle + \cdots + c_k\langle q_k,q_j\rangle = 0.

All terms vanish except the $j$ -th term. Since $\langle q_j,q_j\rangle=1$ , we get

c_j=0.

This holds for every $j$ . Therefore all coefficients are zero, so the set is linearly independent.

This proof is one reason orthonormal systems are algebraically clean. The inner product isolates one coordinate at a time.

49.6 Coordinates in an Orthonormal Basis

Let

B=(q_1,\ldots,q_n)

be an orthonormal basis for $V$ . Every vector $v\in V$ has a unique representation

v = c_1q_1+\cdots+c_nq_n.

To find $c_j$ , take the inner product with $q_j$ :

\langle v,q_j\rangle = \left\langle c_1q_1+\cdots+c_nq_n, q_j \right\rangle.

Using orthonormality,

\langle v,q_j\rangle = c_j.

Therefore

c_j = \langle v,q_j\rangle.

Hence

v = \sum_{j=1}^n \langle v,q_j\rangle q_j.

This is the coordinate formula for an orthonormal basis. It replaces solving a system by taking inner products. Coordinate formulas of this form are a primary advantage of orthogonal and orthonormal bases.

49.7 Coordinate Vector

If $B=(q_1,\ldots,q_n)$ is an orthonormal basis, then the coordinate vector of $v$ relative to $B$ is

[v]_B = \begin{bmatrix} \langle v,q_1\rangle\\ \langle v,q_2\rangle\\ \vdots\\ \langle v,q_n\rangle \end{bmatrix}.

For example, let

q_1 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1\\ 1 \end{bmatrix}, \qquad q_2 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1\\ -1 \end{bmatrix},

and let

v = \begin{bmatrix} 4\\ 2 \end{bmatrix}.

Then

\langle v,q_1\rangle = \frac{1}{\sqrt{2}}(4+2) = 3\sqrt{2},

and

\langle v,q_2\rangle = \frac{1}{\sqrt{2}}(4-2) = \sqrt{2}.

Thus

[v]_B = \begin{bmatrix} 3\sqrt{2}\\ \sqrt{2} \end{bmatrix}.

Therefore

v = 3\sqrt{2}q_1+\sqrt{2}q_2.

49.8 Matrix Form

Let $Q$ be the matrix whose columns are the orthonormal vectors

q_1,\ldots,q_n.

Then

Q = \begin{bmatrix} | & | & & |\\ q_1 & q_2 & \cdots & q_n\\ | & | & & | \end{bmatrix}.

The condition that the columns are orthonormal is

Q^TQ=I

in the real case.

In the complex case, the condition is

Q^*Q=I,

where $Q^*$ is the conjugate transpose.

If $Q$ is square, then

Q^{-1}=Q^T

in the real case, and

Q^{-1}=Q^*

in the complex case.

A real square matrix with orthonormal columns is called an orthogonal matrix. A complex square matrix with orthonormal columns is called a unitary matrix.

49.9 Orthogonal Matrices

A real square matrix $Q$ is orthogonal if

Q^TQ=I.

Since $Q$ is square, this also implies

QQ^T=I.

Thus

Q^{-1}=Q^T.

Orthogonal matrices preserve inner products:

\langle Qx,Qy\rangle = (Qx)^T(Qy) = x^TQ^TQy = x^Ty = \langle x,y\rangle.

They also preserve norms:

\|Qx\|_2=\|x\|_2.

Therefore orthogonal matrices represent rigid linear transformations: rotations, reflections, and combinations of them.

They do not stretch or shrink Euclidean length.

49.10 Unitary Matrices

In complex vector spaces, the analogue of an orthogonal matrix is a unitary matrix.

A square complex matrix $U$ is unitary if

U^*U=I.

Then

U^{-1}=U^*.

Unitary matrices preserve complex inner products:

\langle Ux,Uy\rangle = \langle x,y\rangle.

They also preserve Euclidean norm:

\|Ux\|_2=\|x\|_2.

Unitary matrices appear throughout spectral theory, quantum mechanics, Fourier analysis, and numerical linear algebra. The discrete Fourier transform matrix, after proper normalization, is unitary.

49.11 Projection onto an Orthonormal Basis

Let $W$ be a subspace with orthonormal basis

q_1,\ldots,q_k.

The orthogonal projection of $v$ onto $W$ is

\operatorname{proj}_W(v) = \sum_{j=1}^k \langle v,q_j\rangle q_j.

This formula follows from the coordinate formula, applied only to the subspace $W$ .

The residual

r = v-\operatorname{proj}_W(v)

is orthogonal to every basis vector $q_i$ :

\langle r,q_i\rangle = \langle v,q_i\rangle - \sum_{j=1}^k \langle v,q_j\rangle \langle q_j,q_i\rangle = \langle v,q_i\rangle-\langle v,q_i\rangle = 0.

Therefore

r\in W^\perp.

Projection with an orthonormal basis avoids the matrix inverse appearing in the general projection formula.

49.12 Projection Matrix

Let $Q$ be an $m\times k$ matrix with orthonormal columns. Thus

Q^TQ=I_k.

The projection of $b\in\mathbb{R}^m$ onto $\operatorname{Col}(Q)$ is

p = QQ^Tb.

Hence the projection matrix is

P = QQ^T.

This is simpler than the general formula

P=A(A^TA)^{-1}A^T.

The simplification occurs because $Q^TQ=I$ . Orthonormal columns remove the need to invert the Gram matrix.

The matrix $P=QQ^T$ satisfies

P^2=P

and

P^T=P.

Thus it is an orthogonal projection matrix.

49.13 Parseval Identity

Let $q_1,\ldots,q_n$ be an orthonormal basis for $V$ . If

v = \sum_{j=1}^n c_jq_j,

then

\|v\|^2 = |c_1|^2+\cdots+|c_n|^2.

Since

c_j=\langle v,q_j\rangle,

we have

\|v\|^2 = \sum_{j=1}^n |\langle v,q_j\rangle|^2.

This is Parseval’s identity in finite-dimensional form.

It says that the squared length of a vector equals the sum of the squared magnitudes of its orthonormal coordinates.

In $\mathbb{R}^n$ , this generalizes the usual formula

\|x\|_2^2=x_1^2+\cdots+x_n^2.

49.14 Bessel Inequality

If $q_1,\ldots,q_k$ is an orthonormal set, but not necessarily a basis for all of $V$ , then

\sum_{j=1}^k |\langle v,q_j\rangle|^2 \le \|v\|^2.

This is Bessel’s inequality.

It says that the energy captured by projection onto the span of $q_1,\ldots,q_k$ cannot exceed the total energy of $v$ .

The inequality becomes an equality exactly when $v$ lies in the span of the orthonormal set. In that case, the set captures all of $v$ .

49.15 Distance to a Subspace

Let $W$ have orthonormal basis $q_1,\ldots,q_k$ . The closest vector in $W$ to $v$ is

p = \sum_{j=1}^k \langle v,q_j\rangle q_j.

The distance from $v$ to $W$ is

\operatorname{dist}(v,W)=\|v-p\|.

Using orthogonal decomposition,

v=p+r, \qquad p\in W, \qquad r\in W^\perp.

Then

\|v\|^2=\|p\|^2+\|r\|^2.

Since

\|p\|^2= \sum_{j=1}^k |\langle v,q_j\rangle|^2,

we obtain

\operatorname{dist}(v,W)^2 = \|v\|^2 - \sum_{j=1}^k |\langle v,q_j\rangle|^2.

This formula is useful in least squares and approximation.

49.16 Change of Orthonormal Basis

Let $B=(q_1,\ldots,q_n)$ and $C=(r_1,\ldots,r_n)$ be two orthonormal bases of a real inner product space. The change-of-basis matrix from $B$ -coordinates to $C$ -coordinates is orthogonal.

Indeed, both coordinate systems preserve inner products and lengths. Therefore the transformation between them also preserves inner products and lengths.

In matrix form, if $Q$ and $R$ are the matrices with columns $q_i$ and $r_i$ , then

Q^TQ=I, \qquad R^TR=I.

The change-of-basis matrix is

R^TQ.

It is orthogonal because

(R^TQ)^T(R^TQ) = Q^TRR^TQ = Q^TQ = I.

Thus moving between orthonormal coordinate systems is numerically stable and geometrically rigid.

49.17 Orthonormal Bases and Least Squares

In least squares, one often wants to approximate $b$ by a vector in the column space of a matrix.

If the columns of $A$ are not orthonormal, the projection formula is

p=A(A^TA)^{-1}A^Tb.

If the columns are orthonormal, write the matrix as $Q$ . Then

p=QQ^Tb.

The least squares coefficients are

\hat{x}=Q^Tb.

This is simpler and more stable than solving the normal equations.

This observation motivates the QR factorization. Instead of working directly with $A$ , we factor it as

A=QR,

where $Q$ has orthonormal columns and $R$ is upper triangular. The orthonormal factor $Q$ carries the geometry of the column space.

49.18 Orthonormal Bases in Function Spaces

Orthonormal bases also occur in spaces of functions.

For functions on an interval $[a,b]$ , an inner product may be defined by

\langle f,g\rangle = \int_a^b f(x)g(x)\,dx.

A sequence of functions $\phi_1,\phi_2,\ldots$ is orthonormal if

\langle \phi_i,\phi_j\rangle=\delta_{ij}.

For example, trigonometric functions form orthogonal and, after scaling, orthonormal systems on intervals such as $[-\pi,\pi]$ . In Fourier analysis, a function is represented by coefficients obtained from inner products:

c_j=\langle f,\phi_j\rangle.

This is the infinite-dimensional analogue of coordinates in an orthonormal basis.

49.19 Numerical Importance

Orthonormal bases are important in numerical computation because they control error.

When a matrix has orthonormal columns,

Q^TQ=I,

so multiplying by $Q$ does not amplify Euclidean length. This gives stable algorithms for projections, least squares, eigenvalue computations, and matrix factorizations.

In contrast, a poorly conditioned basis may distort coordinates. Small errors in a vector can become large errors in its coordinate representation.

For this reason, numerical linear algebra often replaces arbitrary bases by orthonormal bases. The Gram-Schmidt process, Householder reflections, and Givens rotations are standard methods for doing this.

49.20 Summary

An orthonormal basis is a basis $q_1,\ldots,q_n$ satisfying

\langle q_i,q_j\rangle=\delta_{ij}.

It gives the expansion

v=\sum_{j=1}^n \langle v,q_j\rangle q_j.

Thus coordinates are obtained by inner products.

If $Q$ is the matrix whose columns are an orthonormal basis, then

Q^TQ=I.

When $Q$ is square,

Q^{-1}=Q^T.

Orthonormal bases simplify projection, least squares, coordinate changes, and norm computations. They preserve geometry and improve numerical stability.