Chapter 64. Spectral Theorem

The spectral theorem describes the best possible form of certain linear transformations.

For a general matrix, diagonalization may fail. A matrix may lack enough eigenvectors. Its eigenvalues may be complex. Its eigenvectors may fail to be orthogonal. The spectral theorem identifies an important class of matrices for which these difficulties disappear.

For real symmetric matrices, the theorem says that every such matrix has an orthonormal basis of eigenvectors. Equivalently, every real symmetric matrix can be diagonalized by an orthogonal matrix. This is the finite-dimensional real form of the spectral theorem.

64.1 Symmetric Matrices

A real square matrix $A$ is symmetric if

A^T=A.

This means that the entry in row $i$ , column $j$ , equals the entry in row $j$ , column $i$ :

a_{ij}=a_{ji}.

For example,

A= \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}

is symmetric.

The matrix

B= \begin{bmatrix} 2 & 4 \\ 1 & 3 \end{bmatrix}

is not symmetric, because

b_{12}=4

but

b_{21}=1.

Symmetric matrices occur naturally in geometry, quadratic forms, least squares, optimization, mechanics, statistics, and graph theory.

64.2 Statement of the Real Spectral Theorem

Let $A$ be a real $n\times n$ symmetric matrix.

Then there exists an orthonormal basis of $\mathbb{R}^n$ consisting of eigenvectors of $A$ .

Equivalently, there exists an orthogonal matrix $Q$ and a real diagonal matrix $D$ such that

A=QDQ^T.

Here

Q^TQ=I,

Q^{-1}=Q^T.

The columns of $Q$ are orthonormal eigenvectors of $A$ . The diagonal entries of $D$ are the corresponding eigenvalues.

This is stronger than ordinary diagonalization. Ordinary diagonalization gives

A=PDP^{-1}.

The spectral theorem gives

A=QDQ^T,

with $Q$ orthogonal.

64.3 Why This Is Special

A general matrix may not be diagonalizable.

For example,

\begin{bmatrix} 2 & 1 \\ 0 & 2 \end{bmatrix}

has only one independent eigenvector.

A general real matrix may also have no real eigenvectors.

For example,

\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}

rotates the plane by $90^\circ$ . Its eigenvalues are $i$ and $-i$ , so it has no real eigenvectors.

A real symmetric matrix avoids both problems. It has real eigenvalues. It has enough eigenvectors. Those eigenvectors can be chosen orthonormally.

This makes symmetric matrices one of the most important classes of matrices in linear algebra.

64.4 Orthogonal Diagonalization

A matrix $A$ is orthogonally diagonalizable if there exists an orthogonal matrix $Q$ and a diagonal matrix $D$ such that

A=QDQ^T.

Since

Q^{-1}=Q^T,

this is a special case of diagonalization:

A=QDQ^{-1}.

The advantage is numerical and geometric. Orthogonal matrices preserve lengths and angles. They do not distort the inner product.

If $Q$ is orthogonal, then for all vectors $x,y$ ,

(Qx)\cdot(Qy)=x\cdot y.

Thus orthogonal diagonalization changes coordinates by a rotation or reflection, then applies independent scaling, then changes coordinates back.

64.5 Matrix Form of the Theorem

Suppose

q_1,q_2,\ldots,q_n

are orthonormal eigenvectors of $A$ , with corresponding eigenvalues

\lambda_1,\lambda_2,\ldots,\lambda_n.

Let

Q= \begin{bmatrix} | & | & & | \\ q_1 & q_2 & \cdots & q_n \\ | & | & & | \end{bmatrix}.

Let

D= \begin{bmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{bmatrix}.

Then

AQ=QD.

Since $Q$ is orthogonal,

A=QDQ^T.

The order of the columns of $Q$ must match the order of the eigenvalues in $D$ .

64.6 Real Eigenvalues

A key fact behind the spectral theorem is that a real symmetric matrix has real eigenvalues.

More generally, a Hermitian matrix has real eigenvalues. A matrix $A$ over $\mathbb{C}$ is Hermitian if

A^*=A,

where $A^*$ is the conjugate transpose.

Suppose

Av=\lambda v

with $v\neq 0$ . If $A$ is real symmetric, then

v^TAv

is a real number.

Using the eigenvalue equation,

v^TAv=v^T(\lambda v)=\lambda v^Tv.

Since

v^Tv>0,

the scalar $\lambda$ must be real.

Over complex inner product spaces, the same argument uses

\langle Av,v\rangle=\langle v,Av\rangle

for Hermitian operators. Hermitian operators have real eigenvalues.

64.7 Orthogonality of Eigenvectors

Another key fact is that eigenvectors belonging to distinct eigenvalues are orthogonal.

Let $A$ be symmetric. Suppose

Av=\lambda v

and

Aw=\mu w,

where

\lambda\neq \mu.

Using symmetry,

(Av)\cdot w = v\cdot(Aw).

Substitute the eigenvalue equations:

(\lambda v)\cdot w = v\cdot(\mu w).

Thus

\lambda(v\cdot w)=\mu(v\cdot w).

(\lambda-\mu)(v\cdot w)=0.

Since $\lambda\neq\mu$ ,

v\cdot w=0.

Therefore $v$ and $w$ are orthogonal.

64.8 Repeated Eigenvalues

If an eigenvalue has multiplicity greater than one, its eigenspace may have dimension greater than one.

Inside that eigenspace, every vector has the same eigenvalue. The spectral theorem says that we can choose an orthonormal basis for each eigenspace.

This is done using Gram-Schmidt orthogonalization.

Suppose

E_\lambda

has basis

v_1,v_2,\ldots,v_k.

Applying Gram-Schmidt produces an orthonormal basis

q_1,q_2,\ldots,q_k

for the same eigenspace.

Since each $q_i$ remains in $E_\lambda$ , each $q_i$ is still an eigenvector with eigenvalue $\lambda$ .

Thus repeated eigenvalues cause no difficulty for symmetric matrices.

64.9 Spectral Decomposition

The spectral theorem can also be written as a sum of rank-one projections.

A=QDQ^T

with orthonormal eigenvectors

q_1,\ldots,q_n

and eigenvalues

\lambda_1,\ldots,\lambda_n,

then

A= \lambda_1 q_1q_1^T + \lambda_2 q_2q_2^T + \cdots + \lambda_n q_nq_n^T.

The matrix

q_iq_i^T

is the orthogonal projection onto the line spanned by $q_i$ .

Thus $A$ is a weighted sum of orthogonal projections. This representation is called the spectral decomposition. The finite-dimensional spectral theorem gives such a decomposition into eigenspaces or projections.

64.10 Projection Form by Eigenspaces

If the distinct eigenvalues of $A$ are

\alpha_1,\alpha_2,\ldots,\alpha_k,

then the space decomposes as an orthogonal direct sum

\mathbb{R}^n = E_{\alpha_1} \oplus E_{\alpha_2} \oplus \cdots \oplus E_{\alpha_k}.

Let $P_i$ be the orthogonal projection onto $E_{\alpha_i}$ . Then

A= \alpha_1P_1+\alpha_2P_2+\cdots+\alpha_kP_k.

This form groups together repeated eigenvalues.

The projection matrices satisfy

P_iP_j=0

for $i\neq j$ , and

P_1+P_2+\cdots+P_k=I.

Thus the identity decomposes into orthogonal spectral components.

64.11 Example: Orthogonal Diagonalization

Let

A= \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}.

The characteristic polynomial is

\det(A-\lambda I) = \det \begin{bmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{bmatrix}.

Thus

\det(A-\lambda I) = (2-\lambda)^2-1.

\lambda^2-4\lambda+3=0.

The eigenvalues are

\lambda_1=3, \qquad \lambda_2=1.

For $\lambda_1=3$ , one eigenvector is

\begin{bmatrix} 1 \\ 1 \end{bmatrix}.

Normalize it:

q_1= \frac{1}{\sqrt2} \begin{bmatrix} 1 \\ 1 \end{bmatrix}.

For $\lambda_2=1$ , one eigenvector is

\begin{bmatrix} 1 \\ -1 \end{bmatrix}.

Normalize it:

q_2= \frac{1}{\sqrt2} \begin{bmatrix} 1 \\ -1 \end{bmatrix}.

Then

Q= \frac{1}{\sqrt2} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}

and

D= \begin{bmatrix} 3 & 0 \\ 0 & 1 \end{bmatrix}.

The spectral theorem gives

A=QDQ^T.

64.12 Checking the Example

First observe that

Q^TQ=I.

Indeed,

Q^TQ = \frac12 \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}.

Now compute

QDQ^T.

Since

QDQ^T = \frac12 \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} 3 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix},

we get

QDQ^T = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}.

Thus

A=QDQ^T.

64.13 Quadratic Forms

The spectral theorem is especially useful for quadratic forms.

A quadratic form on $\mathbb{R}^n$ has the form

x^TAx,

where $A$ is symmetric.

A=QDQ^T,

let

y=Q^Tx.

Since $Q$ is orthogonal, this is a change to orthonormal coordinates.

Then

x^TAx = x^TQDQ^Tx = (Q^Tx)^TD(Q^Tx) = y^TDy.

D=\operatorname{diag}(\lambda_1,\ldots,\lambda_n),

then

y^TDy = \lambda_1y_1^2+\lambda_2y_2^2+\cdots+\lambda_ny_n^2.

Thus the spectral theorem removes all cross terms from a quadratic form.

64.14 Positive Definite Matrices

A symmetric matrix $A$ is positive definite if

x^TAx>0

for every nonzero vector $x$ .

Using the spectral theorem,

x^TAx = \lambda_1y_1^2+\cdots+\lambda_ny_n^2.

Therefore $A$ is positive definite exactly when all eigenvalues are positive.

Similarly:

Matrix type	Eigenvalue condition
Positive definite	All eigenvalues are positive
Positive semidefinite	All eigenvalues are nonnegative
Negative definite	All eigenvalues are negative
Negative semidefinite	All eigenvalues are nonpositive
Indefinite	Eigenvalues have both positive and negative signs

This criterion is one of the main uses of the spectral theorem in optimization and analysis.

64.15 Matrix Powers and Functions

A=QDQ^T,

then powers of $A$ are easy to compute:

A^k=QD^kQ^T.

Since $D$ is diagonal,

D^k= \operatorname{diag}(\lambda_1^k,\ldots,\lambda_n^k).

More generally, for a function $f$ ,

f(A)=Qf(D)Q^T,

where

f(D)=\operatorname{diag}(f(\lambda_1),\ldots,f(\lambda_n)).

This applies to functions such as

A^{1/2}, \qquad e^A, \qquad \log A, \qquad (A+\alpha I)^{-1}.

The spectral theorem therefore reduces many matrix operations to scalar operations on eigenvalues.

64.16 Square Roots of Positive Semidefinite Matrices

If $A$ is symmetric positive semidefinite, all eigenvalues satisfy

\lambda_i\geq 0.

With

A=QDQ^T,

define

A^{1/2}=QD^{1/2}Q^T,

where

D^{1/2} = \operatorname{diag}(\sqrt{\lambda_1},\ldots,\sqrt{\lambda_n}).

Then

A^{1/2}A^{1/2}=A.

This square root is symmetric and positive semidefinite.

Matrix square roots are used in covariance matrices, numerical analysis, optimization, and differential equations.

64.17 Singular Value Decomposition Connection

The spectral theorem applies to symmetric or Hermitian matrices. The singular value decomposition applies to every matrix.

For a real matrix $B$ , the matrix

B^TB

is symmetric positive semidefinite. Therefore it has an orthonormal eigenbasis.

The eigenvalues of $B^TB$ are nonnegative. Their square roots are the singular values of $B$ .

Thus the singular value decomposition is built from the spectral theorem applied to $B^TB$ and $BB^T$ . The spectral decomposition is also commonly viewed as a special case related to the singular value decomposition.

64.18 Complex Version

Over complex vector spaces, the corresponding class is Hermitian matrices.

A matrix $A$ is Hermitian if

A^*=A.

The spectral theorem for Hermitian matrices states that there exists a unitary matrix $U$ and a real diagonal matrix $D$ such that

A=UDU^*.

Here

U^*U=I.

The columns of $U$ form an orthonormal basis of eigenvectors.

More generally, every normal complex matrix is unitarily diagonalizable. A matrix is normal if

AA^*=A^*A.

For normal matrices, the eigenvalues may be complex, but an orthonormal eigenbasis still exists.

64.19 Spectral Theorem for Linear Transformations

Let $V$ be a finite-dimensional real inner product space, and let

T:V\to V

be a self-adjoint linear transformation. Self-adjoint means

\langle T v,w\rangle=\langle v,Tw\rangle

for all $v,w\in V$ .

The spectral theorem states that $V$ has an orthonormal basis consisting of eigenvectors of $T$ .

In such a basis, the matrix of $T$ is diagonal.

Thus the theorem is fundamentally a statement about self-adjoint transformations, not merely about arrays of numbers.

64.20 Common Errors

The first common error is to assume that every diagonalizable matrix is orthogonally diagonalizable. Orthogonal diagonalization is stronger.

The second common error is to forget symmetry. A real matrix with real eigenvalues may still fail to have orthogonal eigenvectors.

The third common error is to write

A=QDQ^{-1}

without using

Q^{-1}=Q^T.

For orthogonal diagonalization, the canonical form is

A=QDQ^T.

The fourth common error is to treat repeated eigenvalues as a problem. For symmetric matrices, repeated eigenspaces can always be given orthonormal bases.

The fifth common error is to ignore the distinction between real symmetric, complex Hermitian, and complex normal matrices.

64.21 Summary

The spectral theorem states that every real symmetric matrix has an orthonormal basis of eigenvectors.

Equivalently, if $A=A^T$ , then there exists an orthogonal matrix $Q$ and a real diagonal matrix $D$ such that

A=QDQ^T.

The columns of $Q$ are orthonormal eigenvectors. The entries of $D$ are eigenvalues.

The theorem makes symmetric matrices structurally transparent. It explains why quadratic forms can be diagonalized by orthogonal changes of coordinates, why positive definiteness is determined by eigenvalues, and why many matrix functions reduce to scalar functions of eigenvalues.