Chapter 38. Projection Operators

A projection operator is a linear operator that leaves its own output unchanged. If $V$ is a vector space, a projection on $V$ is a linear map

such that

Equivalently,

for every $v\in V$. Applying the projection once moves a vector into the target subspace. Applying it again has no further effect. This property is called idempotence. Projection operators are therefore exactly idempotent linear operators.

38.1 First Example

Define

by

This map sends every vector to its shadow on the $xy$-plane.

Apply $P$ twice:

Thus

So $P$ is a projection operator.

The image is the $xy$-plane:

The kernel is the $z$-axis:

The projection keeps the image and removes the kernel.

38.2 Idempotence

The defining equation

means

for every vector $v\in V$.

This has a direct interpretation. The first application of $P$ sends $v$ into $\operatorname{im}(P)$. Once a vector is in $\operatorname{im}(P)$, the projection leaves it fixed.

Indeed, if $w\in\operatorname{im}(P)$, then there is some $v\in V$ such that

Then

Thus every vector in the image is fixed by $P$.

Conversely, if

then $w\in\operatorname{im}(P)$, since $w$ is the image of itself under $P$. Hence

The image of a projection is exactly its fixed subspace.

38.3 Kernel and Image

For every projection $P:V\to V$, the kernel and image describe the whole operator.

The kernel is

The image is

Every vector $v\in V$ can be decomposed as

The first term satisfies

The second term lies in the kernel, because

Therefore

is a decomposition of $v$ into a part in the image and a part in the kernel.

Moreover, this decomposition is unique. If

then $u\in\operatorname{im}(P)$ implies

But $u\in\ker(P)$ implies

Thus

So

Hence

Every projection gives a direct sum decomposition of the vector space.

38.4 Projection Onto a Subspace Along a Complement

Let $V$ be a vector space, and suppose

This means every vector $v\in V$ has a unique decomposition

where

Define

by

That is, $P$ keeps the $U$-component and discards the $W$-component.

Then $P$ is linear. If

then

so

For a scalar $c$,

so

Also,

Thus $P$ is a projection.

Its image is $U$, and its kernel is $W$:

So a projection is the same as a choice of direct sum decomposition.

38.5 The Complementary Projection

If $P$ is a projection on $V$, then

is also a projection.

Compute:

Since

we get

Thus $I-P$ is idempotent.

The projection $I-P$ keeps the part that $P$ removes. For any vector $v$,

The image of $I-P$ is the kernel of $P$:

The kernel of $I-P$ is the image of $P$:

Thus $P$ and $I-P$ are complementary projections.

38.6 Matrix Projections

A square matrix $P$ is called a projection matrix if

Such a matrix defines a projection operator

For example,

satisfies

It projects $\mathbb{R}^2$ onto the $x$-axis:

A projection matrix must be square because it represents a linear operator from a vector space to itself.

38.7 Orthogonal Projections

In an inner product space, an important special case is the orthogonal projection.

An orthogonal projection onto a subspace $U$ writes each vector $v$ as

where

and

Then

The vector $u$ is the closest vector in $U$ to $v$. The difference

is perpendicular to $U$.

For example, the projection from $\mathbb{R}^3$ onto the $xy$-plane is orthogonal because the removed part lies on the $z$-axis, which is perpendicular to the plane.

In real coordinate spaces, an orthogonal projection matrix satisfies

and

Thus it is both idempotent and symmetric. For complex spaces, symmetry is replaced by self-adjointness:

Orthogonal projection matrices are therefore characterized by

in the real case, and

in the complex case.

38.8 Projection Onto a Line

Let $u\in\mathbb{R}^n$ be a nonzero vector. The orthogonal projection of $x\in\mathbb{R}^n$ onto the line spanned by $u$ is

The scalar

is the coordinate of the projection along $u$.

The corresponding matrix is

Indeed,

This matrix is symmetric and idempotent, so it is an orthogonal projection matrix.

38.9 Example: Projection Onto a Line in $\mathbb{R}^2$

Let

Then

Also,

Therefore the projection matrix onto the line spanned by $u$ is

For

we get

The vector $x$ projects exactly onto $u$. The error vector is

Check orthogonality:

The error is perpendicular to the line.

38.10 Projection Onto a Subspace with an Orthonormal Basis

Let $U$ be a subspace of $\mathbb{R}^n$, and let

be an orthonormal basis of $U$. Then the orthogonal projection of $x$ onto $U$ is

If $Q$ is the matrix with columns

then

The projection matrix is

Indeed,

The matrix $QQ^T$ is symmetric and idempotent:

and

38.11 Projection Onto a Column Space

Let $A$ be an $m\times k$ matrix with linearly independent columns. The column space of $A$ is a $k$-dimensional subspace of $\mathbb{R}^m$.

The orthogonal projection onto $\operatorname{col}(A)$ is

This formula generalizes the line projection formula. When $A$ has one column $u$, it becomes

The matrix $A^TA$ is invertible because the columns of $A$ are linearly independent.

The projection $Px$ is the vector in $\operatorname{col}(A)$ closest to $x$, and the residual

is orthogonal to every column of $A$. Projection formulas of this kind are central in least squares and regression.

38.12 Derivation of the Column Space Formula

We seek a vector in $\operatorname{col}(A)$ closest to $x$. Such a vector has the form

for some coefficient vector $\hat c$.

The residual is

For $A\hat c$ to be the orthogonal projection, $r$ must be orthogonal to every column of $A$. This condition is

Expanding gives

So

Since $A^TA$ is invertible,

Therefore

Thus

38.13 Oblique Projections

A projection need not be orthogonal.

Suppose

The projection onto $U$ along $W$ sends

to

If $W=U^\perp$, the projection is orthogonal. If $W$ is another complement, the projection is oblique.

An oblique projection still satisfies

It still has

But the removed part $w$ may not be perpendicular to $U$. Oblique projections are therefore algebraically valid projections, but they do not describe nearest-point projection in the usual Euclidean metric.

38.14 Example of an Oblique Projection

In $\mathbb{R}^2$, let

be the $x$-axis, and let

Every vector

can be written uniquely as

From the second coordinate,

From the first coordinate,

so

Thus the projection onto $U$ along $W$ is

Its matrix is

Check idempotence:

This projection is not orthogonal because

It projects onto the $x$-axis along diagonal lines parallel to $(1,1)$, not along vertical lines.

38.15 Eigenvalues of a Projection

Let $P$ be a projection. If $v$ is an eigenvector with eigenvalue $\lambda$, then

Apply $P$ again:

But $P^2=P$, so

Hence

Since $v\neq 0$,

Thus

Therefore every eigenvalue of a projection is either

or

Vectors in the kernel have eigenvalue $0$. Vectors in the image have eigenvalue $1$. Projection matrices have only the eigenvalues $0$ and $1$.

38.16 Diagonal Form

Because the minimal polynomial of a projection divides

and this polynomial has distinct roots, every projection on a finite-dimensional vector space is diagonalizable.

More concretely, choose a basis

for $\operatorname{im}(P)$, and choose a basis

for $\ker(P)$.

Since

the combined list

is a basis of $V$.

In this basis, $P$ has matrix

Thus a projection is structurally simple: it is identity on one subspace and zero on a complementary subspace.

38.17 Trace and Rank

For a finite-dimensional projection $P$, the trace equals the rank.

In the basis adapted to

the matrix of $P$ is

where

The trace is the sum of diagonal entries:

The rank is also

Therefore

This fact is often useful in matrix analysis, statistics, and numerical linear algebra.

38.18 Products of Projections

The product of two projections is not necessarily a projection.

Let $P$ and $Q$ be projections. The product $PQ$ is a projection if

Compute:

If $P$ and $Q$ commute, meaning

then

Thus, if two projections commute, their product is also a projection.

Without commutativity, the product may fail to be idempotent. Products of projections therefore require care.

38.19 Projections and Least Squares

Projection operators are the algebraic core of least squares.

Given an inconsistent system

there may be no exact solution. Instead, least squares seeks a vector $\hat x$ such that

is as close as possible to $b$ inside the column space of $A$.

This means

where

is the orthogonal projection onto the column space of $A$.

The normal equations

come from the orthogonality condition

Thus least squares is not merely an approximation trick. It is an orthogonal projection problem.

38.20 Summary

A projection operator is a linear operator

satisfying

This means applying $P$ twice is the same as applying it once.

Every projection decomposes the vector space as

It acts as the identity on its image and as zero on its kernel.

If $V=U\oplus W$, then the projection onto $U$ along $W$ is the map

Orthogonal projections occur in inner product spaces when the complement is perpendicular to the target subspace. In real coordinates, an orthogonal projection matrix satisfies

Projection matrices have eigenvalues only $0$ and $1$, are diagonalizable, and satisfy

Projection operators are central in geometry, least squares, regression, numerical linear algebra, and the study of decompositions of vector spaces.