# Chapter 48. Orthogonal Complements # Chapter 48. Orthogonal Complements An orthogonal complement records all directions perpendicular to a given set of vectors. If a subspace describes the directions allowed by a problem, its orthogonal complement describes the directions excluded by it. Let \(V\) be an inner product space, and let \(S \subseteq V\). The orthogonal complement of \(S\) is $$ S^\perp = \{x \in V : \langle x,s\rangle = 0 \text{ for every } s \in S\}. $$ The notation \(S^\perp\) is read as “\(S\) perp.” It is the set of all vectors orthogonal to every vector in \(S\). Standard references define it this way and note that it is always a subspace of the ambient inner product space. ## 48.1 First Examples In \(\mathbb{R}^2\), let $$ S = \operatorname{span} \left\{ \begin{bmatrix} 1 \\ 0 \end{bmatrix} \right\}. $$ Then \(S\) is the \(x\)-axis. A vector $$ x = \begin{bmatrix} a \\ b \end{bmatrix} $$ belongs to \(S^\perp\) precisely when $$ \left\langle \begin{bmatrix} a \\ b \end{bmatrix}, \begin{bmatrix} 1 \\ 0 \end{bmatrix} \right\rangle = 0. $$ This gives $$ a = 0. $$ Therefore $$ S^\perp = \left\{ \begin{bmatrix} 0 \\ b \end{bmatrix} : b\in\mathbb{R} \right\}. $$ Thus the orthogonal complement of the \(x\)-axis is the \(y\)-axis. In \(\mathbb{R}^3\), the orthogonal complement of a line through the origin is the plane through the origin perpendicular to that line. The orthogonal complement of a plane through the origin is the line through the origin perpendicular to that plane. ## 48.2 Orthogonal Complement of a Set The definition applies to any subset \(S\), not only to a subspace. If $$ S = \{s_1,s_2,\ldots,s_k\}, $$ then $$ S^\perp = \{x\in V : \langle x,s_i\rangle=0 \text{ for } i=1,\ldots,k\}. $$ Thus \(S^\perp\) is the common solution set of several homogeneous linear equations. For example, in \(\mathbb{R}^3\), let $$ S = \left\{ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} \right\}. $$ A vector $$ x = \begin{bmatrix} a \\ b \\ c \end{bmatrix} $$ lies in \(S^\perp\) when $$ a+b=0 $$ and $$ b+c=0. $$ Thus $$ a=-b, \qquad c=-b. $$ So $$ x = b \begin{bmatrix} -1 \\ 1 \\ -1 \end{bmatrix}. $$ Therefore $$ S^\perp = \operatorname{span} \left\{ \begin{bmatrix} -1 \\ 1 \\ -1 \end{bmatrix} \right\}. $$ ## 48.3 The Orthogonal Complement Is a Subspace For every subset \(S\subseteq V\), the set \(S^\perp\) is a subspace of \(V\). This remains true even when \(S\) itself is not a subspace. First, the zero vector belongs to \(S^\perp\), since $$ \langle 0,s\rangle = 0 $$ for every \(s\in S\). Now suppose \(x,y\in S^\perp\), and let \(a,b\) be scalars. For every \(s\in S\), $$ \langle ax+by,s\rangle = a\langle x,s\rangle + b\langle y,s\rangle. $$ Since \(x\in S^\perp\) and \(y\in S^\perp\), $$ \langle x,s\rangle=0, \qquad \langle y,s\rangle=0. $$ Therefore $$ \langle ax+by,s\rangle=0. $$ Thus $$ ax+by\in S^\perp. $$ So \(S^\perp\) is closed under linear combinations and is a subspace. ## 48.4 Orthogonal Complement of a Span A vector is orthogonal to a set if and only if it is orthogonal to every linear combination of vectors in that set. Hence $$ S^\perp = \operatorname{span}(S)^\perp. $$ This identity is useful because it allows us to replace a set by its span without changing the orthogonal complement. Proof: Suppose \(x\in S^\perp\). Let $$ w = c_1s_1+\cdots+c_ks_k $$ be a finite linear combination of vectors from \(S\). Then $$ \langle x,w\rangle = \langle x,c_1s_1+\cdots+c_ks_k\rangle. $$ By linearity, $$ \langle x,w\rangle = c_1\langle x,s_1\rangle+\cdots+c_k\langle x,s_k\rangle = 0. $$ Thus \(x\) is orthogonal to every vector in \(\operatorname{span}(S)\). The converse is immediate because $$ S\subseteq \operatorname{span}(S). $$ Therefore $$ S^\perp = \operatorname{span}(S)^\perp. $$ ## 48.5 Inclusion Reverses If $$ S\subseteq T, $$ then $$ T^\perp \subseteq S^\perp. $$ The inclusion reverses direction. This happens because being orthogonal to a larger set is a stronger condition. If a vector is orthogonal to every vector in \(T\), then it is certainly orthogonal to every vector in the smaller set \(S\). For example, in \(\mathbb{R}^3\), if \(S\) is a line inside a plane \(T\), then \(T^\perp\) is a line perpendicular to the plane, while \(S^\perp\) is a plane perpendicular to the line. The complement of the larger subspace is smaller. ## 48.6 Orthogonal Complements in Finite Dimensions Let \(W\) be a subspace of a finite-dimensional inner product space \(V\). Then $$ \dim W + \dim W^\perp = \dim V. $$ Equivalently, $$ \dim W^\perp = \dim V - \dim W. $$ This gives the expected geometric rule. In \(\mathbb{R}^3\), a line has dimension \(1\), so its orthogonal complement has dimension \(2\). A plane has dimension \(2\), so its orthogonal complement has dimension \(1\). In finite-dimensional inner product spaces, a \(k\)-dimensional subspace has an \((n-k)\)-dimensional orthogonal complement. ## 48.7 Trivial Intersection If \(W\) is a subspace of an inner product space \(V\), then $$ W \cap W^\perp = \{0\}. $$ Indeed, if \(x\in W\cap W^\perp\), then \(x\in W\) and \(x\) is orthogonal to every vector in \(W\). Since \(x\in W\), it is orthogonal to itself: $$ \langle x,x\rangle = 0. $$ Positive definiteness gives $$ x=0. $$ Thus the only vector that lies both in a subspace and in its orthogonal complement is the zero vector. ## 48.8 Direct Sum Decomposition If \(W\) is a subspace of a finite-dimensional inner product space \(V\), then $$ V = W \oplus W^\perp. $$ This means every vector \(v\in V\) can be written uniquely as $$ v = w + z, $$ where $$ w\in W, \qquad z\in W^\perp. $$ The uniqueness follows from $$ W\cap W^\perp=\{0\}. $$ The existence follows from the dimension formula: $$ \dim W + \dim W^\perp = \dim V. $$ This decomposition is called the orthogonal decomposition of \(V\) with respect to \(W\). In finite-dimensional inner product spaces, this direct-sum decomposition is one of the central structural properties of orthogonal complements. ## 48.9 Double Orthogonal Complement In finite-dimensional inner product spaces, $$ (W^\perp)^\perp = W. $$ The inclusion $$ W \subseteq (W^\perp)^\perp $$ is direct. Every vector in \(W\) is orthogonal to every vector in \(W^\perp\), so every vector in \(W\) belongs to \((W^\perp)^\perp\). To prove equality, compare dimensions. Since $$ \dim W^\perp = \dim V - \dim W, $$ we have $$ \dim (W^\perp)^\perp = \dim V - \dim W^\perp. $$ Substituting, $$ \dim (W^\perp)^\perp = \dim V - (\dim V - \dim W) = \dim W. $$ Thus \(W\) is a subspace of \((W^\perp)^\perp\) with the same dimension. Therefore $$ (W^\perp)^\perp = W. $$ This finite-dimensional identity must be handled carefully in infinite-dimensional spaces. In Hilbert spaces, the double orthogonal complement of a subspace is its closure, so closedness becomes part of the statement. ## 48.10 Computing Orthogonal Complements In \(\mathbb{R}^n\), orthogonal complements are often computed by solving homogeneous systems. Suppose $$ W = \operatorname{span}\{w_1,\ldots,w_k\}. $$ A vector \(x\in\mathbb{R}^n\) belongs to \(W^\perp\) if and only if $$ w_1^T x = 0, \quad w_2^T x = 0, \quad \ldots, \quad w_k^T x = 0. $$ If we form the matrix $$ A = \begin{bmatrix} w_1^T \\ w_2^T \\ \vdots \\ w_k^T \end{bmatrix}, $$ then the conditions become $$ Ax=0. $$ Therefore $$ W^\perp = \operatorname{Null}(A). $$ So computing an orthogonal complement reduces to computing a null space. ## 48.11 Example in \(\mathbb{R}^4\) Let $$ W= \operatorname{span} \left\{ \begin{bmatrix} 1\\ 1\\ 0\\ 0 \end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\\ 0 \end{bmatrix} \right\}. $$ Let $$ x= \begin{bmatrix} a\\ b\\ c\\ d \end{bmatrix}. $$ The condition \(x\in W^\perp\) gives $$ a+b=0 $$ and $$ b+c=0. $$ Thus $$ a=-b, \qquad c=-b, $$ while \(d\) is free. Hence $$ x= b \begin{bmatrix} -1\\ 1\\ -1\\ 0 \end{bmatrix} + d \begin{bmatrix} 0\\ 0\\ 0\\ 1 \end{bmatrix}. $$ Therefore $$ W^\perp = \operatorname{span} \left\{ \begin{bmatrix} -1\\ 1\\ -1\\ 0 \end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\\ 1 \end{bmatrix} \right\}. $$ Since \(W\) has dimension \(2\) in \(\mathbb{R}^4\), its orthogonal complement also has dimension \(2\), as expected. ## 48.12 Orthogonal Complement and Null Space Let \(A\) be an \(m\times n\) real matrix. The null space of \(A\) is the orthogonal complement of the row space of \(A\): $$ \operatorname{Null}(A) = \operatorname{Row}(A)^\perp. $$ Indeed, $$ Ax=0 $$ means that every row of \(A\) has dot product zero with \(x\). Thus \(x\) is orthogonal to every vector in the row space. Similarly, $$ \operatorname{Null}(A^T) = \operatorname{Col}(A)^\perp. $$ The orthogonal-complement identities for row, column, and null spaces are standard finite-dimensional facts. They express the fundamental relation between equations and orthogonality. ## 48.13 Four Fundamental Subspaces For an \(m\times n\) matrix \(A\), the four fundamental subspaces are: | Subspace | Ambient space | Orthogonal complement | |---|---:|---| | \(\operatorname{Row}(A)\) | \(\mathbb{R}^n\) | \(\operatorname{Null}(A)\) | | \(\operatorname{Null}(A)\) | \(\mathbb{R}^n\) | \(\operatorname{Row}(A)\) | | \(\operatorname{Col}(A)\) | \(\mathbb{R}^m\) | \(\operatorname{Null}(A^T)\) | | \(\operatorname{Null}(A^T)\) | \(\mathbb{R}^m\) | \(\operatorname{Col}(A)\) | Thus $$ \mathbb{R}^n = \operatorname{Row}(A) \oplus \operatorname{Null}(A), $$ and $$ \mathbb{R}^m = \operatorname{Col}(A) \oplus \operatorname{Null}(A^T). $$ These decompositions separate each ambient space into a range part and a null part. They are central in solving linear systems, least squares, and understanding rank. ## 48.14 Orthogonal Complement and Projection The orthogonal complement gives the residual part of a projection. Let \(W\) be a finite-dimensional subspace of an inner product space \(V\). For every \(v\in V\), there exists a unique decomposition $$ v = w + z, $$ where $$ w\in W, \qquad z\in W^\perp. $$ The vector \(w\) is the orthogonal projection of \(v\) onto \(W\), and \(z\) is the residual. Thus $$ z = v-w. $$ The defining condition for projection is $$ v-w \in W^\perp. $$ Equivalently, $$ \langle v-w,u\rangle = 0 $$ for every \(u\in W\). This is the main equation behind least squares approximation. ## 48.15 Projection onto a Subspace with Orthonormal Basis Suppose \(W\) has an orthonormal basis $$ q_1,\ldots,q_k. $$ Then the projection of \(v\) onto \(W\) is $$ \operatorname{proj}_W(v) = \sum_{j=1}^k \langle v,q_j\rangle q_j. $$ The residual is $$ r = v-\operatorname{proj}_W(v). $$ For every \(i\), $$ \langle r,q_i\rangle = \left\langle v-\sum_{j=1}^k \langle v,q_j\rangle q_j, q_i \right\rangle. $$ Using orthonormality, $$ \langle r,q_i\rangle = \langle v,q_i\rangle - \langle v,q_i\rangle = 0. $$ Therefore \(r\in W^\perp\). ## 48.16 Least Squares Interpretation Consider a system $$ Ax=b $$ where \(A\) is an \(m\times n\) matrix and \(b\in\mathbb{R}^m\). If \(b\) does not lie in \(\operatorname{Col}(A)\), the system has no exact solution. The least squares problem asks for \(\hat{x}\) such that $$ A\hat{x} $$ is the closest vector in \(\operatorname{Col}(A)\) to \(b\). The residual $$ r=b-A\hat{x} $$ must lie in the orthogonal complement of the column space: $$ r\in \operatorname{Col}(A)^\perp. $$ Since $$ \operatorname{Col}(A)^\perp=\operatorname{Null}(A^T), $$ we get $$ A^T r = 0. $$ Substituting \(r=b-A\hat{x}\) gives the normal equations: $$ A^T(b-A\hat{x})=0, $$ or $$ A^T A\hat{x}=A^T b. $$ This derivation shows that least squares is fundamentally an orthogonal-complement problem. ## 48.17 Complex Inner Product Spaces In complex inner product spaces, the definition remains $$ S^\perp = \{x\in V : \langle x,s\rangle=0 \text{ for every } s\in S\}. $$ The main change is conjugation. In \(\mathbb{C}^n\), the standard inner product is $$ \langle x,y\rangle = x^*y $$ or, depending on convention, $$ \langle x,y\rangle = y^*x. $$ The zero condition is unaffected by the convention, provided it is used consistently. For a complex matrix \(A\), $$ \operatorname{Null}(A) = \operatorname{Row}(A)^\perp $$ with rows interpreted through the complex inner product. Also, $$ \operatorname{Null}(A^*) = \operatorname{Col}(A)^\perp. $$ The transpose in the real case becomes the conjugate transpose in the complex case. ## 48.18 Infinite-Dimensional Caution In finite dimensions, every subspace is closed, and $$ (W^\perp)^\perp = W. $$ In infinite-dimensional Hilbert spaces, a subspace may fail to be closed. In that setting, $$ (W^\perp)^\perp = \overline{W}, $$ where \(\overline{W}\) is the closure of \(W\). If \(W\) is closed, then $$ (W^\perp)^\perp = W. $$ This distinction is invisible in elementary finite-dimensional linear algebra but becomes important in functional analysis. Orthogonal complements are always closed in Hilbert spaces, even when the original subspace is not closed. ## 48.19 Common Identities For subspaces \(U,W\) of a finite-dimensional inner product space, $$ U\subseteq W \quad \Longrightarrow \quad W^\perp\subseteq U^\perp. $$ Also, $$ (U+W)^\perp = U^\perp\cap W^\perp. $$ Indeed, a vector is orthogonal to every vector in \(U+W\) exactly when it is orthogonal to every vector in \(U\) and every vector in \(W\). In finite dimensions, $$ (U\cap W)^\perp = U^\perp + W^\perp. $$ These identities show that orthogonal complementation exchanges sums and intersections. It reverses inclusion and changes dimension by complementarity. ## 48.20 Summary The orthogonal complement of a set \(S\) is the subspace of all vectors orthogonal to every vector in \(S\): $$ S^\perp = \{x\in V : \langle x,s\rangle=0 \text{ for all } s\in S\}. $$ It is always a subspace. It depends only on the span of \(S\), so $$ S^\perp = \operatorname{span}(S)^\perp. $$ For a finite-dimensional subspace \(W\subseteq V\), $$ \dim W + \dim W^\perp = \dim V, $$ $$ W\cap W^\perp=\{0\}, $$ and $$ V=W\oplus W^\perp. $$ Orthogonal complements connect geometry with computation. They describe null spaces, residuals, projections, least squares, and the four fundamental subspaces of a matrix.