Chapter 25. Row Space and Column Space

The row space and column space are two fundamental subspaces attached to a matrix. They describe what the rows generate and what the columns generate. If $A$ is an $m \times n$ matrix, then the row space is a subspace of $F^n$ , while the column space is a subspace of $F^m$ . The column space is also the image of the linear map $x \mapsto Ax$ .

These spaces connect matrices, linear systems, rank, pivots, and linear transformations. They are part of the standard group of fundamental matrix subspaces, together with the null space and left null space.

25.1 Matrix Rows and Columns

Let

A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}.

The rows of $A$ are vectors in $F^n$ :

r_1,\ r_2,\ \ldots,\ r_m.

The columns of $A$ are vectors in $F^m$ :

a_1,\ a_2,\ \ldots,\ a_n.

Thus the rows and columns usually live in different vector spaces. This distinction matters when $m \neq n$ .

25.2 Row Space

The row space of $A$ , denoted

\operatorname{Row}(A),

is the span of the rows of $A$ :

\operatorname{Row}(A) = \operatorname{span}(r_1,\ldots,r_m).

Since each row has $n$ entries,

\operatorname{Row}(A) \subseteq F^n.

A vector belongs to the row space exactly when it can be written as a linear combination of the rows.

A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix},

then

\operatorname{Row}(A) = \operatorname{span} \left( \begin{bmatrix} 1 & 0 & 2 \end{bmatrix}, \begin{bmatrix} 0 & 1 & 3 \end{bmatrix} \right).

A general vector in the row space has the form

c_1 \begin{bmatrix} 1 & 0 & 2 \end{bmatrix} + c_2 \begin{bmatrix} 0 & 1 & 3 \end{bmatrix} = \begin{bmatrix} c_1 & c_2 & 2c_1+3c_2 \end{bmatrix}.

Thus the row space is a plane through the origin in $F^3$ .

25.3 Column Space

The column space of $A$ , denoted

\operatorname{Col}(A),

is the span of the columns of $A$ :

\operatorname{Col}(A) = \operatorname{span}(a_1,\ldots,a_n).

Since each column has $m$ entries,

\operatorname{Col}(A) \subseteq F^m.

A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 2 & 3 \end{bmatrix},

then

\operatorname{Col}(A) = \operatorname{span} \left( \begin{bmatrix} 1\\ 0\\ 2 \end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 3 \end{bmatrix} \right).

A general vector in the column space has the form

c_1 \begin{bmatrix} 1\\ 0\\ 2 \end{bmatrix} + c_2 \begin{bmatrix} 0\\ 1\\ 3 \end{bmatrix} = \begin{bmatrix} c_1\\ c_2\\ 2c_1+3c_2 \end{bmatrix}.

This is a two-dimensional subspace of $F^3$ .

25.4 Matrix-Vector Multiplication and Column Space

Let

A = \begin{bmatrix} | & | & & | \\ a_1 & a_2 & \cdots & a_n \\ | & | & & | \end{bmatrix}

and let

x = \begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{bmatrix}.

Then

Ax = x_1a_1+x_2a_2+\cdots+x_na_n.

Thus every product $Ax$ is a linear combination of the columns of $A$ . Conversely, every linear combination of the columns is some product $Ax$ .

Therefore

\operatorname{Col}(A)=\{Ax:x\in F^n\}.

This is why the column space is the image, or range, of the linear transformation

x \mapsto Ax.

25.5 Linear Systems and Column Space

The system

Ax=b

has a solution exactly when

b\in \operatorname{Col}(A).

This is the column-space interpretation of consistency.

The columns of $A$ are the available building blocks. The unknowns $x_1,\ldots,x_n$ are the coefficients. Solving $Ax=b$ asks whether $b$ can be built from the columns.

Thus consistency is a geometric question:

\text{Is } b \text{ in the span of the columns of } A?

If yes, the system is consistent. If no, the system is inconsistent.

25.6 Example: Consistency

Let

A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{bmatrix}.

The columns are

a_1 = \begin{bmatrix} 1\\ 0\\ 1 \end{bmatrix}, \qquad a_2 = \begin{bmatrix} 0\\ 1\\ 1 \end{bmatrix}.

A vector $b\in F^3$ lies in $\operatorname{Col}(A)$ if

b = x_1a_1+x_2a_2.

Thus

b = \begin{bmatrix} x_1\\ x_2\\ x_1+x_2 \end{bmatrix}.

So $b=(b_1,b_2,b_3)$ belongs to the column space exactly when

b_3=b_1+b_2.

For example,

\begin{bmatrix} 2\\ 5\\ 7 \end{bmatrix} \in \operatorname{Col}(A),

but

\begin{bmatrix} 2\\ 5\\ 8 \end{bmatrix} \notin \operatorname{Col}(A).

Therefore

Ax= \begin{bmatrix} 2\\ 5\\ 7 \end{bmatrix}

is consistent, while

Ax= \begin{bmatrix} 2\\ 5\\ 8 \end{bmatrix}

is inconsistent.

25.7 Row Space and Linear Equations

Each row of $A$ represents one linear equation in the system

Ax=b.

For example, the row

\begin{bmatrix} 2 & -1 & 3 \end{bmatrix}

represents the left-hand side

2x_1-x_2+3x_3.

The row space consists of all linear combinations of these equation coefficients.

If one row lies in the span of the others, then its equation is algebraically dependent on the others, at least on the left-hand side. Row space records the independent linear constraints imposed by the matrix.

25.8 Row Operations and Row Space

Elementary row operations do not change the row space.

There are three elementary row operations:

Operation	Effect on row space
Swap two rows	Reorders spanning vectors
Multiply a row by a nonzero scalar	Replaces a spanning vector by a nonzero multiple
Add a multiple of one row to another	Replaces one spanning vector by a vector in the same span

Each operation replaces the rows by another list with the same span.

Therefore, if $R$ is row equivalent to $A$ , then

\operatorname{Row}(R)=\operatorname{Row}(A).

This is why row reduction gives a basis for the row space.

25.9 Row Echelon Form and Row Space

If $R$ is an echelon form of $A$ , then the nonzero rows of $R$ form a basis for $\operatorname{Row}(A)$ .

They span the row space because row operations preserve row space.

They are linearly independent because nonzero rows in echelon form have leading entries in different columns. No nonzero row can be built from the rows below it without changing the position of the first nonzero entry.

Thus row reduction gives a direct method:

Row reduce $A$ to echelon form.
Keep the nonzero rows.
These rows form a basis for the row space.

25.10 Example: Basis for Row Space

Let

A = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 2 \\ 1 & 1 & 0 \end{bmatrix}.

Row reduce:

\begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 2 \\ 1 & 1 & 0 \end{bmatrix} \to \begin{bmatrix} 1 & 2 & 1 \\ 0 & 0 & 0 \\ 0 & -1 & -1 \end{bmatrix} \to \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}.

The nonzero rows form a basis for the row space:

\left( \begin{bmatrix} 1 & 2 & 1 \end{bmatrix}, \begin{bmatrix} 0 & 1 & 1 \end{bmatrix} \right).

Therefore

\dim \operatorname{Row}(A)=2.

25.11 Row Operations and Column Space

Elementary row operations generally change the column space.

This is because row operations change the entries inside each column. The transformed columns may live in the same ambient space $F^m$ , but their span can become a different subspace.

For example,

A= \begin{bmatrix} 1\\ 1 \end{bmatrix}

has column space

\operatorname{span} \left( \begin{bmatrix} 1\\ 1 \end{bmatrix} \right).

Apply the row operation $R_2\leftarrow R_2-R_1$ . The result is

R= \begin{bmatrix} 1\\ 0 \end{bmatrix}.

Now

\operatorname{Col}(R) = \operatorname{span} \left( \begin{bmatrix} 1\\ 0 \end{bmatrix} \right),

which is a different line in $F^2$ .

Thus the actual column space is not preserved by row reduction.

25.12 Pivot Columns and Column Space

Although row operations change the column space, they preserve linear dependence relations among the columns. Therefore row reduction can identify which original columns form a basis for the column space. The pivot columns of a row echelon form indicate the corresponding pivot columns of the original matrix. Those original columns form a basis for $\operatorname{Col}(A)$ .

The word original is essential. The pivot columns of the reduced matrix usually form a basis for the column space of the reduced matrix, not for the original column space.

The procedure is:

Row reduce $A$ to echelon form.
Identify pivot columns in the echelon form.
Select the corresponding columns of the original matrix.
These original columns form a basis for $\operatorname{Col}(A)$ .

25.13 Example: Basis for Column Space

Let

A = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 2 \\ 1 & 1 & 0 \end{bmatrix}.

From the previous row reduction,

A \sim \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}.

The pivot columns are columns $1$ and $2$ .

Therefore a basis for the column space of $A$ is given by the first two columns of the original matrix:

\left( \begin{bmatrix} 1\\ 2\\ 1 \end{bmatrix}, \begin{bmatrix} 2\\ 4\\ 1 \end{bmatrix} \right).

The third column is redundant. Indeed,

\begin{bmatrix} 1\\ 2\\ 0 \end{bmatrix} = -\begin{bmatrix} 1\\ 2\\ 1 \end{bmatrix} + \begin{bmatrix} 2\\ 4\\ 1 \end{bmatrix}.

Thus

\dim \operatorname{Col}(A)=2.

25.14 Rank

The rank of a matrix is the dimension of its column space:

\operatorname{rank}(A) = \dim \operatorname{Col}(A).

It is also equal to the dimension of the row space:

\operatorname{rank}(A) = \dim \operatorname{Row}(A).

Thus

\dim \operatorname{Row}(A) = \dim \operatorname{Col}(A).

This equality is one of the central facts of matrix theory. The row space and column space may live in different ambient spaces, but their dimensions are always equal.

The common dimension is the number of pivot positions in any echelon form of $A$ .

25.15 Row Rank Equals Column Rank

The equality

\dim \operatorname{Row}(A)=\dim \operatorname{Col}(A)

is often stated as:

\text{row rank}=\text{column rank}.

Row reduction explains why.

The nonzero rows of an echelon form give a basis for the row space, so the row rank equals the number of nonzero rows.

The pivot columns of the echelon form identify a basis for the original column space, so the column rank equals the number of pivot columns.

In echelon form, the number of nonzero rows equals the number of pivot columns.

Therefore row rank equals column rank.

25.16 Rank and Matrix Shape

If $A$ is an $m\times n$ matrix, then

\operatorname{Row}(A)\subseteq F^n,

and

\operatorname{Col}(A)\subseteq F^m.

Therefore

\operatorname{rank}(A)\leq n

and

\operatorname{rank}(A)\leq m.

\operatorname{rank}(A)\leq \min(m,n).

A tall matrix can have at most as many independent columns as columns. A wide matrix can have at most as many independent rows as rows.

25.17 Full Column Rank

An $m\times n$ matrix has full column rank if

\operatorname{rank}(A)=n.

This means the columns are linearly independent.

In this case, the homogeneous system

Ax=0

has only the trivial solution.

If $m\geq n$ , full column rank is possible. If $m<n$ , it is impossible because there are more columns than the ambient dimension $F^m$ .

Full column rank means that the map

x\mapsto Ax

is injective.

25.18 Full Row Rank

An $m\times n$ matrix has full row rank if

\operatorname{rank}(A)=m.

This means the rows are linearly independent.

If $m\leq n$ , full row rank is possible. If $m>n$ , it is impossible because there are more rows than the ambient dimension $F^n$ .

Full row rank means that the linear transformation

x\mapsto Ax

has image equal to $F^m$ . Thus for every $b\in F^m$ , the system

Ax=b

has at least one solution.

25.19 Four Fundamental Subspaces

For an $m\times n$ matrix $A$ , there are four associated subspaces:

Subspace	Definition	Ambient space
Row space	$\operatorname{Row}(A)$	$F^n$
Column space	$\operatorname{Col}(A)$	$F^m$
Null space	$\operatorname{Null}(A)=\{x:Ax=0\}$	$F^n$
Left null space	$\operatorname{Null}(A^T)=\{y:A^Ty=0\}$	$F^m$

The row space and null space both live in $F^n$ . The column space and left null space both live in $F^m$ .

Later chapters show that the row space is orthogonal to the null space, and the column space is orthogonal to the left null space.

25.20 Summary

The row space of a matrix is the span of its rows. The column space is the span of its columns. These spaces express two complementary ways to read a matrix: by equations and by generated outputs.

The key ideas are:

Concept	Meaning
Row space	Span of the rows of $A$
Column space	Span of the columns of $A$
Column-space test	$Ax=b$ is consistent exactly when $b\in\operatorname{Col}(A)$
Row reduction	Preserves row space
Nonzero echelon rows	Basis for row space
Pivot columns	Corresponding original columns form a basis for column space
Rank	Common dimension of row space and column space
Full column rank	Independent columns
Full row rank	Independent rows

Row space describes the independent equations encoded by a matrix. Column space describes the possible outputs produced by the matrix. Rank measures the common number of independent directions in both descriptions.