Chapter 15. Block Matrices

A block matrix is a matrix partitioned into smaller matrices called blocks or submatrices. The entries of a block matrix are themselves matrices, arranged so that their sizes are compatible. Block notation is used to expose structure, simplify multiplication, describe large systems, and express decompositions. A matrix may be partitioned in many different ways, depending on the calculation being performed.

15.1 Partitioning a Matrix

A matrix can be divided by grouping rows and columns.

For example,

A= \left[ \begin{array}{cc|c} 1&2&3\\ 4&5&6\\ \hline 7&8&9 \end{array} \right]

may be written as

A= \begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix},

where

A_{11}= \begin{bmatrix} 1&2\\ 4&5 \end{bmatrix}, \qquad A_{12}= \begin{bmatrix} 3\\ 6 \end{bmatrix},

A_{21}= \begin{bmatrix} 7&8 \end{bmatrix}, \qquad A_{22}= \begin{bmatrix} 9 \end{bmatrix}.

The horizontal and vertical lines are only visual guides. They are not new algebraic symbols.

15.2 Conformable Blocks

Block notation is valid only when the block sizes fit together.

A= \begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix},

the blocks in the same block row must have the same number of rows. The blocks in the same block column must have the same number of columns.

For example, if

A_{11} \text{ is } 2\times 3,

then $A_{12}$ must also have $2$ rows. If

A_{21} \text{ is } 4\times 3,

then $A_{22}$ must have $4$ rows and the same number of columns as $A_{12}$ .

Thus block matrices obey the same row-column compatibility rules as ordinary matrices, but at the level of submatrices.

15.3 Block Addition

Block matrices with the same partition can be added block by block.

A= \begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix}

and

B= \begin{bmatrix} B_{11}&B_{12}\\ B_{21}&B_{22} \end{bmatrix},

with corresponding blocks of the same size, then

A+B= \begin{bmatrix} A_{11}+B_{11}&A_{12}+B_{12}\\ A_{21}+B_{21}&A_{22}+B_{22} \end{bmatrix}.

For example,

\begin{bmatrix} A&B\\ C&D \end{bmatrix} + \begin{bmatrix} E&F\\ G&H \end{bmatrix} = \begin{bmatrix} A+E&B+F\\ C+G&D+H \end{bmatrix}.

Each block sum must be defined.

15.4 Block Scalar Multiplication

Scalar multiplication is also performed block by block:

c \begin{bmatrix} A&B\\ C&D \end{bmatrix} = \begin{bmatrix} cA&cB\\ cC&cD \end{bmatrix}.

This follows directly from ordinary scalar multiplication of matrix entries.

15.5 Block Matrix Multiplication

Block multiplication follows the same row-column rule as ordinary matrix multiplication.

A= \begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix}

and

B= \begin{bmatrix} B_{11}&B_{12}\\ B_{21}&B_{22} \end{bmatrix},

and all block products are defined, then

AB= \begin{bmatrix} A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22} \\ A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22} \end{bmatrix}.

More generally, if

C=AB,

then the $(i,j)$ -block of $C$ is

C_{ij}=\sum_k A_{ik}B_{kj}.

This is exactly the ordinary multiplication rule, with scalar products replaced by matrix products. The partitions must be conformable: the column partition of $A$ must match the row partition of $B$ .

15.6 Example of Block Multiplication

Let

A= \begin{bmatrix} A_{11}&A_{12} \end{bmatrix}

and

x= \begin{bmatrix} x_1\\ x_2 \end{bmatrix}.

Then

Ax=A_{11}x_1+A_{12}x_2.

This is useful when a vector of unknowns is split into groups.

For example, a system

Ax=b

may be written as

\begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix} \begin{bmatrix} x_1\\ x_2 \end{bmatrix} = \begin{bmatrix} b_1\\ b_2 \end{bmatrix}.

This gives the block system

A_{11}x_1+A_{12}x_2=b_1,

A_{21}x_1+A_{22}x_2=b_2.

Thus block notation separates a large system into coupled subsystems.

15.7 Block Transpose

The transpose of a block matrix transposes each block and also swaps block positions.

A= \begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22} \end{bmatrix},

then

A^T= \begin{bmatrix} A_{11}^T&A_{21}^T\\ A_{12}^T&A_{22}^T \end{bmatrix}.

Notice both changes. The off-diagonal blocks exchange positions, and each block is transposed.

For complex matrices, the conjugate transpose satisfies

A^*= \begin{bmatrix} A_{11}^*&A_{21}^*\\ A_{12}^*&A_{22}^* \end{bmatrix}.

15.8 Block Diagonal Matrices

A block diagonal matrix has square blocks on the diagonal and zero blocks elsewhere:

D= \begin{bmatrix} D_1&0&\cdots&0\\ 0&D_2&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&D_k \end{bmatrix}.

Block diagonal matrices act independently on separate groups of coordinates.

D= \begin{bmatrix} D_1&0\\ 0&D_2 \end{bmatrix}

and

x= \begin{bmatrix} x_1\\ x_2 \end{bmatrix},

then

Dx= \begin{bmatrix} D_1x_1\\ D_2x_2 \end{bmatrix}.

The two parts do not interact.

15.9 Direct Sums

The direct sum of two matrices $A$ and $B$ is the block diagonal matrix

A\oplus B= \begin{bmatrix} A&0\\ 0&B \end{bmatrix}.

For example,

\begin{bmatrix} 1&2\\ 3&4 \end{bmatrix} \oplus \begin{bmatrix} 5&6\\ 7&8 \end{bmatrix} = \begin{bmatrix} 1&2&0&0\\ 3&4&0&0\\ 0&0&5&6\\ 0&0&7&8 \end{bmatrix}.

Direct sums describe transformations that act separately on independent components.

15.10 Block Triangular Matrices

A block upper triangular matrix has the form

T= \begin{bmatrix} A&B\\ 0&D \end{bmatrix},

where $A$ and $D$ are square blocks.

A block lower triangular matrix has the form

L= \begin{bmatrix} A&0\\ C&D \end{bmatrix}.

These matrices generalize ordinary triangular matrices. The zero blocks show that some parts of the system do not feed into others.

For block triangular matrices,

\det \begin{bmatrix} A&B\\ 0&D \end{bmatrix} = \det(A)\det(D),

and

\det \begin{bmatrix} A&0\\ C&D \end{bmatrix} = \det(A)\det(D).

This determinant rule holds when $A$ and $D$ are square blocks.

15.11 Inverse of a Block Diagonal Matrix

A block diagonal matrix is invertible exactly when each diagonal block is invertible.

D= \begin{bmatrix} D_1&0\\ 0&D_2 \end{bmatrix},

and $D_1,D_2$ are invertible, then

D^{-1} = \begin{bmatrix} D_1^{-1}&0\\ 0&D_2^{-1} \end{bmatrix}.

Indeed,

\begin{bmatrix} D_1&0\\ 0&D_2 \end{bmatrix} \begin{bmatrix} D_1^{-1}&0\\ 0&D_2^{-1} \end{bmatrix} = \begin{bmatrix} I&0\\ 0&I \end{bmatrix}.

Thus inversion separates across independent blocks.

15.12 Inverse of a Block Triangular Matrix

Let

M= \begin{bmatrix} A&B\\ 0&D \end{bmatrix},

where $A$ and $D$ are invertible. Then

M^{-1} = \begin{bmatrix} A^{-1}&-A^{-1}BD^{-1}\\ 0&D^{-1} \end{bmatrix}.

Check by multiplication:

\begin{bmatrix} A&B\\ 0&D \end{bmatrix} \begin{bmatrix} A^{-1}&-A^{-1}BD^{-1}\\ 0&D^{-1} \end{bmatrix} = \begin{bmatrix} I&0\\ 0&I \end{bmatrix}.

The lower triangular case is similar:

\begin{bmatrix} A&0\\ C&D \end{bmatrix}^{-1} = \begin{bmatrix} A^{-1}&0\\ -D^{-1}CA^{-1}&D^{-1} \end{bmatrix}.

15.13 The Schur Complement

Let

M= \begin{bmatrix} A&B\\ C&D \end{bmatrix},

where $A$ is square and invertible.

The Schur complement of $A$ in $M$ is

S=D-CA^{-1}B.

This matrix appears when eliminating $x_1$ from the block system

Ax_1+Bx_2=b_1,

Cx_1+Dx_2=b_2.

From the first equation,

x_1=A^{-1}(b_1-Bx_2).

Substitute into the second equation:

C A^{-1}(b_1-Bx_2)+Dx_2=b_2.

Thus

(D-CA^{-1}B)x_2=b_2-CA^{-1}b_1.

The matrix $D-CA^{-1}B$ is the Schur complement.

15.14 Block Inverse Formula

If $A$ and the Schur complement

S=D-CA^{-1}B

are invertible, then

\begin{bmatrix} A&B\\ C&D \end{bmatrix}^{-1} = \begin{bmatrix} A^{-1}+A^{-1}BS^{-1}CA^{-1}&-A^{-1}BS^{-1}\\ -S^{-1}CA^{-1}&S^{-1} \end{bmatrix}.

This formula is useful in theory and in structured computations. It should be applied only when the required inverses exist. A corresponding formula exists when $D$ is invertible, using the Schur complement

A-BD^{-1}C.

Block inverse formulas are standard consequences of block elimination.

15.15 Block Gaussian Elimination

Block Gaussian elimination applies the idea of ordinary elimination to blocks.

For

M= \begin{bmatrix} A&B\\ C&D \end{bmatrix},

with $A$ invertible, eliminate the lower-left block $C$ by multiplying on the left with

\begin{bmatrix} I&0\\ -CA^{-1}&I \end{bmatrix}.

Then

\begin{bmatrix} I&0\\ -CA^{-1}&I \end{bmatrix} \begin{bmatrix} A&B\\ C&D \end{bmatrix} = \begin{bmatrix} A&B\\ 0&D-CA^{-1}B \end{bmatrix}.

The lower-right block is the Schur complement.

This is the block analogue of replacing a row by itself minus a multiple of a pivot row.

15.16 Block LU Factorization

The previous identity gives a block factorization.

If $A$ is invertible and $S=D-CA^{-1}B$ , then

\begin{bmatrix} A&B\\ C&D \end{bmatrix} = \begin{bmatrix} I&0\\ CA^{-1}&I \end{bmatrix} \begin{bmatrix} A&B\\ 0&S \end{bmatrix}.

This is a block LU factorization. The first factor is block lower triangular. The second factor is block upper triangular.

One may also write

\begin{bmatrix} A&B\\ C&D \end{bmatrix} = \begin{bmatrix} I&0\\ CA^{-1}&I \end{bmatrix} \begin{bmatrix} A&0\\ 0&S \end{bmatrix} \begin{bmatrix} I&A^{-1}B\\ 0&I \end{bmatrix}.

This separates the matrix into lower, diagonal, and upper block factors.

15.17 Solving Block Systems

Consider the block system

\begin{bmatrix} A&B\\ C&D \end{bmatrix} \begin{bmatrix} x_1\\ x_2 \end{bmatrix} = \begin{bmatrix} b_1\\ b_2 \end{bmatrix}.

Assume $A$ and $S=D-CA^{-1}B$ are invertible.

The solution can be found in two stages.

First solve the Schur complement equation:

Sx_2=b_2-CA^{-1}b_1.

Then recover

x_1=A^{-1}(b_1-Bx_2).

This reduces the original coupled system to a smaller derived system for $x_2$ , followed by back substitution.

15.18 Block Matrices and Linear Maps

Block matrices often arise from decompositions of vector spaces.

Suppose

V=V_1\oplus V_2

and

W=W_1\oplus W_2.

A linear map

T:V\to W

can be represented as

T= \begin{bmatrix} T_{11}&T_{12}\\ T_{21}&T_{22} \end{bmatrix}.

Here

T_{ij}:V_j\to W_i.

The block $T_{12}$ , for example, describes the part of the input from $V_2$ that contributes to the output in $W_1$ .

This interpretation is often clearer than thinking only in coordinates.

15.19 Block Matrices in Computation

Block matrices are important in computation because large matrices are often handled in pieces.

Blocking can improve memory locality, expose parallelism, and separate independent subproblems. Dense numerical linear algebra libraries often use blocked algorithms for matrix multiplication, LU decomposition, QR decomposition, and Cholesky decomposition.

In sparse systems, block structure may come from physical domains, graph partitions, constraints, or multiple variable types. For example, a discretized differential equation may group unknowns by region, field, or boundary condition.

15.20 Common Mistakes

Mistake	Correction
Multiplying blocks without checking sizes	Each block product must be defined
Assuming block multiplication is commutative	Block products are matrix products, so order matters
Treating zero blocks as scalar zero without size	A zero block has a required shape
Forgetting to transpose individual blocks	Block transpose swaps positions and transposes blocks
Applying determinant formulas to arbitrary block matrices	Simple product formulas need triangular or special structure
Using the Schur complement when the chosen block is not invertible	$A^{-1}$ or $D^{-1}$ must exist

15.21 Summary

Block matrices are ordinary matrices viewed through a partition into submatrices. They allow large matrix expressions to be written and manipulated at a higher structural level.

The main rules are:

Concept	Formula
Block addition	$(A+B)_{ij}=A_{ij}+B_{ij}$
Block multiplication	$(AB)_{ij}=\sum_k A_{ik}B_{kj}$
Block transpose	$(A^T)_{ij}=A_{ji}^T$
Direct sum	$A\oplus B=\begin{bmatrix}A&0\\0&B\end{bmatrix}$
Block triangular determinant	$\det\begin{bmatrix}A&B\\0&D\end{bmatrix}=\det(A)\det(D)$
Schur complement	$S=D-CA^{-1}B$
Block diagonal inverse	$\begin{bmatrix}A&0\\0&D\end{bmatrix}^{-1}=\begin{bmatrix}A^{-1}&0\\0&D^{-1}\end{bmatrix}$

Block notation is not a new kind of algebra. It is ordinary matrix algebra applied with a useful partition. Its value is that it makes structure visible.