Chapter 46. Norms and Metrics

A norm is a rule that assigns a length to each vector. A metric is a rule that assigns a distance to each pair of points. In linear algebra, these two ideas are closely related: a norm gives a distance by measuring the length of a difference,

Thus a normed vector space is both an algebraic object and a geometric object. The vector operations give addition and scalar multiplication. The norm and metric give length, distance, convergence, continuity, and approximation. A normed vector space is a vector space equipped with a norm, and the norm-induced distance makes it a metric space.

46.1 Norms

Let $V$ be a vector space over $\mathbb{R}$ or $\mathbb{C}$. A norm on $V$ is a function

satisfying the following properties for all $u,v \in V$ and all scalars $c$.

First, nonnegativity:

Second, positive definiteness:

Third, absolute homogeneity:

Fourth, the triangle inequality:

These axioms abstract the basic behavior of length. Length is never negative. Only the zero vector has zero length. Scaling a vector scales its length by the absolute value of the scalar. The length of a sum is at most the sum of the lengths.

46.2 The Euclidean Norm

The standard norm on $\mathbb{R}^n$ is the Euclidean norm:

For example, if

then

The Euclidean norm is the norm induced by the standard inner product:

In $\mathbb{R}^2$ and $\mathbb{R}^3$, this is the usual geometric length. In higher dimensions, the same formula remains valid, although direct geometric visualization becomes harder.

46.3 Norms Induced by Inner Products

Every inner product defines a norm by

This construction is fundamental. It shows that inner product spaces are normed vector spaces.

The induced norm satisfies the norm axioms. Positive definiteness follows from the positive definiteness of the inner product. Absolute homogeneity follows from

The triangle inequality follows from the Cauchy-Schwarz inequality.

However, not every norm comes from an inner product. A norm comes from an inner product precisely when it satisfies the parallelogram law:

This distinction matters. Inner product geometry has angles and orthogonality. General normed geometry has length and distance, but it may lack a compatible notion of angle.

46.4 The $p$-Norms

For $1 \le p < \infty$, the $p$-norm on $\mathbb{R}^n$ is

Important special cases include:

The infinity norm is defined by

For example, if

then

Each norm measures size differently. The $1$-norm measures total absolute magnitude. The $2$-norm measures Euclidean length. The infinity norm measures the largest coordinate magnitude.

46.5 Unit Balls

The unit ball of a normed vector space is the set

Different norms produce different unit balls.

In $\mathbb{R}^2$, the Euclidean unit ball is a disk. The $1$-norm unit ball is a diamond:

The infinity norm unit ball is a square:

The shape of the unit ball determines the geometry of the norm. Smooth unit balls behave differently from polygonal unit balls. This affects optimization, approximation, sparsity, and numerical algorithms.

46.6 Metrics

A metric on a set $X$ is a function

satisfying the following properties for all $x,y,z \in X$.

First, nonnegativity:

Second, identity of indiscernibles:

Third, symmetry:

Fourth, the triangle inequality:

A set equipped with a metric is called a metric space.

A metric measures distance. Unlike a norm, it does not require vector addition or scalar multiplication. Therefore metrics apply to more general objects: points, functions, sequences, graphs, probability distributions, and many other sets.

46.7 Metrics Induced by Norms

Every norm on a vector space induces a metric by

This metric is translation invariant:

It is also homogeneous:

These properties reflect the linear structure of the space. The distance between two vectors depends only on their difference.

For example, in $\mathbb{R}^n$, the Euclidean metric is

The $1$-metric is

The infinity metric is

Norm-induced metrics are the main metrics used in finite-dimensional linear algebra.

46.8 Distance and Approximation

Norms are used to compare an exact object with an approximation.

If $x$ is the exact vector and $\hat{x}$ is an approximation, the error vector is

The size of the error is measured by

Different norms emphasize different error behavior.

The $2$-norm measures root-sum-square error. The $1$-norm measures total absolute error. The infinity norm measures worst-coordinate error.

For example, if

then

The choice of norm depends on the problem. If total error matters, use a sum-type norm. If worst-case error matters, use the infinity norm. If Euclidean geometry matters, use the $2$-norm.

46.9 Convergence

A sequence of vectors $(x_k)$ in a normed vector space converges to $x$ if

This means the distance from $x_k$ to $x$ becomes arbitrarily small.

Convergence depends on the norm. In finite-dimensional spaces, all norms define the same convergent sequences. In infinite-dimensional spaces, different norms may produce different notions of convergence.

For example, in $\mathbb{R}^n$, convergence in the Euclidean norm is equivalent to coordinatewise convergence. If

then

if and only if

for every coordinate $i$.

46.10 Cauchy Sequences and Completeness

A sequence $(x_k)$ is Cauchy if for every $\varepsilon>0$, there exists an integer $N$ such that

whenever $m,n \ge N$.

A normed vector space is complete if every Cauchy sequence converges to a vector in the space.

A complete normed vector space is called a Banach space. This concept becomes central in functional analysis, numerical analysis, and differential equations. A Banach space is, by definition, a complete normed space with respect to the metric induced by its norm.

Finite-dimensional normed spaces over $\mathbb{R}$ or $\mathbb{C}$ are complete. Infinite-dimensional spaces require more care.

46.11 Equivalence of Norms in Finite Dimensions

On a finite-dimensional vector space, any two norms are equivalent.

This means that if $\|\cdot\|_a$ and $\|\cdot\|_b$ are two norms on the same finite-dimensional space, then there exist constants $C_1,C_2>0$ such that

for all vectors $x$.

Equivalence of norms means that all norms give the same topology in finite dimensions. They have the same convergent sequences, open sets, closed sets, and continuous functions.

This fact justifies flexible norm choice in many finite-dimensional arguments. One may use the norm that makes the proof or computation simplest.

46.12 Open and Closed Balls

Let $(X,d)$ be a metric space. The open ball centered at $x$ with radius $r>0$ is

The closed ball is

In a normed vector space, these become

and

Balls describe local neighborhoods. They are the basic objects used to define limits, continuity, compactness, and approximation.

46.13 Continuity

A function $f : V \to W$ between normed vector spaces is continuous at $x$ if for every $\varepsilon>0$, there exists $\delta>0$ such that

Continuity means small changes in the input produce small changes in the output.

Linear maps have a particularly simple continuity theory. A linear map $T : V \to W$ is continuous if and only if there exists a constant $C\ge 0$ such that

for all $v \in V$.

In finite-dimensional spaces, every linear map is continuous. In infinite-dimensional spaces, some linear maps may fail to be continuous.

46.14 Operator Norms

Let $T : V \to W$ be a linear map between normed vector spaces. The operator norm of $T$ is

Equivalently,

The operator norm measures the maximum amount by which $T$ can stretch a vector.

For a matrix $A$, the operator norm depends on the vector norm chosen. With the Euclidean norm, the induced matrix norm is

This is the largest singular value of $A$.

Operator norms are essential in perturbation theory, numerical stability, convergence estimates, and matrix analysis.

46.15 Matrix Norms

A matrix norm is a norm on a vector space of matrices. Some matrix norms are induced by vector norms. Others are defined directly from entries.

The Frobenius norm is

The induced $1$-norm is the maximum absolute column sum:

The induced infinity norm is the maximum absolute row sum:

These norms measure matrix size in different ways.

The Frobenius norm treats the matrix as a vector of entries. The induced norms measure worst-case amplification of vectors.

46.16 Lipschitz Bounds

A function $f : V \to W$ is Lipschitz continuous if there exists a constant $L\ge 0$ such that

for all $u,v \in V$.

For a linear map $T$,

Thus every bounded linear map is Lipschitz continuous, with Lipschitz constant $\|T\|$.

This gives a direct interpretation of the operator norm: it is the best global Lipschitz constant of the linear map.

46.17 Relative Error

In numerical work, absolute error is often measured by

Relative error is measured by

assuming $x \ne 0$.

Relative error compares the error to the size of the exact object. It is more informative when values have different scales.

For example, an error of $10^{-3}$ is small if the exact value has norm $10^3$, but large if the exact value has norm $10^{-6}$.

Norms therefore provide the language for quantitative accuracy.

46.18 Norms and Conditioning

The condition number of an invertible matrix $A$, with respect to a chosen norm, is

It measures how sensitive the solution of

is to perturbations in $b$ and $A$.

If $\kappa(A)$ is small, the system is well-conditioned. If $\kappa(A)$ is large, small input errors may cause large output errors.

The value of $\kappa(A)$ depends on the norm. With the Euclidean norm,

where $\sigma_{\max}(A)$ and $\sigma_{\min}(A)$ are the largest and smallest singular values of $A$.

Conditioning is one of the main reasons norms matter in numerical linear algebra.

46.19 Examples

For

the common vector norms are

and

For

the Frobenius norm is

The induced $1$-norm is

The induced infinity norm is

These computations show that matrix size depends on the chosen norm.

46.20 Summary

A norm measures the length of a vector. A metric measures the distance between two points. Every norm induces a metric by

Norms support the study of convergence, continuity, approximation, error, conditioning, and stability. Inner products produce important norms, especially the Euclidean norm, but many useful norms do not come from inner products.

The choice of norm should match the structure of the problem. Euclidean norms fit geometric and spectral arguments. The $1$-norm fits total magnitude and sparsity. The infinity norm fits worst-case control. Matrix and operator norms describe the size and amplification behavior of linear transformations.