# Chapter 46. Norms and Metrics # Chapter 46. Norms and Metrics A norm is a rule that assigns a length to each vector. A metric is a rule that assigns a distance to each pair of points. In linear algebra, these two ideas are closely related: a norm gives a distance by measuring the length of a difference, $$ d(u,v) = \|u-v\|. $$ Thus a normed vector space is both an algebraic object and a geometric object. The vector operations give addition and scalar multiplication. The norm and metric give length, distance, convergence, continuity, and approximation. A normed vector space is a vector space equipped with a norm, and the norm-induced distance makes it a metric space. ## 46.1 Norms Let \(V\) be a vector space over \(\mathbb{R}\) or \(\mathbb{C}\). A norm on \(V\) is a function $$ \|\cdot\| : V \to \mathbb{R} $$ satisfying the following properties for all \(u,v \in V\) and all scalars \(c\). First, nonnegativity: $$ \|v\| \ge 0. $$ Second, positive definiteness: $$ \|v\| = 0 \quad \text{if and only if} \quad v = 0. $$ Third, absolute homogeneity: $$ \|cv\| = |c|\|v\|. $$ Fourth, the triangle inequality: $$ \|u+v\| \le \|u\| + \|v\|. $$ These axioms abstract the basic behavior of length. Length is never negative. Only the zero vector has zero length. Scaling a vector scales its length by the absolute value of the scalar. The length of a sum is at most the sum of the lengths. ## 46.2 The Euclidean Norm The standard norm on \(\mathbb{R}^n\) is the Euclidean norm: $$ \|x\|_2 = \sqrt{x_1^2+x_2^2+\cdots+x_n^2}. $$ For example, if $$ x = \begin{bmatrix} 3 \\ 4 \end{bmatrix}, $$ then $$ \|x\|_2 = \sqrt{3^2+4^2}=5. $$ The Euclidean norm is the norm induced by the standard inner product: $$ \|x\|_2 = \sqrt{\langle x,x\rangle}. $$ In \(\mathbb{R}^2\) and \(\mathbb{R}^3\), this is the usual geometric length. In higher dimensions, the same formula remains valid, although direct geometric visualization becomes harder. ## 46.3 Norms Induced by Inner Products Every inner product defines a norm by $$ \|v\| = \sqrt{\langle v,v\rangle}. $$ This construction is fundamental. It shows that inner product spaces are normed vector spaces. The induced norm satisfies the norm axioms. Positive definiteness follows from the positive definiteness of the inner product. Absolute homogeneity follows from $$ \|cv\| = \sqrt{\langle cv,cv\rangle} = \sqrt{|c|^2\langle v,v\rangle} = |c|\|v\|. $$ The triangle inequality follows from the Cauchy-Schwarz inequality. However, not every norm comes from an inner product. A norm comes from an inner product precisely when it satisfies the parallelogram law: $$ \|u+v\|^2+\|u-v\|^2 = 2\|u\|^2+2\|v\|^2. $$ This distinction matters. Inner product geometry has angles and orthogonality. General normed geometry has length and distance, but it may lack a compatible notion of angle. ## 46.4 The \(p\)-Norms For \(1 \le p < \infty\), the \(p\)-norm on \(\mathbb{R}^n\) is $$ \|x\|_p = \left( |x_1|^p + |x_2|^p + \cdots + |x_n|^p \right)^{1/p}. $$ Important special cases include: | Norm | Formula | Common name | |---|---|---| | \(p=1\) | \(\|x\|_1=\sum_i |x_i|\) | Manhattan norm | | \(p=2\) | \(\|x\|_2=(\sum_i x_i^2)^{1/2}\) | Euclidean norm | | \(p=\infty\) | \(\|x\|_\infty=\max_i |x_i|\) | Maximum norm | The infinity norm is defined by $$ \|x\|_\infty = \max_{1 \le i \le n} |x_i|. $$ For example, if $$ x = \begin{bmatrix} -2 \\ 5 \\ 3 \end{bmatrix}, $$ then $$ \|x\|_1 = 10, \qquad \|x\|_2 = \sqrt{38}, \qquad \|x\|_\infty = 5. $$ Each norm measures size differently. The \(1\)-norm measures total absolute magnitude. The \(2\)-norm measures Euclidean length. The infinity norm measures the largest coordinate magnitude. ## 46.5 Unit Balls The unit ball of a normed vector space is the set $$ B = \{x \in V : \|x\| \le 1\}. $$ Different norms produce different unit balls. In \(\mathbb{R}^2\), the Euclidean unit ball is a disk. The \(1\)-norm unit ball is a diamond: $$ |x_1| + |x_2| \le 1. $$ The infinity norm unit ball is a square: $$ \max(|x_1|,|x_2|) \le 1. $$ The shape of the unit ball determines the geometry of the norm. Smooth unit balls behave differently from polygonal unit balls. This affects optimization, approximation, sparsity, and numerical algorithms. ## 46.6 Metrics A metric on a set \(X\) is a function $$ d : X \times X \to \mathbb{R} $$ satisfying the following properties for all \(x,y,z \in X\). First, nonnegativity: $$ d(x,y) \ge 0. $$ Second, identity of indiscernibles: $$ d(x,y)=0 \quad \text{if and only if} \quad x=y. $$ Third, symmetry: $$ d(x,y)=d(y,x). $$ Fourth, the triangle inequality: $$ d(x,z) \le d(x,y)+d(y,z). $$ A set equipped with a metric is called a metric space. A metric measures distance. Unlike a norm, it does not require vector addition or scalar multiplication. Therefore metrics apply to more general objects: points, functions, sequences, graphs, probability distributions, and many other sets. ## 46.7 Metrics Induced by Norms Every norm on a vector space induces a metric by $$ d(u,v)=\|u-v\|. $$ This metric is translation invariant: $$ d(u+a,v+a)=d(u,v). $$ It is also homogeneous: $$ d(cu,cv)=|c|d(u,v). $$ These properties reflect the linear structure of the space. The distance between two vectors depends only on their difference. For example, in \(\mathbb{R}^n\), the Euclidean metric is $$ d_2(u,v) = \|u-v\|_2 = \sqrt{(u_1-v_1)^2+\cdots+(u_n-v_n)^2}. $$ The \(1\)-metric is $$ d_1(u,v) = \|u-v\|_1 = \sum_{i=1}^n |u_i-v_i|. $$ The infinity metric is $$ d_\infty(u,v) = \|u-v\|_\infty = \max_i |u_i-v_i|. $$ Norm-induced metrics are the main metrics used in finite-dimensional linear algebra. ## 46.8 Distance and Approximation Norms are used to compare an exact object with an approximation. If \(x\) is the exact vector and \(\hat{x}\) is an approximation, the error vector is $$ e = x-\hat{x}. $$ The size of the error is measured by $$ \|e\|. $$ Different norms emphasize different error behavior. The \(2\)-norm measures root-sum-square error. The \(1\)-norm measures total absolute error. The infinity norm measures worst-coordinate error. For example, if $$ e = \begin{bmatrix} 0.01 \\ -0.03 \\ 0.02 \end{bmatrix}, $$ then $$ \|e\|_1 = 0.06, \qquad \|e\|_2 = \sqrt{0.0014}, \qquad \|e\|_\infty = 0.03. $$ The choice of norm depends on the problem. If total error matters, use a sum-type norm. If worst-case error matters, use the infinity norm. If Euclidean geometry matters, use the \(2\)-norm. ## 46.9 Convergence A sequence of vectors \((x_k)\) in a normed vector space converges to \(x\) if $$ \lim_{k\to\infty}\|x_k-x\| = 0. $$ This means the distance from \(x_k\) to \(x\) becomes arbitrarily small. Convergence depends on the norm. In finite-dimensional spaces, all norms define the same convergent sequences. In infinite-dimensional spaces, different norms may produce different notions of convergence. For example, in \(\mathbb{R}^n\), convergence in the Euclidean norm is equivalent to coordinatewise convergence. If $$ x_k = \begin{bmatrix} x_{k1} \\ \vdots \\ x_{kn} \end{bmatrix}, $$ then $$ x_k \to x $$ if and only if $$ x_{ki} \to x_i $$ for every coordinate \(i\). ## 46.10 Cauchy Sequences and Completeness A sequence \((x_k)\) is Cauchy if for every \(\varepsilon>0\), there exists an integer \(N\) such that $$ \|x_m-x_n\| < \varepsilon $$ whenever \(m,n \ge N\). A normed vector space is complete if every Cauchy sequence converges to a vector in the space. A complete normed vector space is called a Banach space. This concept becomes central in functional analysis, numerical analysis, and differential equations. A Banach space is, by definition, a complete normed space with respect to the metric induced by its norm. Finite-dimensional normed spaces over \(\mathbb{R}\) or \(\mathbb{C}\) are complete. Infinite-dimensional spaces require more care. ## 46.11 Equivalence of Norms in Finite Dimensions On a finite-dimensional vector space, any two norms are equivalent. This means that if \(\|\cdot\|_a\) and \(\|\cdot\|_b\) are two norms on the same finite-dimensional space, then there exist constants \(C_1,C_2>0\) such that $$ C_1\|x\|_a \le \|x\|_b \le C_2\|x\|_a $$ for all vectors \(x\). Equivalence of norms means that all norms give the same topology in finite dimensions. They have the same convergent sequences, open sets, closed sets, and continuous functions. This fact justifies flexible norm choice in many finite-dimensional arguments. One may use the norm that makes the proof or computation simplest. ## 46.12 Open and Closed Balls Let \((X,d)\) be a metric space. The open ball centered at \(x\) with radius \(r>0\) is $$ B_r(x)=\{y \in X : d(x,y)0\), there exists \(\delta>0\) such that $$ \|y-x\|_V < \delta \quad \Longrightarrow \quad \|f(y)-f(x)\|_W < \varepsilon. $$ Continuity means small changes in the input produce small changes in the output. Linear maps have a particularly simple continuity theory. A linear map \(T : V \to W\) is continuous if and only if there exists a constant \(C\ge 0\) such that $$ \|T(v)\|_W \le C\|v\|_V $$ for all \(v \in V\). In finite-dimensional spaces, every linear map is continuous. In infinite-dimensional spaces, some linear maps may fail to be continuous. ## 46.14 Operator Norms Let \(T : V \to W\) be a linear map between normed vector spaces. The operator norm of \(T\) is $$ \|T\| = \sup_{v\ne 0} \frac{\|T(v)\|_W}{\|v\|_V}. $$ Equivalently, $$ \|T\| = \sup_{\|v\|_V=1}\|T(v)\|_W. $$ The operator norm measures the maximum amount by which \(T\) can stretch a vector. For a matrix \(A\), the operator norm depends on the vector norm chosen. With the Euclidean norm, the induced matrix norm is $$ \|A\|_2 = \sup_{x\ne 0} \frac{\|Ax\|_2}{\|x\|_2}. $$ This is the largest singular value of \(A\). Operator norms are essential in perturbation theory, numerical stability, convergence estimates, and matrix analysis. ## 46.15 Matrix Norms A matrix norm is a norm on a vector space of matrices. Some matrix norms are induced by vector norms. Others are defined directly from entries. The Frobenius norm is $$ \|A\|_F = \sqrt{ \sum_{i=1}^m\sum_{j=1}^n |a_{ij}|^2 }. $$ The induced \(1\)-norm is the maximum absolute column sum: $$ \|A\|_1 = \max_j \sum_i |a_{ij}|. $$ The induced infinity norm is the maximum absolute row sum: $$ \|A\|_\infty = \max_i \sum_j |a_{ij}|. $$ These norms measure matrix size in different ways. The Frobenius norm treats the matrix as a vector of entries. The induced norms measure worst-case amplification of vectors. ## 46.16 Lipschitz Bounds A function \(f : V \to W\) is Lipschitz continuous if there exists a constant \(L\ge 0\) such that $$ \|f(u)-f(v)\|_W \le L\|u-v\|_V $$ for all \(u,v \in V\). For a linear map \(T\), $$ \|T(u)-T(v)\| = \|T(u-v)\| \le \|T\|\|u-v\|. $$ Thus every bounded linear map is Lipschitz continuous, with Lipschitz constant \(\|T\|\). This gives a direct interpretation of the operator norm: it is the best global Lipschitz constant of the linear map. ## 46.17 Relative Error In numerical work, absolute error is often measured by $$ \|x-\hat{x}\|. $$ Relative error is measured by $$ \frac{\|x-\hat{x}\|}{\|x\|}, $$ assuming \(x \ne 0\). Relative error compares the error to the size of the exact object. It is more informative when values have different scales. For example, an error of \(10^{-3}\) is small if the exact value has norm \(10^3\), but large if the exact value has norm \(10^{-6}\). Norms therefore provide the language for quantitative accuracy. ## 46.18 Norms and Conditioning The condition number of an invertible matrix \(A\), with respect to a chosen norm, is $$ \kappa(A)=\|A\|\|A^{-1}\|. $$ It measures how sensitive the solution of $$ Ax=b $$ is to perturbations in \(b\) and \(A\). If \(\kappa(A)\) is small, the system is well-conditioned. If \(\kappa(A)\) is large, small input errors may cause large output errors. The value of \(\kappa(A)\) depends on the norm. With the Euclidean norm, $$ \kappa_2(A) = \frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}, $$ where \(\sigma_{\max}(A)\) and \(\sigma_{\min}(A)\) are the largest and smallest singular values of \(A\). Conditioning is one of the main reasons norms matter in numerical linear algebra. ## 46.19 Examples For $$ x = \begin{bmatrix} 1 \\ -2 \\ 2 \end{bmatrix}, $$ the common vector norms are $$ \|x\|_1 = 5, $$ $$ \|x\|_2 = 3, $$ and $$ \|x\|_\infty = 2. $$ For $$ A = \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix}, $$ the Frobenius norm is $$ \|A\|_F = \sqrt{1^2+(-2)^2+3^2+4^2} = \sqrt{30}. $$ The induced \(1\)-norm is $$ \|A\|_1 = \max\{ |1|+|3|,\ |-2|+|4| \} = \max\{4,6\} = 6. $$ The induced infinity norm is $$ \|A\|_\infty = \max\{ |1|+|-2|,\ |3|+|4| \} = \max\{3,7\} = 7. $$ These computations show that matrix size depends on the chosen norm. ## 46.20 Summary A norm measures the length of a vector. A metric measures the distance between two points. Every norm induces a metric by $$ d(u,v)=\|u-v\|. $$ Norms support the study of convergence, continuity, approximation, error, conditioning, and stability. Inner products produce important norms, especially the Euclidean norm, but many useful norms do not come from inner products. The choice of norm should match the structure of the problem. Euclidean norms fit geometric and spectral arguments. The \(1\)-norm fits total magnitude and sparsity. The infinity norm fits worst-case control. Matrix and operator norms describe the size and amplification behavior of linear transformations.