| Section | Title |
|---|---|
| 1 | Chapter 19. Theory and Foundations |
| 2 | Algebraic Semantics |
| 3 | Categorical Semantics |
| 4 | Differential Categories |
| 5 | Lambda Calculus and AD |
| 6 | Program Equivalence |
| 7 | Formal Verification |
| 8 | Denotational Models |
| 9 | Differentiation as Functorial Transformation |
Chapter 19. Theory and FoundationsAutomatic differentiation is often described by a simple rule:
Algebraic SemanticsAutomatic differentiation is often introduced operationally. A program executes elementary operations, and derivative information propagates alongside the computation. This...
Categorical SemanticsAlgebraic semantics describes differentiation through derivations, tangent maps, and linear structure. Categorical semantics goes further. It studies differentiation as a...
Differential CategoriesCartesian differential categories model differentiation in categories with products. Differential categories generalize this idea further by shifting attention from cartesian...
Lambda Calculus and ADAutomatic differentiation becomes substantially more difficult once programs contain higher-order functions.
Program EquivalenceAutomatic differentiation transforms programs. A fundamental semantic question therefore arises:
Formal VerificationAutomatic differentiation systems are trusted infrastructure. Scientific computing, machine learning, optimization, simulation, and control systems depend on gradients being...
Denotational ModelsOperational semantics explains how automatic differentiation executes. Denotational semantics explains what differentiable programs mean.
Differentiation as Functorial TransformationThe preceding sections described automatic differentiation through algebraic, categorical, logical, and denotational models. These viewpoints converge on one central idea: