Dual numbers give the cleanest algebraic model of forward mode automatic differentiation. They extend ordinary real numbers with a formal infinitesimal part. Instead of...
| Section | Title |
|---|---|
| 1 | Chapter 7. Dual Numbers and Algebraic Structures |
| 2 | Nilpotent Elements |
| 3 | Geometric Interpretation |
| 4 | Truncated Polynomial Algebras |
| 5 | Hyper-Dual Numbers |
| 6 | Differential Algebras |
| 7 | Category-Theoretic View |
| 8 | Differential Lambda Calculus |
| 9 | Differential Lambda Calculus |
Chapter 7. Dual Numbers and Algebraic StructuresDual numbers give the cleanest algebraic model of forward mode automatic differentiation. They extend ordinary real numbers with a formal infinitesimal part. Instead of...
Nilpotent ElementsThe defining feature of dual numbers is the existence of a nonzero element whose square vanishes:
Geometric InterpretationDual numbers provide an algebraic mechanism for differentiation, but they also have a precise geometric meaning. A dual number represents a point together with an...
Truncated Polynomial AlgebrasDual numbers capture first-order derivatives because the infinitesimal element satisfies
Hyper-Dual NumbersDual numbers compute first derivatives exactly. Truncated polynomial algebras extend this to higher-order derivatives, but practical higher-order differentiation introduces an...
Differential AlgebrasDual numbers and hyper-dual numbers are special cases of a broader algebraic structure called a differential algebra. This framework abstracts differentiation away from...
Category-Theoretic ViewAutomatic differentiation can be described operationally through dual numbers and computational graphs. It can also be described abstractly using category theory.
Differential Lambda CalculusAutomatic differentiation is deeply connected to functional programming and lambda calculus. Programs can be viewed as mathematical functions, and differentiation can be...
Differential Lambda CalculusAutomatic differentiation is deeply connected to functional programming and lambda calculus. Programs can be viewed as mathematical functions, and differentiation can be...