A conditional is a program construct that chooses one computation among several possible computations. In ordinary code, this is written as if, else, switch, case, pattern...
| Section | Title |
|---|---|
| 1 | Chapter 9. Differentiation of Control Flow |
| 2 | forward |
| 3 | Recursion |
| 4 | Dynamic Graphs |
| 5 | Piecewise Differentiability |
| 6 | Non-Smooth Programs |
| 7 | Differentiating Stateful Systems |
| 8 | Exception Handling and Undefined Regions |
Chapter 9. Differentiation of Control FlowA conditional is a program construct that chooses one computation among several possible computations. In ordinary code, this is written as if, else, switch, case, pattern...
forwardA loop repeats a computation until a condition fails or a fixed iteration count is reached. In automatic differentiation, loops are important because many numerical algorithms...
RecursionRecursion is control flow where a function calls itself. In automatic differentiation, recursion behaves like a loop with a call stack. Each recursive call contributes one...
Dynamic GraphsA dynamic graph is a computation graph built while the program runs. Its structure depends on ordinary runtime values: branches, loop counts, recursive calls, tensor shapes,...
Piecewise DifferentiabilityA piecewise differentiable function is built from several differentiable pieces joined by boundaries. Each piece has an ordinary derivative inside its region. At the...
Non-Smooth ProgramsA non-smooth program contains operations where the derivative is undefined, discontinuous, set-valued, or unstable under small perturbations. These programs arise naturally in...
Differentiating Stateful SystemsA stateful system is a program whose output depends not only on its explicit inputs, but also on stored state. The state may live in variables, objects, arrays, files, random...
Exception Handling and Undefined RegionsPrograms do not only branch between valid computations. They also fail, stop early, raise exceptions, return sentinel values, or enter undefined numerical regions. These...