A straight-line program is the simplest model of computation used in automatic differentiation. It is a program with a fixed sequence of assignments, no branches, no loops,...
| Section | Title |
|---|---|
| 1 | Chapter 3. Programs as Mathematical Objects |
| 2 | Intermediate Variables |
| 3 | Dependency Graphs |
| 4 | Control Flow |
| 5 | Loops and Recurrence Relations |
| 6 | Memory and State |
| 7 | Purity and Side Effects |
| 8 | Differentiable Subprograms |
Chapter 3. Programs as Mathematical ObjectsA straight-line program is the simplest model of computation used in automatic differentiation. It is a program with a fixed sequence of assignments, no branches, no loops,...
Intermediate VariablesIntermediate variables are the named values created between program inputs and program outputs. They make automatic differentiation mechanical.
Dependency GraphsA dependency graph describes how values in a computation depend on earlier values. Automatic differentiation operates on these dependencies.
Control FlowControl flow determines which operations a program executes. Straight-line programs have a fixed sequence of operations, but ordinary programs contain branches, loops,...
Loops and Recurrence RelationsLoops express repeated computation. Recurrence relations express the same idea mathematically: each state is computed from one or more earlier states.
Memory and StateAutomatic differentiation operates on computations, but computations execute inside a memory model. Variables occupy storage locations, arrays are mutated, buffers are reused,...
Purity and Side EffectsA pure computation is easier to differentiate because every output is determined only by its explicit inputs. There is no hidden state, no external mutation, and no dependence...
Differentiable SubprogramsA differentiable subprogram is a program fragment that can participate in derivative propagation as a coherent unit. Instead of differentiating an entire application...