Source transformation is an implementation strategy for automatic differentiation in which a program that computes a function is rewritten into another program that computes...
| Section | Title |
|---|---|
| 1 | Chapter 11. Compiler and Runtime Design |
| 2 | Operator Overloading |
| 3 | Intermediate Representations |
| 4 | SSA Form |
| 5 | Graph IRs |
| 6 | Tracing Systems |
| 7 | Staging and Partial Evaluation |
| 8 | Memory Planning |
| 9 | Kernel Fusion |
| 10 | Ahead-of-Time vs Just-in-Time Differentiation |
Chapter 11. Compiler and Runtime DesignSource transformation is an implementation strategy for automatic differentiation in which a program that computes a function is rewritten into another program that computes...
Operator OverloadingOperator overloading implements automatic differentiation by changing the meaning of ordinary arithmetic operations for special numeric objects.
Intermediate RepresentationsAn intermediate representation, or IR, is the internal program form used by a compiler or AD system after parsing and before final code generation.
SSA FormStatic single assignment form, or SSA, is an intermediate representation where each variable is assigned exactly once.
Graph IRsA graph intermediate representation models a program as nodes and edges.
Tracing SystemsTracing is an implementation strategy where an AD system observes a program while it runs and records the operations that occur.
Staging and Partial EvaluationStaging is the separation of a program into phases.
Memory PlanningMemory planning determines where values are stored, how long they remain alive, and when storage can be reused.
Kernel FusionKernel fusion combines several small operations into one larger executable unit.
Ahead-of-Time vs Just-in-Time DifferentiationAutomatic differentiation can be performed before a program runs, while it runs, or in a staged phase between the two.