Lisp is one of the natural homes of automatic differentiation. It treats programs as data, has a simple expression syntax, and supports macro systems that can transform code...
| Section | Title |
|---|---|
| 1 | Chapter 12. AD in Modern Programming Languages |
| 2 | Functional Languages |
| 3 | Differentiable Programming |
| 4 | AD in C and C++ |
| 5 | AD in Python |
| 6 | AD in Rust |
| 7 | AD in Julia |
| 8 | AD in Swift |
| 9 | Type Systems for Differentiation |
| 10 | Effect Systems and Mutation |
Chapter 12. AD in Modern Programming LanguagesLisp is one of the natural homes of automatic differentiation. It treats programs as data, has a simple expression syntax, and supports macro systems that can transform code...
Functional LanguagesFunctional programming languages provide a natural semantic foundation for automatic differentiation. Programs are expressed as compositions of functions, immutable values,...
Differentiable ProgrammingDifferentiable programming treats differentiation as a general programming-language feature. A program can contain numerical kernels, control flow, data structures, solvers,...
AD in C and C++C and C++ are important targets for automatic differentiation because much scientific, engineering, graphics, finance, and machine learning infrastructure is written in these...
AD in PythonPython became the dominant language for modern machine learning and differentiable computing because it combines a simple programming model with access to high-performance...
AD in RustRust is an attractive language for automatic differentiation because it combines low-level performance with strong static guarantees. It gives the programmer control over...
AD in JuliaJulia was designed for high-performance technical computing. It combines interactive syntax with a compiler capable of specializing code aggressively based on types. This...
AD in SwiftSwift became an important experiment in language-integrated automatic differentiation because it attempted to make differentiation a core compiler feature rather than a...
Type Systems for DifferentiationAutomatic differentiation interacts deeply with type systems because differentiation changes the structure of computation. A derivative operator maps one function into another...
Effect Systems and MutationAutomatic differentiation is easiest to define for pure functions. A pure function behaves like a mathematical mapping: it consumes inputs, produces outputs, and has no...