# Tapenade ## Tapenade Tapenade is a source-transformation automatic differentiation system developed at INRIA. Like ADIFOR, it takes an existing program and produces a new differentiated program. Its main targets are Fortran and C, with strong emphasis on scientific and engineering codes. Tapenade is historically important because it pushed source-transformation AD beyond simple forward propagation. It supports both tangent mode and adjoint mode, making it useful for sensitivity analysis, inverse problems, and large simulations where the number of inputs is much larger than the number of outputs. ## Tangent and Adjoint Transformation Tapenade uses two main transformations. Tangent mode computes directional derivatives. Given $$ y = f(x), $$ it propagates perturbations forward: $$ \dot y = J_f(x)\dot x. $$ This is the same basic idea as forward mode. Each active variable has a primal value and a tangent value. The transformed program follows the original execution order. Adjoint mode computes gradients by propagating sensitivities backward. For a scalar objective $$ L = f(x), $$ Tapenade generates code that accumulates $$ \bar x = \frac{\partial L}{\partial x}. $$ For a statement such as ```fortran z = x * y ``` the adjoint transformation conceptually performs: ```fortran bar_x = bar_x + bar_z * y bar_y = bar_y + bar_z * x ``` The adjoint of `z` is distributed backward to the adjoints of `x` and `y`. ## Program Reversal Reverse mode requires more than reversing arithmetic rules. The transformed program must run backward through the original computation. This creates a systems problem: values needed during the backward sweep may no longer be available. Tapenade handles this using program reversal. The generated adjoint code usually has: | Phase | Role | |---|---| | Forward sweep | computes primal values and records needed intermediate state | | Backward sweep | propagates adjoints in reverse execution order | | Storage management | saves or recomputes values required by the backward pass | For straight-line code, reversal is simple. For loops, conditionals, procedure calls, mutation, and arrays, reversal becomes more complex. The system must preserve enough information to replay the control path and recover overwritten values. ## Activity and Usefulness Analysis Tapenade performs static analyses to reduce derivative work. Activity analysis identifies variables that depend on independent variables and influence dependent variables. Passive variables do not need tangent or adjoint storage. Usefulness analysis goes further. Some active values may affect only computations whose derivatives are irrelevant to the requested output. Those values can sometimes be omitted from derivative propagation. These analyses reduce code size, memory traffic, and runtime. In large simulation codes, this distinction is material. A naive transformation can make the differentiated program several times larger and slower than necessary. ## Handling Arrays and Procedures Tapenade is designed for real procedural programs. It handles arrays, subroutines, functions, and call graphs. When a procedure is active, Tapenade generates a differentiated version of that procedure. Tangent and adjoint variables are passed through the transformed interface. Original call: ```fortran call step(state, params) ``` Tangent-style call: ```fortran call step_d(state, stated, params, paramsd) ``` Adjoint-style call: ```fortran call step_b(state, stateb, params, paramsb) ``` The naming convention varies by configuration, but the core idea is stable: the derivative program has an expanded interface containing primal variables and derivative variables. Procedure transformation is one reason source-transformation AD is powerful. It can differentiate across abstraction boundaries. It is also one reason these systems are operationally complex. The tool must analyze the call graph and produce consistent derivative interfaces. ## Checkpointing Adjoint mode often needs many intermediate values from the forward execution. Storing every value can exceed memory capacity. Recomputing everything can be too slow. Tapenade supports checkpointing strategies. A checkpoint stores selected program states during the forward sweep. During the backward sweep, missing intermediate values are recomputed from these checkpoints. The tradeoff is direct: | Strategy | Memory | Runtime | |---|---:|---:| | Store everything | high | low | | Recompute everything | low | high | | Checkpoint selectively | moderate | moderate | Checkpointing is central in adjoint differentiation of long time-stepping simulations, such as fluid dynamics, weather models, and differential equation solvers. ## Strengths Tapenade’s main strength is whole-program source transformation. It can operate on large procedural codes where derivatives are needed but manual differentiation would be error-prone and expensive. It also supports both tangent and adjoint modes. This makes it more flexible than systems focused only on forward mode. Its generated code remains ordinary Fortran or C. This is valuable when the target environment depends on existing compilers, numerical libraries, and HPC build systems. Tapenade also exposes the derivative program. Engineers can inspect, profile, and sometimes manually tune the generated code. ## Limitations Tapenade inherits the usual difficulties of source-transformation AD. The transformed code can be large. Procedure interfaces can become more complex. Build systems may need adjustment. External libraries require derivative definitions, wrappers, or user intervention. Mutation and aliasing complicate correctness. If a variable is overwritten in the primal program, the adjoint program may need the old value. If two references alias the same memory, the transformation must respect that aliasing. Adjoint mode also introduces control-flow recording. Branch choices, loop counts, and overwritten values may need to be stored. For large programs, the resulting tape or checkpoint structure can dominate runtime behavior. ## Historical Role Tapenade represents the mature source-transformation line of AD systems. ADIFOR showed that Fortran source transformation could work at scale. Tapenade broadened the model with robust tangent and adjoint generation for Fortran and C programs. Its design also makes clear why automatic differentiation is partly a compiler problem. Correct AD for real programs requires parsing, static analysis, call-graph transformation, data-flow reasoning, alias handling, memory planning, and code generation. Tapenade is therefore an important bridge between classical scientific computing AD and modern compiler-based differentiable programming.