# Appendix

## ADIFOR

ADIFOR, short for Automatic Differentiation of Fortran, is one of the classical source-transformation systems for automatic differentiation. It was designed for numerical programs written in Fortran, especially scientific computing codes where derivatives were needed for optimization, sensitivity analysis, and parameter estimation.

The central idea of ADIFOR is simple: take an existing Fortran program that computes a function, analyze its source code, and generate a new Fortran program that computes both the original values and selected derivatives.

Given a program for

$$
y = f(x),
$$

ADIFOR generates code for

$$
(y, J) = (f(x), \frac{\partial f}{\partial x}),
$$

or, more commonly in large programs, code for Jacobian-vector products or selected derivative directions.

## Source Transformation Instead of Operator Overloading

ADIFOR belongs to the source-transformation family of AD tools. This means it works by rewriting the program before compilation.

The workflow is roughly:

1. The user writes an ordinary Fortran program.
2. The user identifies independent variables and dependent variables.
3. ADIFOR parses and analyzes the program.
4. ADIFOR emits differentiated Fortran source code.
5. The generated source is compiled by a normal Fortran compiler.

This differs from operator overloading systems, where arithmetic types are replaced by special derivative-carrying types at runtime. ADIFOR instead produces explicit derivative code. That makes the generated program more transparent to compilers and often better suited to traditional high-performance Fortran environments.

For example, suppose the original code contains:

```fortran
z = x * y + sin(x)
```

A forward-mode transformed version may introduce derivative arrays:

```fortran
z = x * y + sin(x)

do i = 1, p
  g_z(i) = g_x(i) * y + x * g_y(i) + cos(x) * g_x(i)
end do
```

Here `p` is the number of derivative directions. The variables `g_x`, `g_y`, and `g_z` store tangent components. The generated code follows the chain rule at the level of elementary program statements.

## Forward Mode Orientation

ADIFOR is primarily associated with forward mode AD. Forward mode propagates derivatives alongside primal values from inputs to outputs. It is especially natural for Fortran programs because many scientific codes already follow a structured, imperative evaluation order.

For a scalar assignment

$$
v = \phi(u_1, u_2, \ldots, u_k),
$$

ADIFOR generates derivative propagation code of the form

$$
\dot v =
\frac{\partial \phi}{\partial u_1}\dot u_1
+
\frac{\partial \phi}{\partial u_2}\dot u_2
+
\cdots
+
\frac{\partial \phi}{\partial u_k}\dot u_k.
$$

If several derivative directions are used, each variable carries a derivative vector rather than a single tangent value. This lets one run several columns of a Jacobian in one execution.

For a function

$$
f : \mathbb{R}^n \to \mathbb{R}^m,
$$

forward mode is efficient when the number of required input directions is small. It is less efficient when the full Jacobian is needed and $n$ is large, since computing all columns usually requires $n$ derivative directions.

## Derivative Arrays

A characteristic feature of ADIFOR-generated code is the use of derivative arrays. Each active variable has a primal value and a corresponding derivative object.

Conceptually:

| Original variable | Derivative variable | Meaning |
|---|---|---|
| `x` | `g_x` | derivative directions for `x` |
| `y` | `g_y` | derivative directions for `y` |
| `z` | `g_z` | derivative directions for `z` |

If `x` is a scalar and `p` derivative directions are propagated, then `g_x` is an array of length `p`. If `x` is already an array, then its derivative object has an additional derivative dimension.

This design makes derivative propagation explicit but also increases memory pressure. For large scientific programs, derivative storage can dominate the cost of the transformed program.

## Activity Analysis

ADIFOR performs activity analysis. A variable is active if its value depends on an independent variable and can influence a dependent variable. Variables outside this dependency path do not need derivative storage or derivative propagation.

For example:

```fortran
a = x * x
b = 3.0
c = a + y
```

If `x` and `y` are independent variables and `c` is the dependent variable, then `a` and `c` are active. The constant `b` is passive unless it participates in an active computation.

Activity analysis matters because naive differentiation would attach derivatives to too many variables. In large Fortran codes, many variables are loop counters, constants, flags, array dimensions, or intermediate values that do not require derivative propagation.

Good activity analysis reduces:

| Cost | Effect of activity analysis |
|---|---|
| Memory use | fewer derivative arrays |
| Runtime | fewer derivative updates |
| Code size | fewer generated derivative statements |
| Compiler burden | smaller transformed programs |

## Handling Fortran Programs

ADIFOR was important because it targeted real scientific Fortran programs, not only small mathematical expressions. This required dealing with subroutines, common blocks, arrays, loops, and procedure calls.

A source transformation tool must decide how derivatives flow across subroutine boundaries. If a subroutine reads active inputs and writes active outputs, the differentiated program needs a corresponding differentiated subroutine.

Original:

```fortran
call compute(x, y)
```

Transformed:

```fortran
call g_compute(g_p, x, g_x, y, g_y)
```

The transformed call passes both primal variables and derivative variables. The exact calling convention depends on the ADIFOR version and configuration, but the principle is consistent: derivative state is made explicit in the generated Fortran interface.

This explicit interface can be intrusive. Existing build systems, libraries, and hand-written Fortran interfaces may need adaptation. But the benefit is that the generated code remains ordinary Fortran.

## Strengths

ADIFOR showed that automatic differentiation could be applied to large existing scientific codes. Its main strengths were practical rather than merely theoretical.

First, it preserved the programming model used by numerical scientists. Users could continue writing Fortran and compile with standard Fortran compilers.

Second, it produced source code that could be inspected, compiled, profiled, and optimized. This was valuable in high-performance computing environments where black-box runtime systems were difficult to trust.

Third, forward-mode source transformation gave accurate derivatives up to floating point rounding. Unlike finite differences, it avoided truncation error and step-size selection.

Fourth, it supported derivative computation through ordinary program structure, including loops and subroutines. This made it suitable for simulation codes where the mathematical function was implicit in thousands of lines of imperative code.

## Limitations

ADIFOR also exposes several limitations of early source-transformation AD.

The first limitation is mode choice. Forward mode is efficient for a small number of input directions. For functions with many inputs and few outputs, reverse mode is usually asymptotically preferable. Many optimization and machine learning problems have exactly that shape.

The second limitation is code growth. Source transformation can generate large derivative programs. Every active assignment may expand into multiple derivative assignments.

The third limitation is language complexity. Fortran programs can contain aliasing, external procedures, common blocks, I/O, mutation, and numerical library calls. A robust AD tool must either transform these correctly, require user annotations, or reject unsupported patterns.

The fourth limitation is derivative storage. If many derivative directions are propagated at once, derivative arrays become large. This affects cache behavior and memory bandwidth.

The fifth limitation is integration friction. Generated code changes interfaces and build artifacts. In large legacy systems, this can be harder to manage than the derivative mathematics itself.

## ADIFOR in the History of AD Systems

ADIFOR occupies an important place in the history of automatic differentiation because it demonstrated that AD could be a program transformation technique for production scientific codes.

It helped establish several ideas that remain central:

| Idea | Later importance |
|---|---|
| Source transformation | used by modern compiler-based AD tools |
| Activity analysis | central to efficient derivative generation |
| Explicit tangent variables | still used in forward-mode IR transformations |
| Differentiated subroutines | required in whole-program AD |
| Generated derivative code | important for auditability and optimization |

Modern systems such as Tapenade, Enzyme, JAX transformations, Zygote, and compiler-integrated AD tools differ greatly in implementation, but they inherit the same fundamental problem: transform a program that computes values into a program that computes derivatives.

ADIFOR is therefore best understood as an early, influential realization of AD as compilation. It took the chain rule out of mathematical notation and embedded it into generated Fortran source.