# Differentiable Physics Engines

## Differentiable Physics Engines

A differentiable physics engine computes gradients of physical simulation outputs with respect to inputs, parameters, or control signals. Instead of treating simulation as a terminal forward process, the simulator becomes part of the computational graph.

The system computes:

$$
x_{t+1} = f(x_t, u_t, \theta)
$$

where:

| Symbol | Meaning |
|---|---|
| $x_t$ | System state |
| $u_t$ | Control input |
| $\theta$ | Physical parameters |
| $f$ | Physics transition operator |

A loss over trajectories:

$$
L(x_0, x_1, \ldots, x_T)
$$

can then be differentiated with respect to:

- initial conditions
- controller parameters
- forces
- geometry
- masses
- friction coefficients
- material constants

Automatic differentiation propagates gradients backward through the simulation itself.

### Physics Simulation as Computation

A classical simulator performs:

```text
initial state
  -> timestep integration
  -> collision handling
  -> constraint solving
  -> next state
```

repeated over time.

A differentiable simulator augments this with backward propagation:

```text
loss
  -> adjoint propagation
  -> gradients of states and parameters
```

The simulation becomes an optimization-compatible dynamical system.

### Why Differentiable Physics Matters

Many real problems require optimizing physical behavior rather than merely simulating it.

Examples include:

| Problem | Optimization Target |
|---|---|
| Robotics | Control policy |
| Animation | Motion trajectories |
| Material design | Elastic parameters |
| Inverse mechanics | Hidden forces |
| System identification | Physical constants |
| Reinforcement learning | Long-term reward |
| Scientific inference | Unknown governing parameters |
| Trajectory optimization | Minimal energy or time |

Without differentiability, optimization requires black-box search or reinforcement methods with weak gradient signals.

Differentiable simulation provides direct sensitivity information.

### Dynamical Systems

A continuous dynamical system is:

$$
\frac{dx}{dt} = g(x, u, \theta)
$$

Numerical integration produces discrete updates:

$$
x_{t+1} = x_t + \Delta t \cdot g(x_t, u_t, \theta)
$$

for explicit Euler integration.

Automatic differentiation computes:

$$
\frac{\partial L}{\partial \theta}
$$

through all timesteps.

This is structurally similar to recurrent neural networks and backpropagation through time.

### Simulation Rollout

A rollout is:

$$
x_0
\rightarrow
x_1
\rightarrow
x_2
\rightarrow
\cdots
\rightarrow
x_T
$$

The total loss may be:

$$
L = \sum_{t=0}^{T} \ell(x_t, u_t)
$$

Reverse mode computes adjoints backward:

$$
\lambda_t =
\frac{\partial L}{\partial x_t}
$$

using:

$$
\lambda_t =
\lambda_{t+1}
\frac{\partial x_{t+1}}{\partial x_t}
+
\frac{\partial \ell_t}{\partial x_t}
$$

This propagates future consequences backward through physical evolution.

### Example: Projectile Optimization

Suppose a projectile state is:

$$
x_t = (p_t, v_t)
$$

with updates:

$$
p_{t+1} = p_t + \Delta t \, v_t
$$

$$
v_{t+1} = v_t + \Delta t \, g
$$

A target loss:

$$
L = \|p_T - p^\star\|^2
$$

can be differentiated with respect to initial velocity:

$$
\frac{\partial L}{\partial v_0}
$$

allowing gradient-based optimization of launch conditions.

### System Identification

A differentiable simulator can infer unknown physical parameters.

Suppose:

$$
x_{t+1} = f(x_t; \theta)
$$

and observations are:

$$
y_t
$$

The objective:

$$
L =
\sum_t
\|x_t(\theta) - y_t\|^2
$$

optimizes:

- mass
- damping
- stiffness
- drag
- friction
- elasticity

This is inverse physics.

### Rigid Body Simulation

Rigid body systems evolve according to:

$$
M(q)\ddot{q} + C(q,\dot{q}) + G(q) = \tau
$$

where:

| Symbol | Meaning |
|---|---|
| $q$ | Generalized coordinates |
| $M$ | Mass matrix |
| $C$ | Coriolis and centrifugal forces |
| $G$ | Gravity |
| $\tau$ | Applied forces |

Differentiable rigid body simulation computes gradients through:

- forward dynamics
- integration
- constraint resolution
- contact forces

This enables optimization of robot motion and control.

### Contact and Collision

Collision handling is one of the hardest parts of differentiable physics.

Classical contact is discontinuous:

```text
if penetration:
    apply impulse
```

Tiny perturbations may abruptly create or remove contacts.

This causes unstable gradients.

Approaches include:

| Method | Strategy |
|---|---|
| Soft contact | Smooth penetration forces |
| Penalty methods | Continuous force approximation |
| Implicit differentiation | Differentiate solver equilibrium |
| Relaxed collision models | Approximate contact manifolds |
| Event-aware gradients | Handle discontinuities explicitly |

Contact dynamics remain a major research problem.

### Soft Body Simulation

Soft materials deform continuously.

A deformation field:

$$
u(x)
$$

describes displacement.

Energy-based models often define:

$$
E(u; \theta)
$$

The dynamics minimize or evolve according to this energy.

Differentiable soft body simulation computes gradients through:

- elastic deformation
- stress computation
- finite element assembly
- constitutive models

Applications include:

- cloth simulation
- biological tissue
- deformable robotics
- material optimization

### Fluid Simulation

Fluid systems satisfy equations such as Navier-Stokes:

$$
\frac{\partial v}{\partial t}
+
(v \cdot \nabla)v =
-\nabla p
+
\nu \nabla^2 v
+
f
$$

Differentiable fluid simulation computes sensitivities of flow behavior.

This supports:

| Application | Goal |
|---|---|
| Aerodynamic optimization | Minimize drag |
| Flow control | Stabilize turbulence |
| Weather inference | Estimate hidden parameters |
| Shape optimization | Improve flow geometry |
| Scientific calibration | Match observations |

Large-scale fluid differentiation is computationally expensive because state dimensionality is enormous.

### Differentiable Constraints

Many simulators solve constrained systems:

$$
g(x) = 0
$$

Examples:

- joints
- incompressibility
- nonpenetration
- conservation laws

Constraint solvers may involve iterative optimization.

Differentiation through solvers can occur via:

| Method | Idea |
|---|---|
| Unrolled differentiation | Differentiate every iteration |
| Implicit differentiation | Differentiate equilibrium conditions |
| Custom adjoints | Manually derived backward rules |

Implicit differentiation is often more memory efficient.

### Implicit Differentiation

Suppose the simulator solves:

$$
F(x^\star, \theta) = 0
$$

Differentiating gives:

$$
\frac{\partial F}{\partial x}
\frac{dx^\star}{d\theta}
+
\frac{\partial F}{\partial \theta} =
0
$$

Therefore:

$$
\frac{dx^\star}{d\theta} = -
\left(
\frac{\partial F}{\partial x}
\right)^{-1}
\frac{\partial F}{\partial \theta}
$$

This avoids storing all iterative solver states.

### Differentiable Controllers

A controller may generate forces:

$$
u_t = \pi(x_t; \phi)
$$

where $\phi$ are policy parameters.

The simulator defines dynamics:

$$
x_{t+1} = f(x_t, u_t)
$$

The full system becomes:

```text
state
  -> controller
  -> forces
  -> physics
  -> next state
```

Gradients flow through both controller and physics.

This enables trajectory optimization and model-based reinforcement learning.

### Model Predictive Control

Differentiable simulation is central in model predictive control.

The controller solves:

$$
\min_{u_{0:T}}
\sum_t \ell(x_t, u_t)
$$

subject to dynamics:

$$
x_{t+1} = f(x_t, u_t)
$$

Differentiability allows efficient optimization of control sequences using gradient methods.

### Differentiable Robotics

Modern robotics pipelines increasingly combine:

```text
policy network
  -> physics simulator
  -> trajectory
  -> reward loss
```

Gradients may optimize:

- motor torques
- gait parameters
- grasp trajectories
- balance policies
- actuator calibration

Differentiable simulation often improves sample efficiency relative to black-box reinforcement learning.

### Learned Physics

Some systems replace analytic physics with learned approximations:

$$
x_{t+1} = f_\theta(x_t, u_t)
$$

The learned model may represent:

- fluid dynamics
- rigid body interactions
- cloth deformation
- contact dynamics

These systems trade exact physical guarantees for differentiability and efficiency.

Hybrid systems are common:

```text
analytic physics
  + learned residual model
```

### Adjoint Methods

Large scientific simulators often use adjoint methods instead of general-purpose AD.

An adjoint system propagates sensitivities backward through time:

$$
\lambda_t =
\left(
\frac{\partial f}{\partial x_t}
\right)^T
\lambda_{t+1}
+
\frac{\partial \ell_t}{\partial x_t}
$$

Adjoint methods are mathematically equivalent to reverse-mode differentiation but optimized for PDE-scale computation.

### Numerical Stability

Differentiating long simulations introduces instability.

Common problems include:

| Problem | Cause |
|---|---|
| Exploding gradients | Chaotic dynamics |
| Vanishing gradients | Dissipative systems |
| Contact instability | Discontinuous forces |
| Integrator sensitivity | Numerical discretization |
| Solver divergence | Ill-conditioned constraints |
| Floating point drift | Long trajectory accumulation |

Chaotic systems may have gradients that become meaningless over long horizons.

### Memory Complexity

Reverse-mode differentiation stores intermediate states:

$$
x_0, x_1, \ldots, x_T
$$

Large simulations may require terabytes of memory.

Techniques include:

| Method | Purpose |
|---|---|
| Checkpointing | Recompute states during backward pass |
| Reversible integration | Recover earlier states |
| Implicit adjoints | Avoid full storage |
| Truncated gradients | Limit backward horizon |
| Mixed precision | Reduce memory cost |

Memory management is often the dominant systems problem.

### Parallel and GPU Simulation

Differentiable simulators increasingly target GPUs.

Challenges include:

- parallel contact handling
- differentiable sparse solvers
- dynamic topology
- irregular memory access
- synchronization costs

Modern simulators integrate tightly with tensor runtimes such as CUDA-based computation graphs.

### Hybrid Symbolic-Continuous Systems

Practical physics engines often combine:

| Component | Type |
|---|---|
| Collision detection | Symbolic |
| Constraint graph | Symbolic |
| Dynamics integration | Numerical |
| Material models | Continuous |
| Controllers | Learned |
| Contact relaxation | Approximate differentiable |

Fully smooth physics engines are rare. Most systems selectively introduce differentiability where optimization benefits justify the complexity.

### Failure Modes

Differentiable simulators fail in recognizable ways.

| Failure | Cause |
|---|---|
| Contact explosions | Unstable collision gradients |
| Energy drift | Integration error |
| Unrealistic optimization | Exploiting simulator artifacts |
| Gradient chaos | Sensitive dynamics |
| Solver nonconvergence | Ill-conditioned systems |
| Soft-contact artifacts | Unrealistic penetration |
| Overfitting to simulator | Reality gap |

Optimization can exploit weaknesses in the simulator rather than discovering physically meaningful behavior.

### Systems Architecture

A differentiable physics engine typically contains:

| Component | Purpose |
|---|---|
| State representation | Physical configuration |
| Integrator | Time evolution |
| Constraint solver | Physical validity |
| Contact system | Collision response |
| Tensor backend | GPU acceleration |
| Differentiation runtime | Backward propagation |
| Adjoint solver | Efficient sensitivity computation |
| Optimization loop | Parameter updates |

At scale, simulator design becomes inseparable from differentiation strategy.

### Relation to Automatic Differentiation

Differentiable physics extends automatic differentiation into dynamical systems.

The simulator becomes a recurrent differentiable program:

$$
x_{t+1} = f(x_t, u_t, \theta)
$$

Automatic differentiation propagates gradients through:

- integration
- constraints
- forces
- contacts
- material response
- control systems

The central challenge is not differentiating elementary arithmetic. The challenge is preserving meaningful gradients through long, nonlinear, often discontinuous physical evolution.

### Core Idea

A differentiable physics engine transforms simulation into an optimization-compatible process. Physical trajectories become differentiable computations whose parameters can be learned, inferred, or controlled through gradient-based methods.

This allows observations, rewards, or objectives defined at the end of a simulation to shape the physical parameters and control signals that generated the trajectory.