Differentiable Physics Engines

A differentiable physics engine computes gradients of physical simulation outputs with respect to inputs, parameters, or control signals. Instead of treating simulation as a terminal forward process, the simulator becomes part of the computational graph.

The system computes:

x_{t+1} = f(x_t, u_t, \theta)

where:

Symbol	Meaning
$x_t$	System state
$u_t$	Control input
$\theta$	Physical parameters
$f$	Physics transition operator

A loss over trajectories:

L(x_0, x_1, \ldots, x_T)

can then be differentiated with respect to:

initial conditions
controller parameters
forces
geometry
masses
friction coefficients
material constants

Automatic differentiation propagates gradients backward through the simulation itself.

Physics Simulation as Computation

A classical simulator performs:

initial state
  -> timestep integration
  -> collision handling
  -> constraint solving
  -> next state

repeated over time.

A differentiable simulator augments this with backward propagation:

loss
  -> adjoint propagation
  -> gradients of states and parameters

The simulation becomes an optimization-compatible dynamical system.

Why Differentiable Physics Matters

Many real problems require optimizing physical behavior rather than merely simulating it.

Examples include:

Problem	Optimization Target
Robotics	Control policy
Animation	Motion trajectories
Material design	Elastic parameters
Inverse mechanics	Hidden forces
System identification	Physical constants
Reinforcement learning	Long-term reward
Scientific inference	Unknown governing parameters
Trajectory optimization	Minimal energy or time

Without differentiability, optimization requires black-box search or reinforcement methods with weak gradient signals.

Differentiable simulation provides direct sensitivity information.

Dynamical Systems

A continuous dynamical system is:

\frac{dx}{dt} = g(x, u, \theta)

Numerical integration produces discrete updates:

x_{t+1} = x_t + \Delta t \cdot g(x_t, u_t, \theta)

for explicit Euler integration.

Automatic differentiation computes:

\frac{\partial L}{\partial \theta}

through all timesteps.

This is structurally similar to recurrent neural networks and backpropagation through time.

Simulation Rollout

A rollout is:

x_0 \rightarrow x_1 \rightarrow x_2 \rightarrow \cdots \rightarrow x_T

The total loss may be:

L = \sum_{t=0}^{T} \ell(x_t, u_t)

Reverse mode computes adjoints backward:

\lambda_t = \frac{\partial L}{\partial x_t}

using:

\lambda_t = \lambda_{t+1} \frac{\partial x_{t+1}}{\partial x_t} + \frac{\partial \ell_t}{\partial x_t}

This propagates future consequences backward through physical evolution.

Example: Projectile Optimization

Suppose a projectile state is:

x_t = (p_t, v_t)

with updates:

p_{t+1} = p_t + \Delta t \, v_t

v_{t+1} = v_t + \Delta t \, g

A target loss:

L = \|p_T - p^\star\|^2

can be differentiated with respect to initial velocity:

\frac{\partial L}{\partial v_0}

allowing gradient-based optimization of launch conditions.

System Identification

A differentiable simulator can infer unknown physical parameters.

Suppose:

x_{t+1} = f(x_t; \theta)

and observations are:

y_t

The objective:

L = \sum_t \|x_t(\theta) - y_t\|^2

optimizes:

mass
damping
stiffness
drag
friction
elasticity

This is inverse physics.

Rigid Body Simulation

Rigid body systems evolve according to:

M(q)\ddot{q} + C(q,\dot{q}) + G(q) = \tau

where:

Symbol	Meaning
$q$	Generalized coordinates
$M$	Mass matrix
$C$	Coriolis and centrifugal forces
$G$	Gravity
$\tau$	Applied forces

Differentiable rigid body simulation computes gradients through:

forward dynamics
integration
constraint resolution
contact forces

This enables optimization of robot motion and control.

Contact and Collision

Collision handling is one of the hardest parts of differentiable physics.

Classical contact is discontinuous:

if penetration:
    apply impulse

Tiny perturbations may abruptly create or remove contacts.

This causes unstable gradients.

Approaches include:

Method	Strategy
Soft contact	Smooth penetration forces
Penalty methods	Continuous force approximation
Implicit differentiation	Differentiate solver equilibrium
Relaxed collision models	Approximate contact manifolds
Event-aware gradients	Handle discontinuities explicitly

Contact dynamics remain a major research problem.

Soft Body Simulation

Soft materials deform continuously.

A deformation field:

u(x)

describes displacement.

Energy-based models often define:

E(u; \theta)

The dynamics minimize or evolve according to this energy.

Differentiable soft body simulation computes gradients through:

elastic deformation
stress computation
finite element assembly
constitutive models

Applications include:

cloth simulation
biological tissue
deformable robotics
material optimization

Fluid Simulation

Fluid systems satisfy equations such as Navier-Stokes:

\frac{\partial v}{\partial t} + (v \cdot \nabla)v = -\nabla p + \nu \nabla^2 v + f

Differentiable fluid simulation computes sensitivities of flow behavior.

This supports:

Application	Goal
Aerodynamic optimization	Minimize drag
Flow control	Stabilize turbulence
Weather inference	Estimate hidden parameters
Shape optimization	Improve flow geometry
Scientific calibration	Match observations

Large-scale fluid differentiation is computationally expensive because state dimensionality is enormous.

Differentiable Constraints

Many simulators solve constrained systems:

g(x) = 0

Examples:

joints
incompressibility
nonpenetration
conservation laws

Constraint solvers may involve iterative optimization.

Differentiation through solvers can occur via:

Method	Idea
Unrolled differentiation	Differentiate every iteration
Implicit differentiation	Differentiate equilibrium conditions
Custom adjoints	Manually derived backward rules

Implicit differentiation is often more memory efficient.

Implicit Differentiation

Suppose the simulator solves:

F(x^\star, \theta) = 0

Differentiating gives:

\frac{\partial F}{\partial x} \frac{dx^\star}{d\theta} + \frac{\partial F}{\partial \theta} = 0

Therefore:

\frac{dx^\star}{d\theta} = - \left( \frac{\partial F}{\partial x} \right)^{-1} \frac{\partial F}{\partial \theta}

This avoids storing all iterative solver states.

Differentiable Controllers

A controller may generate forces:

u_t = \pi(x_t; \phi)

where $\phi$ are policy parameters.

The simulator defines dynamics:

x_{t+1} = f(x_t, u_t)

The full system becomes:

state
  -> controller
  -> forces
  -> physics
  -> next state

Gradients flow through both controller and physics.

This enables trajectory optimization and model-based reinforcement learning.

Model Predictive Control

Differentiable simulation is central in model predictive control.

The controller solves:

\min_{u_{0:T}} \sum_t \ell(x_t, u_t)

subject to dynamics:

x_{t+1} = f(x_t, u_t)

Differentiability allows efficient optimization of control sequences using gradient methods.

Differentiable Robotics

Modern robotics pipelines increasingly combine:

policy network
  -> physics simulator
  -> trajectory
  -> reward loss

Gradients may optimize:

motor torques
gait parameters
grasp trajectories
balance policies
actuator calibration

Differentiable simulation often improves sample efficiency relative to black-box reinforcement learning.

Learned Physics

Some systems replace analytic physics with learned approximations:

x_{t+1} = f_\theta(x_t, u_t)

The learned model may represent:

fluid dynamics
rigid body interactions
cloth deformation
contact dynamics

These systems trade exact physical guarantees for differentiability and efficiency.

Hybrid systems are common:

analytic physics
  + learned residual model

Adjoint Methods

Large scientific simulators often use adjoint methods instead of general-purpose AD.

An adjoint system propagates sensitivities backward through time:

\lambda_t = \left( \frac{\partial f}{\partial x_t} \right)^T \lambda_{t+1} + \frac{\partial \ell_t}{\partial x_t}

Adjoint methods are mathematically equivalent to reverse-mode differentiation but optimized for PDE-scale computation.

Numerical Stability

Differentiating long simulations introduces instability.

Common problems include:

Problem	Cause
Exploding gradients	Chaotic dynamics
Vanishing gradients	Dissipative systems
Contact instability	Discontinuous forces
Integrator sensitivity	Numerical discretization
Solver divergence	Ill-conditioned constraints
Floating point drift	Long trajectory accumulation

Chaotic systems may have gradients that become meaningless over long horizons.

Memory Complexity

Reverse-mode differentiation stores intermediate states:

x_0, x_1, \ldots, x_T

Large simulations may require terabytes of memory.

Techniques include:

Method	Purpose
Checkpointing	Recompute states during backward pass
Reversible integration	Recover earlier states
Implicit adjoints	Avoid full storage
Truncated gradients	Limit backward horizon
Mixed precision	Reduce memory cost

Memory management is often the dominant systems problem.

Parallel and GPU Simulation

Differentiable simulators increasingly target GPUs.

Challenges include:

parallel contact handling
differentiable sparse solvers
dynamic topology
irregular memory access
synchronization costs

Modern simulators integrate tightly with tensor runtimes such as CUDA-based computation graphs.

Hybrid Symbolic-Continuous Systems

Practical physics engines often combine:

Component	Type
Collision detection	Symbolic
Constraint graph	Symbolic
Dynamics integration	Numerical
Material models	Continuous
Controllers	Learned
Contact relaxation	Approximate differentiable

Fully smooth physics engines are rare. Most systems selectively introduce differentiability where optimization benefits justify the complexity.

Failure Modes

Differentiable simulators fail in recognizable ways.

Failure	Cause
Contact explosions	Unstable collision gradients
Energy drift	Integration error
Unrealistic optimization	Exploiting simulator artifacts
Gradient chaos	Sensitive dynamics
Solver nonconvergence	Ill-conditioned systems
Soft-contact artifacts	Unrealistic penetration
Overfitting to simulator	Reality gap

Optimization can exploit weaknesses in the simulator rather than discovering physically meaningful behavior.

Systems Architecture

A differentiable physics engine typically contains:

Component	Purpose
State representation	Physical configuration
Integrator	Time evolution
Constraint solver	Physical validity
Contact system	Collision response
Tensor backend	GPU acceleration
Differentiation runtime	Backward propagation
Adjoint solver	Efficient sensitivity computation
Optimization loop	Parameter updates

At scale, simulator design becomes inseparable from differentiation strategy.

Relation to Automatic Differentiation

Differentiable physics extends automatic differentiation into dynamical systems.

The simulator becomes a recurrent differentiable program:

x_{t+1} = f(x_t, u_t, \theta)

Automatic differentiation propagates gradients through:

integration
constraints
forces
contacts
material response
control systems

The central challenge is not differentiating elementary arithmetic. The challenge is preserving meaningful gradients through long, nonlinear, often discontinuous physical evolution.

Core Idea

A differentiable physics engine transforms simulation into an optimization-compatible process. Physical trajectories become differentiable computations whose parameters can be learned, inferred, or controlled through gradient-based methods.

This allows observations, rewards, or objectives defined at the end of a simulation to shape the physical parameters and control signals that generated the trajectory.