Signal Processing

Signal processing studies how information is represented, transformed, filtered, compressed, reconstructed, and estimated from signals. A signal may be a time series, an image, an audio waveform, a radar return, a sensor stream, or a multidimensional field.

Automatic differentiation is useful because many signal processing systems can be written as differentiable programs:

x \to T_\theta(x) \to y \to L(y)

where $x$ is an input signal, $T_\theta$ is a transform or filter with parameters $\theta$ , and $L$ is an objective. AD gives derivatives with respect to filter coefficients, model parameters, input samples, reconstruction variables, or calibration constants.

Linear Filtering

A discrete linear time-invariant filter has the form

y[n] = \sum_{k} h[k]x[n-k],

where $h$ is the impulse response.

This is convolution:

y = h * x.

If a loss depends on the filtered output,

L = \ell(y),

then AD computes derivatives with respect to both the signal and the filter:

\frac{\partial L}{\partial x}, \qquad \frac{\partial L}{\partial h}.

The reverse pass through convolution is another convolution-like operation. This is one reason convolutional neural networks fit naturally into AD systems: backpropagation through convolution is a standard signal processing operation.

Frequency-Domain Methods

Many signal processing algorithms use the Fourier transform.

X[k] = \sum_{n=0}^{N-1} x[n]e^{-2\pi i kn/N}.

The inverse transform reconstructs the signal:

x[n] = \frac{1}{N} \sum_{k=0}^{N-1} X[k]e^{2\pi i kn/N}.

Because the discrete Fourier transform is linear, its derivative is the transform itself. Reverse-mode differentiation through an FFT uses the corresponding adjoint transform.

This makes frequency-domain objectives easy to differentiate, for example:

L(x) = \|\, |\operatorname{FFT}(x)| - a \,\|^2.

Such objectives appear in phase retrieval, spectral matching, audio synthesis, diffraction imaging, and inverse scattering.

Complex-Valued Differentiation

Signal processing often uses complex numbers. AD systems must define how derivatives behave for complex-valued programs.

A complex signal can be treated as a pair of real signals:

z = x + iy.

For real-valued losses,

L : \mathbb{C}^n \to \mathbb{R},

gradients are usually interpreted with respect to the real and imaginary parts. This avoids assuming that every operation is holomorphic. Many common operations, such as magnitude,

|z| = \sqrt{x^2+y^2},

are not complex analytic, but they are differentiable as real functions except at singular points.

Practical AD systems for signal processing should specify whether gradients use real-imaginary derivatives or Wirtinger-style notation.

Adaptive Filters

An adaptive filter updates its coefficients from data. A simple finite impulse response filter computes

\hat y[n] = \sum_{k=0}^{K-1} w_k x[n-k].

Given target signal $d[n]$ , the error is

e[n]=d[n]-\hat y[n].

The least-squares objective is

L(w) = \frac{1}{2} \sum_n e[n]^2.

AD gives

\nabla_w L,

which can be used in gradient descent, stochastic gradient descent, or more specialized adaptive algorithms.

Traditional algorithms such as LMS can be viewed as hand-derived gradient methods. AD generalizes the same idea to complex filter structures where manual derivative derivation becomes tedious.

Inverse Problems in Signal Processing

Many signal processing tasks are inverse problems. We observe

y = A x + \epsilon,

and want to recover $x$ . Here $A$ may represent blur, downsampling, masking, compression, room acoustics, sensor geometry, or a Fourier sampling operator.

A common reconstruction objective is

L(x) = \frac{1}{2}\|Ax-y\|^2 + \lambda R(x),

where $R(x)$ is a prior or regularizer.

AD supports reconstruction by providing gradients with respect to $x$ . This allows the use of generic optimization methods even when $A$ , $R$ , or both are implemented as programs rather than matrices.

Examples include:

Task	Forward operator
Deblurring	Convolution with point-spread function
Super-resolution	Blur then downsample
Compressed sensing	Random projection or masked Fourier samples
MRI reconstruction	Fourier sampling on k-space
Audio dereverberation	Room impulse response convolution
Tomographic reconstruction	Projection operator

Differentiable Transforms

Many transforms are differentiable or piecewise differentiable:

Transform	AD treatment
FFT	Linear adjoint transform
DCT	Linear adjoint transform
Wavelet transform	Filter-bank derivatives
Short-time Fourier transform	Windowed linear transform
Mel filterbank	Matrix multiplication and nonlinear scaling
Cepstrum	FFT, log magnitude, inverse FFT
Convolution	Cross-correlation in reverse pass

This makes AD useful for end-to-end systems that combine classical signal processing with learned models.

For example, an audio model may compute:

waveform
    -> STFT
    -> magnitude
    -> mel filterbank
    -> log compression
    -> neural model
    -> loss

The whole pipeline can be differentiated, subject to care around zeros, logarithms, and magnitude operations.

Sparse and Structured Signals

Many signals have sparse structure. Compressed sensing uses the assumption that $x$ is sparse in some basis:

x = \Psi \alpha,

where most entries of $\alpha$ are zero or near zero.

A typical objective is

L(\alpha) = \frac{1}{2}\|A\Psi\alpha-y\|^2 + \lambda \|\alpha\|_1.

The $\ell_1$ norm is non-smooth at zero. AD can still compute subgradient-like values depending on implementation, but optimization requires care.

Smooth approximations are often used:

|\alpha| \approx \sqrt{\alpha^2+\epsilon}.

This gives stable gradients while preserving sparsity pressure.

Differentiable Sampling

Sampling and resampling are central in signal processing. For a continuous signal approximation $x(t)$ , resampling evaluates

y[n]=x(\tau_n),

where $\tau_n$ may depend on parameters.

If interpolation is differentiable, then AD can compute derivatives with respect to both samples and sampling locations.

This is useful in:

image registration,
time warping,
differentiable rendering,
beamforming,
sensor calibration,
spatial transformer networks.

Nearest-neighbor sampling is discontinuous with respect to coordinates. Linear, cubic, or spline interpolation gives more useful derivatives.

Beamforming and Array Processing

Array processing combines measurements from multiple sensors. A beamformer output may be

y(t) = \sum_m w_m x_m(t-\tau_m(\theta)),

where $w_m$ are weights and $\tau_m(\theta)$ are direction-dependent delays.

AD can compute derivatives with respect to:

Variable	Meaning
$w_m$	Beamforming weights
$\theta$	Source direction
Sensor positions	Array calibration
Signal samples	Input sensitivity

Differentiable beamforming appears in radar, sonar, microphone arrays, radio astronomy, and wireless communication.

State-Space Models and Kalman Filters

Signal processing often represents systems with state-space models:

x_{t+1}=A x_t + B u_t + \eta_t,

y_t=C x_t + \epsilon_t.

Kalman filtering estimates hidden states from noisy measurements. Its recursion is differentiable, provided matrix inversions and covariance updates are handled carefully.

AD can differentiate a Kalman filter with respect to:

transition matrices,
observation matrices,
noise covariance parameters,
initial state,
control parameters.

For long sequences, reverse-mode AD faces the same memory issue as recurrent neural networks. Checkpointing or custom smoother adjoints may be needed.

Phase Retrieval

Phase retrieval reconstructs a signal from magnitude-only measurements:

y = |Ax|^2.

The phase is missing, so the inverse problem is nonconvex.

A loss may be

L(x) = \frac{1}{2} \| |Ax|^2 - y \|^2.

AD computes gradients through the linear transform, magnitude, and squared magnitude. This allows gradient-based reconstruction methods, though initialization and nonconvexity remain major issues.

Learned Signal Processing

Modern systems often combine hand-designed transforms with learned components. Examples include:

learned denoisers,
neural vocoders,
differentiable codecs,
learned image reconstruction,
neural beamformers,
learned compression,
differentiable equalizers.

A useful design pattern is to keep classical signal-processing operators explicit and differentiable, then insert learned modules where the model lacks structure.

This gives a hybrid pipeline:

structured physical transform
    -> learned correction
    -> differentiable objective
    -> gradient-based training or inference

Numerical Issues

Signal processing pipelines contain operations that can produce unstable gradients.

Operation	Issue
$\log x$	Singular near zero
$	z
Phase angle	Discontinuous modulo $2\pi$
Hard thresholding	Zero or undefined gradients
Quantization	Discontinuous
Clipping	Saturated gradients
Sorting peaks	Non-smooth selection

Practical differentiable systems often replace hard operations with smooth approximations during training or optimization.

Examples include soft thresholding, smooth clipping, differentiable peak picking, and noise-aware objectives.

Custom Derivative Rules

Generic AD can differentiate most signal processing code, but custom rules are often better.

Component	Better derivative rule
FFT	Use known adjoint transform
Convolution	Use optimized correlation kernels
Linear solve	Use transpose solve
Interpolation	Define boundary derivative explicitly
Quantization	Use surrogate gradient if needed
STFT/ISTFT	Preserve window and overlap-add structure

Custom rules improve performance and make derivative semantics explicit.

Summary

Signal processing is a natural domain for automatic differentiation because many operators are linear, structured, and compositional. AD provides gradients through filters, transforms, reconstruction objectives, adaptive systems, beamformers, and state-space estimators.

The main difficulties come from complex-valued computations, non-smooth operations, sampling decisions, quantization, phase ambiguity, and long recurrent filters. Effective differentiable signal processing combines AD with known adjoint operators, stable numerical design, and carefully chosen smooth approximations.