L1 and L2 RegularizationA neural network is trained by minimizing a loss function. For a supervised learning problem, this loss measures how far the model predictions are from the target values.
Early StoppingNeural networks are usually trained iteratively. An optimizer repeatedly updates model parameters to reduce the training loss.
DropoutDropout is a regularization method that randomly removes parts of a neural network during training.
Data AugmentationData augmentation is a regularization method that creates modified versions of training examples while preserving their labels.
Label SmoothingLabel smoothing is a regularization method for classification.
Stochastic DepthStochastic depth is a regularization method for deep residual networks.
Mixup and CutMixMixup and CutMix are data augmentation methods that create new training examples by combining two examples and their labels. They regularize the model by discouraging overly sharp decision boundaries.
Stochastic DepthStochastic depth regularizes deep residual networks by randomly skipping residual branches during training.