Skip connections, also known as residual connections, are shortcuts that bypass one or more layers in a neural network. They address issues like vanishing gradients and enable training deeper networks. Introduced in ResNet, skip connections allow the model to learn identity mappings.
By directly adding input from an earlier layer to a later layer, skip connections preserve original information and make optimization easier. For example, in ResNet, the output of a convolutional block is added to its input, forming a residual connection.
Skip connections have proven essential in advancing architectures for computer vision, like DenseNet and Transformer-based models. They also improve convergence and generalization in tasks like image classification and natural language processing.