Self-Supervised Learning
1. Self-Supervised Learning
There are two types of SSL: auxiliary pretext tasks and contrastive learning.
1.1. Auxiliary Pretext Tasks
Early methods of SSL primarily defined auxiliary pretext tasks as a way to learn representations using pseudo-labels, or labels that were created automatically based on the dataset's attributes. These were then used for tasks such as classification, detection, and segmentation, among others. Auxiliary pretext tasks can include predicting the rotation degree, filling in a missing part of an image colorizing a grayscale image, predicting the relative position of a patch, and more.
1.2. Contrastive Learning
We want similar images to be close in the embedding space. But we don't want trivial solutions like the identity function. Therefore we need to use negative examples and do the contrastive learning.