Semi-Supervised Learning with Pseudo-Labeling

The need for semi-supervised learning stems from the cost of annotating datasets. Datasets do not always come with fully labeled data. Henceforth, we can cleverly perform compound training with labeled and unlabeled data. One of the semi-supervised methods we can employ is pseudo-labeling. At first, we train our model with labeled data and generate labels for unlabeled data based on it. Next, we train the pre-trained model with both labeled and unlabeled data. This is possible due to pseudo-labels generated earlier. In this project, DenseNet-121 is utilized. The model is trained on CIFAR-10 with 1000 labels.

Experiment

To cater to your curiosity, you can catch the implementation here.

Result

Quantitative Result

Refer to the table below to discern the quantitative results.

Test Metric	Score
Accuracy	62.77%
Loss	1.274

Accuracy and Loss Curves

Labeled Data

The loss curve on the labeled train and validation sets of the model.

The accuracy curve on the labeled train and validation sets of the model.

Labeled and Unlabeled Data

The loss curve on the labeled and unlabeled train and validation sets of the model.

The accuracy curve on the labeled and unlabeled train and validation sets of the model.

Qualitative Result

This 3 × 3 image grid shows the qualitative results on the test set.

The grid of qualitative results.

Credit

• The CIFAR-10 dataset
• Learning Multiple Layers of Features from Tiny Images
• Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks
• Densely Connected Convolutional Networks
• PyTorch Lightning