Big Self-Supervised Models are Strong Semi-Supervised Learners Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton arXiv 2020 PDF, SSL By SeonghoonYu July 26th, 2021 Summary This paper achieves SOTA performance by combine the pre-trained model on self-supervised learning with knowledge distilation. Namely, They show that using pre-trained model on SSL as teacher model fo..
