딥러닝 공부방

Self-Labelling via Simultaneous Clustering and Representation Learning Yuki M. Asano, Christian Rupprecht, Andrea Vedaldi arxiv 2019 PDF, Self-Supervised Learning By SeonghoonYu July 19th, 2021 Summary 신경망이 출력한 feature vector를 clustering에 할당하는데, 이 할당하는 과정을 최적 운송(optimal transport) 문제로 보고 sinkhorn algorithm으로 assignment matrix Q를 계산합니다. Q는 feature vector와 clustering의 유사도를 계산하여 clustering을 할당하는 역할..

Mean teachers are better role models: Weight-averaged consistency targets imporve semi-supervised deep learning results Antti Tarvainen, Harri Valpola, arxiv 2017 PDF, Semi Supervised Learning By SeonghoonYu July 18th, 2021 Summary Previous best performance model of semi-supervised learning is Temporal Ensembling having a problem. Since each target is updated only once per epoch, the learned inf..

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset Joao Carreira, Andrew Zisserman, arXiv 2017 PDF, VD By SeonghoonYu July 17th, 2021 Summary They achive SOTA performence in video action recognition using two method. (1) Apply ImageNet pre-trained 2D Conv model to 3D Conv model for the video classification by repeating the weights of the 2D filters N times along the time dimensi..

Bootstrap Your Own Latent A New Approach to Self-Supervised Learning Jean-Bastien Grill, Florian Strub, Florent Altche, Corentin Tallec, Pierre H.Richemon, arXiv 2020 PDF, score [8/10], SSL By SeonghoonYu July 16th, 2021 Summary They suggest a new approch to self-supervised learning. (1) use two network referred to as online and target network and then update target network with a slow-moving av..

Understanding the Behaviour of Contrastive Loss Feng Wang, Huaping Liu, arxiv 2020 PDF, Self-Supervised Learning By SeonghoonYu July 15th, 2021 Summary There exists a uniformity-tolerance dilemma in unsupervised contrastive learning. and the temporature plays a key role in controlling the local separation and global uniformity of embedding distribution. So the choice of temperature is important ..