딥러닝 공부방

Unsupervised Feature Learning via Non-Parametric Instance Discrimination Zhirong Wu, Yuanjun Xiong, Stella X.Yu, Dahua Lin, arXiv 2018 PDF, Self-Suervised Learning, By SeonghoonYu, July 22th, 2021 Summary The feature representations can be learned by discriminating among individual instances without any notion of semantic categories. We can find that Figure shows an image from class leopard is r..

Learning Deep Representations by Mutual Information Estimation and Maximization R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, arXiv 2018 PDF, SSL By SeonghoonYu July 21th, 2021 Summary This paper updates model's parameters by maximizing mutial information between immediate feature maps and flattened last feature maps obtained from ConvNet. To do this, they use Jensen-Shannon divergence(..

Deep Clustering for Unsupervised Learning of Visual Features Mathilde Caron, Piotr Bojanowski, Armand Joulin, Matthijs Douze, arXiv 2018 PDF, Self Supervised Learning By SeonghoonYu July 15th, 2021 Summary This paper is clustering based self-supervised learning in an offline fashion. This model jointly learns the parameters of a neural network and the cluster assignments of the resulting feature..

SlowFast Networks for Video Recognition Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming He, arXiv 2018 PDF, Video By SeonghoonYu July 20th, 2021 Summary They presents a two-pathway SlowFast model for video recognition. Two pathways seperately work at low and high temporal resolutions. (1) One is Slow pathway designed to capture sementic information that can be given by a few sparse f..

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron, Ishan Misra, Jullien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin arxiv 2020 PDF, Self-Supervised Learning By SeonghoonYu July 19th, 2021 Summary This paper propose an online clustering-based self-supervised method learning visual features in an online fashion without supervision Typical clusterin..

Self-Labelling via Simultaneous Clustering and Representation Learning Yuki M. Asano, Christian Rupprecht, Andrea Vedaldi arxiv 2019 PDF, Self-Supervised Learning By SeonghoonYu July 19th, 2021 Summary 신경망이 출력한 feature vector를 clustering에 할당하는데, 이 할당하는 과정을 최적 운송(optimal transport) 문제로 보고 sinkhorn algorithm으로 assignment matrix Q를 계산합니다. Q는 feature vector와 clustering의 유사도를 계산하여 clustering을 할당하는 역할..

Mean teachers are better role models: Weight-averaged consistency targets imporve semi-supervised deep learning results Antti Tarvainen, Harri Valpola, arxiv 2017 PDF, Semi Supervised Learning By SeonghoonYu July 18th, 2021 Summary Previous best performance model of semi-supervised learning is Temporal Ensembling having a problem. Since each target is updated only once per epoch, the learned inf..

Temporal Ensembling for Semi-Supervised Learning Samuli Laine, Timo Aila, arxiv 2016 PDF, Semi-Supervised Learning By SeonghoonYu July 18th, 2021 Summary They propose $\sqcap$-model and temporal Ensemling in a semi-supervised learning setting only a small portion of training data is labeled. During training, $\sqcap$-model evaluates each training input $x_i$, resulting in prediction vetors $z_i$..

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset Joao Carreira, Andrew Zisserman, arXiv 2017 PDF, VD By SeonghoonYu July 17th, 2021 Summary They achive SOTA performence in video action recognition using two method. (1) Apply ImageNet pre-trained 2D Conv model to 3D Conv model for the video classification by repeating the weights of the 2D filters N times along the time dimensi..

Bootstrap Your Own Latent A New Approach to Self-Supervised Learning Jean-Bastien Grill, Florian Strub, Florent Altche, Corentin Tallec, Pierre H.Richemon, arXiv 2020 PDF, score [8/10], SSL By SeonghoonYu July 16th, 2021 Summary They suggest a new approch to self-supervised learning. (1) use two network referred to as online and target network and then update target network with a slow-moving av..