Byeongho Heo

Masking meets Supervision: A Strong Learning Alliance

2023 · arXiv (Cornell University)

Pre-training with random masked inputs has emerged as a novel trend in self-supervised training. However, supervised learning still faces a challenge in adopting masking augmentations, primarily due to unstable training. In this paper, we propose …