Xi Weng

Modulate Your Spectrum in Self-Supervised Learning

2023 · arXiv (Cornell University)

Whitening loss offers a theoretical guarantee against feature collapse in self-supervised learning (SSL) with joint embedding architectures. Typically, it involves a hard whitening approach, transforming the embedding and applying loss to the whitened output. In …