Oggi Rudovic

DiceHuBERT: Distilling HuBERT with a Self-Supervised Learning Objective

2025 · arXiv (Cornell University)

We introduce DiceHuBERT, a knowledge distillation framework for compressing HuBERT, a widely used self-supervised learning (SSL)-based speech foundation model. Unlike existing distillation methods that rely on layer-wise and feature-wise mapping between teacher and student models, …