Brano Kusy

On learning denoisable student logits

2026 · Pattern Recognition

Knowledge Distillation (KD) aims to train a student model to mimic the behavior of a more powerful teacher model. In this paper, we reveal that through the lens of diffusion processes, student logits can be …