Panpan Zou

Multi scale Feature Extraction and Fusion for Online Knowledge Distillation

2022 · arXiv (Cornell University)

Online knowledge distillation conducts knowledge transfer among all student models to alleviate the reliance on pre-trained models. However, existing online methods rely heavily on the prediction distributions and neglect the further exploration of the representational …