Ngoc-Trung Tran

Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?

2022 · arXiv (Cornell University)

This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). Contemporary findings addressing this thesis statement take dichotomous standpoints: Muller et al. (2019) and Shen et al. (2021b). Critically, there is no …