Mingli Song

Deep Model Transferability from Attribution Maps

2019 · arXiv (Cornell University)

Exploring the transferability between heterogeneous tasks sheds light on their intrinsic interconnections, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, …

Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

2021 · arXiv (Cornell University)

Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain. Prior KD approaches, despite their gratifying results, have largely relied on the premise that …