Jungeun Kim

ReffAKD: Resource-efficient Autoencoder-based Knowledge Distillation

2024 · arXiv (Cornell University)

In this research, we propose an innovative method to boost Knowledge Distillation efficiency without the need for resource-heavy teacher models. Knowledge Distillation trains a smaller ``student'' model with guidance from a larger ``teacher'' model, which …