Nojun Kwak

Feature Fusion for Online Mutual Knowledge Distillation

2019 · arXiv (Cornell University)

We propose a learning framework named Feature Fusion Learning (FFL) that efficiently trains a powerful classifier through a fusion module which combines the feature maps generated from parallel neural networks. Specifically, we train a number …

On the Orthogonality of Knowledge Distillation with Other Techniques: From an Ensemble Perspective

2020 · arXiv (Cornell University)

To put a state-of-the-art neural network to practical use, it is necessary to design a model that has a good trade-off between the resource consumption and performance on the test set. Many researchers and engineers …