Philip C. Woodland

Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models

2021 · arXiv (Cornell University)

Self-supervised pre-training is an effective approach to leveraging a large amount of unlabelled data to reduce word error rates (WERs) of automatic speech recognition (ASR) systems. Since it is impractical to use large pre-trained models …