Klemen Kotar

WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words

2023 · arXiv (Cornell University)

Training on multiple modalities of input can augment the capabilities of a language model. Here, we ask whether such a training regime can improve the quality and efficiency of these systems as well. We focus …