Laura M. Ferrari

From Multimodal to Unimodal Attention in Transformers using Knowledge\n Distillation

2021 · arXiv (Cornell University)

Multimodal Deep Learning has garnered much interest, and transformers have\ntriggered novel approaches, thanks to the cross-attention mechanism. Here we\npropose an approach to deal with two key existing challenges: the high\ncomputational resource demanded and the issue …