Potsawee Manakul

Sparsity and Sentence Structure in Encoder-Decoder Attention of Summarization Systems

2021 · arXiv (Cornell University)

Transformer models have achieved state-of-the-art results in a wide range of NLP tasks including summarization. Training and inference using large transformer models can be computationally expensive. Previous work has focused on one important bottleneck, the …