Ki‐Jong Han

LittleBird: Efficient Faster & Longer Transformer for Question Answering

2022 · arXiv (Cornell University)

BERT has shown a lot of sucess in a wide variety of NLP tasks. But it has a limitation dealing with long inputs due to its attention mechanism. Longformer, ETC and BigBird addressed this issue …