Jiawen Xie

Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers

2024

Although dominant in natural language processing, transformer-based models still struggle with long-sequence processing, due to the computational costs of their self-attention operations, which increase exponentially as the length of the input sequence grows.To address this …