BART-based Hierarchical Attentional Network for Sentence Ordering

In this paper, we introduce a novel BART-based Hierarchical Attentional Ordering Network (BHAONet), aiming to address the coherence modeling challenge within paragraphs, which stands as a cornerstone in comprehension, generation, and reasoning tasks. By leveraging the pre-trained BART model to encode the entire sequence, we can effectively exploit global semantic and contextual information. Moreover, the token-level and sentence-level hierarchical attentional layers are incorporated to encourage the model to focus on features at various levels of granularity. In addition, a transformer-guided pointer network is developed for decoding. Extensive experiments conducted on benchmark datasets demonstrate the effectiveness and superiority of our proposed model.