Baosong Yang

Information Aggregation for Multi-Head Attention with Routing-by-Agreement

2019

Jian Li, Baosong Yang, Zi-Yi Dou, Xing Wang, Michael R. Lyu, Zhaopeng Tu. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long …

KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

2021 · arXiv (Cornell University)

Generative commonsense reasoning requires machines to generate sentences describing an everyday scenario given several concepts, which has attracted much attention recently. However, existing models cannot perform as well as humans, since sentences they produce are …

Multi-Head Attention with Disagreement Regularization

2018

Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions. In this work, we introduce a disagreement regularization to explicitly encourage the diversity among multiple attention …

Modeling Localness for Self-Attention Networks

2018

Self-attention networks have proven to be of profound value for its strength of capturing global dependencies. In this work, we propose to model localness for self-attention networks, which enhances the ability of capturing useful local …