Junshuang Wu

On Scalar Embedding of Relative Positions in Attention Models

2021 · Proceedings of the AAAI Conference on Artificial Intelligence

Attention with positional encoding has been demonstrated as a powerful component in modern neural network models, such as transformers. However, why positional encoding works well in attention models remains largely unanswered. In this paper, we …

Hierarchical Modeling of Label Dependency and Label Noise in Fine-grained Entity Typing

2021

Fine-grained entity typing (FET) aims to annotate the entity mentions in a sentence with fine-grained type labels. It brings plentiful semantic information for many natural language processing tasks. Existing FET approaches apply hard attention to …