Christian Hansen

Multi-Head Self-Attention with Role-Guided Masks

2020 · arXiv (Cornell University)

The state of the art in learning meaningful semantic representations of words is the Transformer model and its attention mechanisms. Simply put, the attention mechanisms learn to attend to specific parts of the input dispensing …