Walter Dempsey

Kernel Deformed Exponential Families for Sparse Continuous Attention

2021 · arXiv (Cornell University)

Attention mechanisms take an expectation of a data representation with respect to probability weights. This creates summary statistics that focus on important features. Recently, (Martins et al. 2020, 2021) proposed continuous attention mechanisms, focusing on …