Musheng Chen

Spectral attention for transformers: frequency-domain filtering of attention maps

2026 · Journal of King Saud University - Computer and Information Sciences

Abstract This paper introduces spectral attention, which filters the attention score matrix directly in the frequency domain via FFT/IFFT with learnable, per-head masks. This complements the time-domain view by enabling explicit control over low-, mid-, …