Haonan Zhang

Sticking to the Mean: Detecting Sticky Tokens in Text Embedding Models

2025 · arXiv (Cornell University)

Despite the widespread use of Transformer-based text embedding models in NLP tasks, surprising 'sticky tokens' can undermine the reliability of embeddings. These tokens, when repeatedly inserted into sentences, pull sentence similarity toward a certain value, …