Dmitry Nikolaev

Representation biases in sentence transformers

2023

Variants of the BERT architecture specialised for producing full-sentence representations often achieve better performance on downstream tasks than sentence embeddings extracted from vanilla BERT. However, there is still little understanding of what properties of inputs …

An Attribution Method for Siamese Encoders

2023 · arXiv (Cornell University)

Despite the success of Siamese encoder models such as sentence transformers (ST), little is known about the aspects of inputs they pay attention to. A barrier is that their predictions cannot be attributed to individual …