He, Di

Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder

2021 · arXiv (Cornell University)

Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space. Autoencoder-based language models are appealing in dense retrieval as they train the encoder to output high-quality embedding that can reconstruct …