Ehsan Doostmohammadi

Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models

2023

Augmenting language models with a retrieval mechanism has been shown to significantly improve their performance while keeping the number of parameters low. Retrieval-augmented models commonly rely on a semantic retrieval mechanism based on the similarity …

Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency

2025 · arXiv (Cornell University)

Retrieval-augmented language models have demonstrated performance comparable to much larger models while requiring fewer computational resources. The effectiveness of these models crucially depends on the overlap between query and retrieved context, but the optimal degree …