Beyond Retrieval: Topic-based Alignment of Scientific Papers to Research Proposal

The inception of a research agenda typically commences with the creation of a comprehensive research proposal.The efficacy of the proposal often hinges on its ability to connect with the existing scientific literature that supports its ideas.To effectively assess the relevance of existing articles to a research proposal, it is imperative to categorize these articles into high-level thematic groups, referred to as topics, that align with the proposal.This paper introduces a novel task of aligning scientific articles, relevant to a proposal, with researcherprovided proposal topics.Additionally, we construct a dataset to serve as a benchmark for this task.We establish human and Large Language Model (LLM) baselines and propose a novel three-stage approach to address this challenge.We synthesize and use pseudolabels that map proposal topics to text spans from cited articles to train Language Models (LMs) for two purposes: (i) as a retriever, to extract relevant text spans from cited articles for each topic, and (ii) as a classifier, to categorize the articles into the proposal topics.Our retriever-classifier pipeline, which employs very small open-source LMs fine-tuned with our constructed dataset, achieves results comparable to a vanilla paid LLM-based classifier, demonstrating its efficacy.However, a notable gap of 23.57F1 score between our approach and the human baseline highlights the complexity of this task and emphasizes the need for further research.