A Hybrid Model Based on Pre-trained Language Model and Graph Neural Networks for Scientific Paper Recommendation

Due to the overload of published scientific articles, paper recommendation has long been a critical research problem for automatically recommending high quality scientific papers for researchers. For scientific paper recommendation, the existing methods are mainly based on the user-paper interaction and the textual content of the paper. Although many scientific paper recommendation systems have been proposed, there are still two shortcomings: 1) the interaction information-based approaches cannot capture the higher-order user paper interaction information. 2) the content-based approaches cannot exploit the deep semantic information of the text, which impairs the interpretability of the models. To improve recommendation quality and model interpretability, we propose Citation Graph Collaborative Filtering (CGCF), which unifies document representation and Graph Neural Network (GNN). Our method consists of three steps:1) Build the user-paper bipartite graph based on citation relationship. 2) Use a pre-trained language model to initialize the paper’s embedding with its title and abstract. 3) Tune node embeddings using high-order connectivity information captured by several stacked message-passing layers on the graph built in the first step and finally make predictions predicted by these learned embeddings. Experiment shows that our model performs better while ensuring training efficiency.