Enhancing Multiple-Choice Question Answering with External Knowledge and Lightweight Model Optimization

In recent years, many studies have attempted to integrate external knowledge with Large Language Models (LLMs) to improve their performance on multiple-choice question answering tasks. However, effectively incorporating external knowledge and extracting useful information from it remains a challenge. To address this issue, this paper proposes a novel framework that can effectively extract extensive external knowledge from both pre-trained LLMs and pertinent training datasets. Furthermore, it involves a distilled lightweight model capable of effectively refining and synthesizing knowledge obtained from various sources with minimal computational resources. Experiments on the CommonsenseQA and OpenbookQA datasets demonstrate that this framework can substantially enhance performance on the question answering task in comparison to baseline models.