Laurent Callot

Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation

2024 · arXiv (Cornell University)

We propose a new method to measure the task-specific accuracy of Retrieval-Augmented Large Language Models (RAG). Evaluation is performed by scoring the RAG on an automatically-generated synthetic exam composed of multiple choice questions based on …