Peter I. Frazier

Unbiased Comparative Evaluation of Ranking Functions

2016 · arXiv (Cornell University)

Eliciting relevance judgments for ranking evaluation is labor-intensive and costly, motivating careful selection of which documents to judge. Unlike traditional approaches that make this selection deterministically, probabilistic sampling has shown intriguing promise since it enables …