Carlos Guestrin

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

2019

Although current evaluation of questionanswering systems treats predictions in isolation, we need to consider the relationship between predictions to measure true understanding. A model should be penalized for answering "no" to "Is the rose red?" …