Benjamnin Van Durme

Addressing the Binning Problem in Calibration Assessment through Scalar Annotations

2024 · Transactions of the Association for Computational Linguistics

Abstract Computational linguistics models commonly target the prediction of discrete—categorical—labels. When assessing how well-calibrated these model predictions are, popular evaluation schemes require practitioners to manually determine a binning scheme: grouping labels into bins to approximate …