Chenglei Si

SHUOWEN-JIEZI: Linguistically Informed Tokenizers For Chinese Language Model Pretraining

2021 · arXiv (Cornell University)

Conventional tokenization methods for Chinese pretrained language models (PLMs) treat each character as an indivisible token (Devlin et al., 2019), which ignores the characteristics of the Chinese writing system. In this work, we comprehensively study …

Re-Examining Calibration: The Case of Question Answering

2022 · arXiv (Cornell University)

For users to trust model predictions, they need to understand model outputs, particularly their confidence - calibration aims to adjust (calibrate) models' confidence to match expected accuracy. We argue that the traditional calibration evaluation does …