Cost-Quality Adaptive Active Learning for Chinese Clinical Named Entity Recognition

Clinical Named Entity Recognition (CNER) aims to automatically identity clinical terminologies in Electronic Health Records (EHRs), which is a fundamental and crucial step for clinical research. To train a high-performance model for CNER, it usually requires a large number of EHRs with high-quality labels. However, labeling EHRs, especially Chinese EHRs, is time-consuming and expensive. One effective solution to this issue is active learning, where a model asks labelers to annotate data which is beneficial to model performance improvement. Conventional active learning assumes a single labeler that always replies noiseless answers to queried labels. However, in real settings, multiple labelers provide diverse quality of annotation with varied cost and labelers with low overall annotation quality can still assign correct labels for some specific instances. In this paper, we propose a Cost-Quality Adaptive Active Learning (CQAAL) approach for CNER in Chinese EHRs, which maintains a balance between the annotation quality, labeling cost and the informativeness of selected instances. Specifically, our proposed CQAAL method selects cost-effective instance-labeler pairs to achieve better annotation quality with lower cost in an adaptive manner. Computational results on the CCKS2017 dataset demonstrate the superiority and effectiveness of CQAAL.