Transfer bi-directional LSTM RNN for named entity recognition in Chinese electronic medical records

In this paper, a transfer bi-directional recurrent neural networks (RNN) is proposed for named entity recognition (NER) in Chinese electronic medical records (EMRs) that aims to extract medical knowledge such as phrases recording diseases and treatments automatically. We propose a two-step procedure where the first step is to train a shallow bi-directional RNN in the general domain, and the second step is to transfer knowledge from the general domain to train a deeper bi-directional RNN for recognizing medical concepts from Chinese EMRs. Specifically, this is achieved by initializing the shallow parts of the deeper network in the second step with parameter weights from the bi-directional RNN trained in the first step. Then the deeper networks are re-trained on the Chinese EMRs. Experimental results show that NER performances are improved by the transferred knowledge significantly.