Qi Ju

K-BERT: Enabling Language Representation with Knowledge Graph

2020 · Proceedings of the AAAI Conference on Artificial Intelligence

Pre-trained language representation models, such as BERT, capture a general language representation from large-scale corpora, but lack domain-specific knowledge. When reading a domain text, experts make inferences with relevant knowledge. For machines to achieve this …

FastBERT: a Self-distilling BERT with Adaptive Inference Time

2020

Pre-trained language models like BERT have proven to be highly performant. However, they are often computationally expensive in many practical scenarios, for such heavy models can hardly be readily implemented with limited resources. To improve …