Yasheng Wang

PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model

2022 · arXiv (Cornell University)

In this paper, we introduce PanGu-Bot, a Chinese pre-trained open-domain dialogue generation model based on a large pre-trained language model (PLM) PANGU-alpha (Zeng et al.,2021). Different from other pre-trained dialogue models trained over a massive …

SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation

2021 · arXiv (Cornell University)

Code representation learning, which aims to encode the semantics of source code into distributed vectors, plays an important role in recent deep-learning-based models for code intelligence. Recently, many pre-trained language models for source code (e.g., …