Steven Cao

Multilingual Constituency Parsing with Self-Attention and Pre-Training

2019

We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions. We first compare the benefits of no pre-training, fastText We also find that pre-training is …

Multilingual Alignment of Contextual Word Representations

2020 · arXiv (Cornell University)

We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT. In particular, after our proposed alignment procedure, BERT exhibits significantly improved zero-shot performance …