Nikita Kitaev

Constituency Parsing with a Self-Attentive Encoder

2018

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-ofthe-art discriminative constituency parser. The use of attention makes explicit the manner in which information is propagated between …

Multilingual Constituency Parsing with Self-Attention and Pre-Training

2019

We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions. We first compare the benefits of no pre-training, fastText We also find that pre-training is …

Multilingual Alignment of Contextual Word Representations

2020 · arXiv (Cornell University)

We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT. In particular, after our proposed alignment procedure, BERT exhibits significantly improved zero-shot performance …